These improvements avoid a race condition in which a prepared statement is removed from the cache before it is marked as used by the client.
This race condition was noticed when running benchmark creating 7000 _unique_ prepared statements per second.
Metrics introduced:
* ConnPool_get_conn_failure : connection pool cannot provide any connection
* ConnPool_get_conn_immediate : connection is provided from per-thread cache
* ConnPool_get_conn_success : the session is able to get a connection, either from per-thread cache or connection pool
- default_hostgroup was not set: this causes multiplexing to always be disabled
- fd in client_myds for mirror session was not initialized: this causes random crashes
- default_hostgroup was not set: this causes multiplexing to always be disabled
- fd in client_myds for mirror session was not initialized: this causes random crashes
MySQL_STMTs_meta() is aware to which session it belongs
entries are removed from MySQL_STMTs_meta() when unused
find_prepared_statement_by_hash() increases the client ref count
Hardcoded a maximum number of statements
The called of MySQL_STMTs_local functions doesn't must specify if it is a client or a backend.
Added also a function in MySQL_Connection to make the connection aware it is a client.
The current version of ProxySQL filters SHOW WARNINGS returning
an empty resultset.
This implementation is logically incorrect, and in future version a better
implementation will be developed
MySQL_STMT_Manager now stores 2 reference counters per statement: one for client and one for server
Statement client side are removed with STMT_CLOSE command is processed
Bytes sends and received are accounted globally and per server
These are 2 memory leaks related to prepared statements
* Resultset (MySQL_ResultSet() not deallocated
* mysql_stmt_free_result() is now correctly called
So far, single thread sysbench is working.
Multi-threaded and resume from broken connections is still not working.
It means it still needs a lot of changes.
A new Query_Processor_Output (qpo) is allocated for every query.
Allocation is now done once per session, thus speeding up execution time
under memory profiler
Issue #527
Limit the amount of data received by a backend during a single poll() iteration.
Right now the default is mysql_thread___threshold_resultset_size*2, and it is arbitary.
Unrelated to #527, MySQL_Connection::async_ping() was modified to return a different value in case of timeout
jemalloc configured with purge:decay_time
Temporary disabled jemalloc to test a heap profiler (heaptrack)
MySQL_ResultSet::buffer_to_PSarrayOut() calls realloc() if the buffer preallocated is too big
MySQL_Session::writeout() sends as much data as possible to the client
This variable limits the amount of data that the client can send to the backend.
Note that ProxySQL won't really limit the amount of data that the client can send to the proxy, but the amount of data that the prox can send to the backend.
This commits also fixes issue #526Closes#526
When a server isn't responding to ping, it is flagged as shunned and all the connections need to be dropped.
All the connections must be dropped before the server is brought back online
Some legacy code is not currently used:
- removed reference to MySQL_Data_Stream::move_from_OUT_to_OUTpending()
- removed reference to MySQL_Data_Stream::PSarrayOUTpending
Expanded table `mysql_query_rules` to add `mirror_hostgroup`
Expanded struct `_Query_Processor_rule_t` and class `Query_Processor_Output` to support `mirror_hostgroup`
Added new variable `mirror_hostgroup` in `MySQL_Session`
The following logic ensure that a mirror_hostgroup set when processing the query in the original session is preserved when reprocessing the query in the mirror session:
* when a new mirror session is *created* `mirror_hostgroup` is set according to the result of Query_Processor
* when a mirror session is *executed* , `mirror_hostgroup` is copied into `default_hostgroup`
Current limitations:
* queries larger than 15MB aren't mirrorred
* upgrade from 1.1 is still not possible it will wipe table mysql_query_rules
Add new variable MySQL_Session::mirror (false by default)
Add new struct MySQL_Session::mirrorPkt (false by default)
These functions return immediately if this is a mirror session:
* MySQL_Protocol::generate_pkt_EOF()
* MySQL_Protocol::generate_pkt_ERR()
* MySQL_Protocol::generate_pkt_OK()
* MySQL_Protocol::generate_pkt_column_count()
* MySQL_Protocol::generate_pkt_field()
* MySQL_Protocol::generate_pkt_row3()
MySQL_ResultSet::MySQL_ResultSet() exits almost immediately if this is a mirror session
Most of the code of these functions is not executed if this is a mirror session:
* MySQL_ResultSet::add_eof()
* MySQL_ResultSet::get_resultset()
In MySQL_Session::writeout() :
* `client_myds->write_to_net_poll()` is called only if this is not a mirror session
In MySQL_Session::handler():
* if query is a `SELECT` (hardcoded for now) :
** create a new session and mark it as `mirror=true`
** duplicate the query and send it to the new session
* if `mirror==true` :
** pretend to read a new query from c`lient_myds`
** proceed normally (with the only exception that no data can be sent to a client)
In MySQL_Thread::process_all_sessions() :
* if `mirror==true` and `status==WAITING_CLIENT_DATA` it means the session is "completed" so it get destroyed
Limitations:
* it currently doesn't support large packets
* doesn't support different charset
* doesn't support transactions
Moved some code from MySQL_Session:handler_special_queries() into two new functions:
- MySQL_Session::handler_CommitRollback()
- MySQL_Session::handler_SetAutocommit()
These two new functions are called nowhere: de facto they are disabled
The follow states are disabled in Admin module is the backend:
- CHANGING_USER_SERVER
- CHANGING_SCHEMA
- CHANGING_CHARSET
- CHANGING_AUTOCOMMIT
- mysql-monitor_username is always added as a possible user : this allows MySQL_Monitor to monitor the backends
- admin-admin_credentials and admin-stats_credentials are always added as possible users
- mysql-monitor_username , admin-admin_credentials and stats-admin_credentials do not shows in mysql_users table (by design)
- the above users have a max_connections limit of 1000 (hardcoded)
In order to understand the performance improvement related to how ProxySQL filters autocommit/commit/rollback, the follows variables were added in GLOBAL MYSQL STATUS
- Com_autocommit
- Com_autocommit_filtered
- Com_commit
- Com_commit_filtered
- Com_rollback
- Com_rollback_filtered
- Re-enabled MySQL_Logger
- handling special case in which mysql-commands_stats and mysql-query_digests are disabled
- specifies the length of the query
- Query_Info is always initialized
Summary:
removed KV_Btree_Array.cpp
removed query_SQL from MySQL_Data_Stream
added some documentation for QC_entry_t
Query_Cache::get() and Query_Cache::set() now requires also the length of the key
Query_Cache::get() and Query_Cache::set() now process hk (the hash key) using the length of the key
Query Cache enhancement on user/schema
Query Cache now supports distinct entries based on user/schema .
This is achieved passing to Query_Cache::set() and Query_Cache::get() the hash that identifies user and schema
Test Plan: None
Reviewers: rene
Differential Revision: http://phab.sysown.com/D3
removed KV_Btree_Array.cpp
removed query_SQL from MySQL_Data_Stream
added some documentation for QC_entry_t
Query_Cache::get() and Query_Cache::set() now requires also the length of the key
Query_Cache::get() and Query_Cache::set() now process hk (the hash key) using the length of the key
- MySQL_Session::handler_special_queries() processes set autocommit from clients
- to the client is always returned the autocommit value set in the session
- set autocommit is either just acknowledged to the client, or forwarded to the backend based on the status of transactions
- unless there are query rules to handle SET AUTOCOMMIT , the statement is forwarded to the first found active transaction: this assumes there is just one transactions. Although multiple transactions are possible, handling multiple transactions at the same time seems a dangerous adventure
Added MySQL_Session::RequestEnd() to wrap CurrentQuery.end() and other state changes : this to minimize bugs
MySQL_Session::RequestEnd() sets query end time instead of Query_Info::end()
MySQL_Session::RequestEnd() is called also in case of special queries handled by handler_special_queries()
Now ProxySQL is able to detect error 1290 (The MySQL server is running with the --read-only option so it cannot execute this statement) and tries to re-execute the query.
This will make traffic switch more graceful
Status variables implemented so far:
- Client_Connections_aborted
- Client_Connections_connected
- Client_Connections_created
- Questions
- Slow_queries
Also created alias command SHOW MYSQL STATUS
- added global variables mysql-query_digests
- initialization if either mysql_thread___commands_stats or mysql_thread___query_digests are enabled
- query_parser_free() checks if QueryParserArgs was initialized
- Query_Processor::query_parser_init() checks the command type only if mysql_thread___commands_stats is true
- Query_Processor::query_parser_init() run mysql_query_digest() only if mysql_thread___query_digests is true
- Query_Processor::query_parser_update_counters() updates query digests only if digest_text is not null
- start time is always update (this is relevant for issue #327)
SET NAMES sent from the application only change the charset in the client connection.
When a backend connection is taken from the connection pool and charset mismatches, SET NAMES is sent to the backend
- added variables mysql_thread___connect_retries_delay and mysql_thread___sessions_sort
- sessions that cannot get a connections are sorted
- commands_stats now defaults to true
To handle connect timeout:
- added MySQL_Data_Stream::destroy_MySQL_Connection()
- new async status: ASYNC_CONNECT_TIMEOUT
- "select @@version_comment limit 1" is answered by proxysql and not forwarded to any backend
- added MySQL_Data_Stream variables connect_retries_on_failure and max_connect_time
- added global variables mysql-connect_timeout_server_max and mysql-connect_retries_on_failure
Improvement: proxysql returns more info in COM_STATISTICS: clients connected and uptime
Simplified the ping within MySQL_Thread
SQLite3_row::add_fields() to handle empty string
prot_status was not initialized in MySQL_Protocol
Session's default schema was always mysql_thread___default_schema , now fixed with correct value
default_schema is now read from config file
Added functions:
MySQL_Protocol::generate_COM_QUERY()
MySQL_Connection::set_charset()
MySQL_Session::handler___status_CHANGING_CHARSET()
MySQL_Session::handler___client_DSS_QUERY_SENT___send_SET_NAMES_to_backend()
Added CHANGING_CHARSET in enum session_status, and handled in MySQL_Session
Still TODO:
track and handle client’s change of charset
Implemented functions:
- MySQL_HostGroups_Manager::dump_table_mysql_servers()
- MySQL_HostGroups_Manager::generate_mysql_servers_table()
- Standard_ProxySQL_Admin::save_mysql_servers_runtime_to_database() (issue #17)
Improved Standard_ProxySQL_Admin::load_mysql_servers_to_runtime() to process also status
Improved MySQL_HostGroups_Manager::commit() to modified MySrvCs
Database in MyHGM is accessible via Admin (issue #212)
Enforced debugging if specified at the command line (issue #213)
Class Standard_MySQL_Thread was restructured
mysql_thread___ thread variables are moved out to global variables
Standard_Proxy_Admin calls Standard_MySQL_Thread::refresh_variables after initializing the thread structure
For better debugging, added functions:
MySQL_Data_Stream::clean_net_failure()
MySQL_Data_Stream::set_net_failure()
MySQL_Data_Stream::setDSS_STATE_QUERY_SENT_NET()
MySQL_Session::set_unhealthy()
Added condition mybe->server_myds->PSarrayOUTpending->len==0 to determine if DSS can be STATE_QUERY_SENT_NET
Developed class StatCounters to keep time series counters
Added 2 new instances of StatCounters:
- MySQL_Session::command_counters;
- ProxySQL_Poll::loop_counters;
Counters are dispayed in "dbg thread status"
Defined new variables mysql-connect_timeout_server_error and mysql-connect_timeout_server (issue #202 and #203) .
They are defined but not implemented yet.
Fixed two crashing bug when "gdb command" was trying to access:
- its own Query_Info if this one was not initialized
- Query_Info of another thread if this one was not initialized
Introduced new variable ProxySQL_Poll:poll_timeout that Session can overwrite to apply a timeout different than mysql_thread___poll_timeout
Added variable MySQL_Connection::last_time_used , updated at creation time and when calling push_MyConn_to_pool().
for each Connection add a "last_time_used"
Added function get_multiple_idle_connections()
Implementing Standard_MySQL_Threads_Handler::connection_manager_thread() , but doesn't seems to be a good idea
====
New attempt for connection handling
MySQL_Session has a new bool variable : connections_handler
Standard_MySQL_Thread has a new variable PtrArray : mysql_sessions_connections_handler
Added new function Standard_MySQL_Thread::register_session_connection_handler
SESSIONS_FOR_CONNECTIONS_HANDLER sessions are created in Standard_MySQL_Thread::init()
Added new function Standard_MySQL_Thread::process_all_sessions_connections_handler()
New variables in Standard_MySQL_Thread:
MySQL_Connection **my_idle_conns;
bool processing_idles;
unsigned long long last_processing_idles;
Added DSS STATE_PING_SENT
MySQL_Session::handler___status_WAITING_SERVER_DATA___STATE_PING_SENT
Removed MySQL_Connection *myconn from MySQL Backend
Removed server_myds and server_myds from MySQL_Session
Fixed a bug that caused an infinite loop in case the backed was not able to create new connection
In order to facilitate the reuse of code in Standard_ProxySQL_Admin.cpp :
- moved remove_spaces() from Standard_ProxySQL_Admin.cpp to gen_utils.cpp
- moved Standard_ProxySQL_Admin::SQLite3_to_MySQL() to MySQL_Session::SQLite3_to_MySQL()
Moved few macros from Standard_Query_Processor.cpp to proxysql_macros.h
Added functions:
- MySQL_Session::SQL3_Session_status()
- MySQL_Session::handler___status_WAITING_CLIENT_DATA___STATE_SLEEP___MYSQL_COM_QUERY_debug()
- Standard_MySQL_Thread::SQL3_Thread_status()
- MySQL_Threads_Handler::SQL3_Threads_status() (not completed)
Modified class Query_Info to copy queries : this can be a small performance penalty, but can improve functionalities
Added mysql commands:
- DBG SESSION STATUS
- DBG THREAD STATUS
- DBG THREADS STATUS
Added variable MySQL_Thread::thread_mutex , a mutex to protect the thread: this will be used to implement access to MySQL Thread via Admin
MySQL_Thread now have a new per thread variable updated via Admin Interface :
- mysql_thread___session_debug
ProxySQL_Poll now counts number of poll() in variable "loops"
If mysql_thread___session_debug is enabled and a query starts with "dbg " it is interpreted as a command for proxysql.
In Standard_ProxySQL_Admin added a variable admin_mutex to protect the update of statistics.
Fixed minor bugs.
For issue #187:
Introduced the follow functions:
- MySQL_Session::handler___status_CHANGING_USER_SERVER()
- MySQL_Session::handler___client_DSS_QUERY_SENT___send_CHANGE_USER_to_backend
- MySQL_Protocol::generate_COM_CHANGE_USER()
Modified MySQL_Session::handler___status_CHANGING_SCHEMA() to return a boolean to handle failures
Connections where INIT_DB or COM_CHANGE_USER fails are not reusable
Added session_statuses CHANGING_USER_SERVER and CHANGING_USER_CLIENT (currently not used, but needed for issue #187)
For issue #188:
Added variable MySQL_Connection_userinfo::hash : this is compared into MySQL_Session
Added function MySQL_Connection_userinfo::compute_hash() to automatically compute MySQL_Connection_userinfo::hash when a variable is changed
hash is compute with SpookyHash::Hash64()
Fixed bug #189:
removed an parameters incorrectly passed to fprintf
Fixed crashing bug in Admin module when ProxySQL is started with --no-start
When a client connection is terminated, the backend connection is returned to connection pool on if not in a transaction