Introduced 2 new global variables:
* mysql-stats_time_backend_query (default true)
* mysql-stats_time_query_processor (default true)
For backward compatibility, they are both enabled by default
Automatically reset mysql->insert_id when a connection is sent to connection pool.
If is not reset to 0 , when a backend connections is assigned to a client, this one can potentially read the last inserted id from the previous client.
Enforced max_writers
Enforced writer_is_also_reader
Increased the number of entries in mysql_server_group_replication_log to 100 per node
Lowered the minimum value for mysql-monitor_groupreplication_healthcheck_interval to 50ms
Lowered the minimum value for mysql-monitor_groupreplication_healthcheck_timeout to 50ms
CentOS 5 doesn't have support for epoll_create1(), so I've defined a
macro to fall back on epoll_create() in that case.
Note that the `1` argument to epoll_create() is actually ignored, and is
only there to conform to the requirement that it be larger than 0.
The 2 new variables are:
* mysql-query_digests_max_digest_length : defines the maximum length of digest_text as reported in stats_mysql_query_digest
* mysql-query_digests_max_query_length : defines the maximum query length processed when computing query's digest and digext_text
* reduce build time by 25%;
* remove circular dependency on `cpp.h` and other headers.
* fixes issue with `my_global.h` from mariadbclient which
redefines `__attribute__`, that leads to broken STL and GCC
intrinsics if `<algorithm>` wasn't included prior including this header;
* fixes another issue with redefined `__attribute__` which leads to
`_conn_exchange_t` not being aligned to cache line size;
* removes `pthread_setstacksize` calls as `my_global.h` redefines `pthread_setstacksize`
to be `pthread_dummy(0)` which led to stack size being never adjusted;
* add missing include guards to some headers;
* remove unused proxysql_hash.h header.
This commit is able to speed up LOAD MYSQL SERVERS TO RUNTIME up to 10x for very large mysql_servers table with tens of thousands of servers.
Main improvements:
* removed 1 unnecessary call to `generate_mysql_servers_table()`
* replaced 1 call to `generate_mysql_servers_table()` with UPDATE statements when necessary
* implemented new function `servers_add()` that replaces `server_add()` to perform bulk operation
* replaced calls to `sqlite3_exec()` with proper parameters bindings
* use of multirows INSERT statements (32 rows at the time)
Miror improvements:
* added new variable mysql-hostgroup_manager_verbose , 1 by default for backward compatibility and debugging
* in not debug mode, access to HGM db has no shared cache
* in not debug mode, tables in HGM have checks disabled
Further note:
* Once issue #797 is completed, `mysql_servers` in MyHGM can be simplified further
Increased mysql-monitor_ping_timeout from 100 to 1000 milliseconds.
Also increased monitor_read_only_timeout from 100 to 800 milliseconds as read_only timeout is even more important than ping timeout.
Metrics introduced:
* ConnPool_get_conn_failure : connection pool cannot provide any connection
* ConnPool_get_conn_immediate : connection is provided from per-thread cache
* ConnPool_get_conn_success : the session is able to get a connection, either from per-thread cache or connection pool
MySQL_STMTs_meta() is aware to which session it belongs
entries are removed from MySQL_STMTs_meta() when unused
find_prepared_statement_by_hash() increases the client ref count
Hardcoded a maximum number of statements
The called of MySQL_STMTs_local functions doesn't must specify if it is a client or a backend.
Added also a function in MySQL_Connection to make the connection aware it is a client.
When query rules are processed, even if flagOUT is set rules are processed
in numerical order.
This new variable allows to jump back (reset rule_id to 0) N amount of times.
Using this variable is possible to implement loops.
Futhermore, a loop is possible with only one rule setting flagIN=flagOUT .
The default is 0. The maximum value is 1000000 .
This variable limits the amount of data that the client can send to the backend.
Note that ProxySQL won't really limit the amount of data that the client can send to the proxy, but the amount of data that the prox can send to the backend.
This commits also fixes issue #526Closes#526
Issue #576
Added some inline timers
Reduced main loop sleep from 1 sec to 0.5 sec
Admin module is signaled to quit ASAP
MySQL Threads are signaled to quit ASAP
Add new variable MySQL_Session::mirror (false by default)
Add new struct MySQL_Session::mirrorPkt (false by default)
These functions return immediately if this is a mirror session:
* MySQL_Protocol::generate_pkt_EOF()
* MySQL_Protocol::generate_pkt_ERR()
* MySQL_Protocol::generate_pkt_OK()
* MySQL_Protocol::generate_pkt_column_count()
* MySQL_Protocol::generate_pkt_field()
* MySQL_Protocol::generate_pkt_row3()
MySQL_ResultSet::MySQL_ResultSet() exits almost immediately if this is a mirror session
Most of the code of these functions is not executed if this is a mirror session:
* MySQL_ResultSet::add_eof()
* MySQL_ResultSet::get_resultset()
In MySQL_Session::writeout() :
* `client_myds->write_to_net_poll()` is called only if this is not a mirror session
In MySQL_Session::handler():
* if query is a `SELECT` (hardcoded for now) :
** create a new session and mark it as `mirror=true`
** duplicate the query and send it to the new session
* if `mirror==true` :
** pretend to read a new query from c`lient_myds`
** proceed normally (with the only exception that no data can be sent to a client)
In MySQL_Thread::process_all_sessions() :
* if `mirror==true` and `status==WAITING_CLIENT_DATA` it means the session is "completed" so it get destroyed
Limitations:
* it currently doesn't support large packets
* doesn't support different charset
* doesn't support transactions
Related to bug https://mariadb.atlassian.net/browse/MDEV-8338 .
Created new variable: mysql-bug_mdev_8338
If this variable is set, multiplexing is disabled if a query has NOW() or CURDATE()
The follow states are disabled in Admin module is the backend:
- CHANGING_USER_SERVER
- CHANGING_SCHEMA
- CHANGING_CHARSET
- CHANGING_AUTOCOMMIT
- mysql-monitor_username is always added as a possible user : this allows MySQL_Monitor to monitor the backends
- admin-admin_credentials and admin-stats_credentials are always added as possible users
- mysql-monitor_username , admin-admin_credentials and stats-admin_credentials do not shows in mysql_users table (by design)
- the above users have a max_connections limit of 1000 (hardcoded)
In order to understand the performance improvement related to how ProxySQL filters autocommit/commit/rollback, the follows variables were added in GLOBAL MYSQL STATUS
- Com_autocommit
- Com_autocommit_filtered
- Com_commit
- Com_commit_filtered
- Com_rollback
- Com_rollback_filtered
Bug fixes:
Fixed a bug introduced by 414588e
Removed a setting in MySQL_Data_Stream::return_MySQL_Connection_To_Pool()
Performance improvement for large resultset:
Reverted and improved changes in 4a5bb59
monitor_replication_lag_interval
monitor_replication_lag_timeout
Drafting MySQL_Monitor::monitor_replication_lag() as a clone of MySQL_Monitor::monitor_ping()
Status variables implemented so far:
- Client_Connections_aborted
- Client_Connections_connected
- Client_Connections_created
- Questions
- Slow_queries
Also created alias command SHOW MYSQL STATUS
- added global variables mysql-query_digests
- initialization if either mysql_thread___commands_stats or mysql_thread___query_digests are enabled
- query_parser_free() checks if QueryParserArgs was initialized
- Query_Processor::query_parser_init() checks the command type only if mysql_thread___commands_stats is true
- Query_Processor::query_parser_init() run mysql_query_digest() only if mysql_thread___query_digests is true
- Query_Processor::query_parser_update_counters() updates query digests only if digest_text is not null
- start time is always update (this is relevant for issue #327)
SET NAMES sent from the application only change the charset in the client connection.
When a backend connection is taken from the connection pool and charset mismatches, SET NAMES is sent to the backend
signal_all_threads() now accepts an argument (default 0). If the argument is greater than 0, the threads release the thread_mutex, and sleep for the specified millisecond .
In this way processlist will immediately access all threads
If the number of free connections is greater than free_connections_pct percentage of max_connections for that specific host,
all connections idle for longer than ping_interval_server are closed instead of sending PING
- added variables mysql_thread___connect_retries_delay and mysql_thread___sessions_sort
- sessions that cannot get a connections are sorted
- commands_stats now defaults to true
To handle connect timeout:
- added MySQL_Data_Stream::destroy_MySQL_Connection()
- new async status: ASYNC_CONNECT_TIMEOUT
- "select @@version_comment limit 1" is answered by proxysql and not forwarded to any backend
- added MySQL_Data_Stream variables connect_retries_on_failure and max_connect_time
- added global variables mysql-connect_timeout_server_max and mysql-connect_retries_on_failure
Improvement: proxysql returns more info in COM_STATISTICS: clients connected and uptime
Simplified the ping within MySQL_Thread
SQLite3_row::add_fields() to handle empty string
prot_status was not initialized in MySQL_Protocol
Session's default schema was always mysql_thread___default_schema , now fixed with correct value
default_schema is now read from config file
Dependencies:
- added libevent
- added mariadb-connector-c for non-blocking API
- added patch for mariadb-connector-c ( see https://mariadb.atlassian.net/browse/CONC-131 )
MySQL Monitoring:
- added table mysql_server_connect_log
- added table mysql_server_ping_log
- added access to admindb
- implemented monitoring using MariaDB client library non-blocking API and libevent, using MySQL_Monitor_State_Data to track state machine (experimental, and may change a lot in the near future)
- implemented MySQL_Monitor_Connection_Pool , a new connection pool only for MySQL Monitoring
- in MySQL_Thread added variable monitor_history to define the length of logs
- in MySQL_Thread added variables monitor_username and monitor_password to define monitoring credentials
- in MySQL_Thread added variable monitor_timer_cached to define if monitoring timers can be cached
- not completed feature: MySQL_Thread added variables monitor_query_variables, monitor_query_status, monitor_query_interval and monitor_query_timeout to define how global variables and global status are collected
Admin:
- added access to monitordb, to query MySQL_Monitor tables