Not only the default value is change from 1000 to 1 , but it also perform an upgrade from earlier release.
If during bootstrap a value of 1000 is found, it is automatically changed to 1 and a warning is generated.
This release also prevents the value 1000 to be a valid one during bootstrap.
This means that it is still possible to set the value to 1000 and to have a running proxysql instance with value 1000 , but during bootstrap it will be switched to 1.
If you really want to set mysql-session_idle_ms close to 1000 , it is recommended to set it to a closer value like 999 or 1001
Add initial support for FR limiting the number of connection errors that
can be initiated from a particular address before deniying future
connections from that address. Feature is controlled by new introduced
variables:
- mysql-client_host_error_counts
- mysql-client_host_cache_size
Added test that verifies that client connections are dropped because of:
- mysql-wait_timeout
- mysql-max_transaction_time
IT also helped finding a bug in how mysql-max_transaction_time was handled
1. Introduced new global variable: 'monitor_groupreplication_max_transaction_behind_for_read_only',
that modifies the behavior of 'group_replication_lag'.
2. Improved logic making use of 'MyHGC_find' instead of directly
searching 'MyHostGroups' structure.
3. Improved 'group_replication_lag' documentation with new
implementation updates.
4. Introduced changes to 'update_group_replication_set_writer'
preserving writters placed in 'OFFLINE_SOFT' state.
- Labelling has been simplified to group metrics which
are part of a common group.
- Help has been fixed to reflect this labelling change.
- Doxygen doc and comment marks has been added to signal
metrics grouping.
This commit also add support for including more gauge metrics based on
'MySQL_Thread_status_variables', using the same approach previously used
for the 'counter' metrics.
- Metrics now follow prometheus base unit convention.
- Many metrics now make use of labels instead of enconding
the info in the metric name.
- Almost all metrics now have a meaningful 'help' message.
Former max_transaction_time is now renamed to max_transaction_idle_time.
mysql-max_transaction_idle_time defines the maximum time a transaction can stay idle.
mysql-max_transaction_time defines the maximum time of a transaction since its start.
Renamed variable mysql-monitor_replication_lag_retries to mysql-monitor_replication_lag_count to match other variables naming convention.
As it is a counter, it minimum value is changed to 1.
Maximum value sets to 10.
Use of thread storage variable mysql_thread___monitor_replication_lag_count instead of GloMTH->variables.monitor_replication_lag_retries to avoid dirty reads .
`MySQL_Threads_Handler::listener_del` is not aware of `perthrsocks` array
used in `MySQL_Threads_Handler::listener_add`, because of this, isn't
able to properly track which descriptors are being used in each thread.
This patch makes the threads themselves responsible for closing the
descriptors that they are using for listeners.
Changing the mapping logic in SESSION_TRACK_GTIDS in frontend and backend connections.
Up to now, frontend and backend `SESSION_TRACK_GTIDSs match.
This is now changed:
* backend connections are by default set to `mysql-default_session_track_gtids`
* if `mysql-default_session_track_gtids=OFF` (the default) , `session_track_gtids` is not changed on backend
* if the client asks for `session_track_gtids=OFF` , proxysql ignores it (it just acknowledge it)
* if the client asks for `session_track_gtids=OWN_GTID` , proxysql will apply it
* if the client asks for `session_track_gtids=ALL_GTIDS` , proxysql will switch to OWN_GTID and generate a warning
* if the backend doesn't support `session_track_gtids` (for example in MySQL 5.5 and MySQL 5.6), proxysql won't apply it. It knows checking server capabilities
This commit also deprecates function `MySQL_Session::handler_again___verify_backend__generic_variable()`
ProxySQL is now also able to know when proxysql binlog reader is perhaps in use tracking all gtid_port in mysql_servers.
This feature in future will be used to deprecate `mysql-default_session_track_gtids`
Extended `mysql_variable_st` to add a new variable `is_global_variable`.
Using this variable we define the global variables.
Modified `mysql_tracked_variables[]` to also specify `is_global_variable`.
Removed `default_charset` from `MySQL_Threads_Handler::variables` to use `mysql_tracked_variables[]` instead.
Removed `default_charset` also from `mysql_thread_variables_names[]`.
Fixed `MySQL_Threads_Handler::get_variable_string()` : it was *not* returning `mysql-default_` variables.
`MySQL_Threads_Handler::set_variable()` adds `mysql-default_` variables only if `is_global_variable` is set.
In `MySQL_Threads_Handler::set_variable()` replaced a `strcasecmp()` with `strcmp()` for `mysql-default_` variables.
`MySQL_Threads_Handler::get_variables_list()` returns only `mysql-default_` variables with `is_global_variable` enabled.
`MySQL_Threads_Handler::has_variable()` for `mysql-default_` variables returns true only if variable has `is_global_variable` enabled.
`MySQL_Thread::refresh_variables()` does not refresh `mysql-default_` variables if `is_global_variable` is not enabled.
`MySQL_Thread::MySQL_Thread()` initializes to NULL all `mysql_thread___default_variables[]` variables.
Completely disabled in `ProxySQL_Admin::flush_mysql_variables___database_to_runtime()` the logic applied to all charset variables.
In the same methos was implemented the logic described in #1785 :
`mysql-default_charset` and `mysql-default_collation_connection` must be compatible:
* if only one is present, the other is configured automatically
* if not compatible, `mysql-default_charset` is configured from `mysql-default_collation_connection`
- GCC 4.8 and 5 have issues with uniform initialization syntax, for
this reason, all it's uses have been replaced with assignements.
- Per-module metrics initialization functions are no longer required,
they have been replaced with the new generic ones.
* removed 14 statuses
* configured the MySQL_Session status to SETTING_VARIABLE for 14 session variables
* extended MySQL_Session to introduce a new variable that tracks the session variable to change
* MySQL_Threads_Handler::SQL3_Processlist() can handle all variables
* removed unnecessary loops
* improved performance removing unnecessary function calls
Added new array mysql_tracked_variables[] that defines the tracked variables.
Ideally, tracking a new session variable will requires adding only a couple of lines of code.
Removed session variables from mysql_thread_variables_names:
- now both MySQL_Threads_Handler::get_variables_list() and MySQL_Threads_Handler::has_variable() rely on mysql_tracked_variables[]
Simplified MySQL_Variables, as it now also uses mysql_tracked_variables[] .
Simplify parser in MySQL_Session
Generalized code to handle two variables together:
- sql_auto_is_null
- sql_safe_updates
Migrating more session variables to new algorithm
- collation_connection
- net_write_timeout
- max_join_size
Migrating more session variables to new algorithm
- collation_connection
- net_write_timeout
- max_join_size
Added new array mysql_tracked_variables[] that defines the tracked variables.
Ideally, tracking a new session variable will requires adding only a couple of lines of code.
Removed session variables from mysql_thread_variables_names:
- now both MySQL_Threads_Handler::get_variables_list() and MySQL_Threads_Handler::has_variable() rely on mysql_tracked_variables[]
Simplified MySQL_Variables, as it now also uses mysql_tracked_variables[] .
Simplify parser in MySQL_Session
Generalized code to handle two variables together:
- sql_auto_is_null
- sql_safe_updates
Migrating more session variables to new algorithm
- collation_connection
- net_write_timeout
- max_join_size
Migrating more session variables to new algorithm
- collation_connection
- net_write_timeout
- max_join_size
libsqlinjection generates a lot of false positives.
This commit introduces a new table: mysql_firewall_whitelist_sqli_fingerprints
This table can list fingerprints generated by libsqlinjection:
if the fingerprint is listed in this table, proxysql will consider it as
a false positive.
This commit also enables SQLi algorithm only if the query is not already
explicitly whitelisted.
Library libjection is being used.
New variable mysql-automatic_detect_sqli defines if the feature is enabled or not.
If an SQL injection is detected:
- the client connection is immediately terminated
- an entry in the error log is reported
* introduced 2 new variables:
- mysql-firewall_whitelist_enabled : defines if the firewall with whitelist algorithm is enabled or not
- firewall_whitelist_errormsg : error message returned to the client if an error message is not already set
* Query_Processor returns firewall_whitelist_mode to the calling MySQL_Session, but the logging on error log currently happens within the Query_Processor itself
* if mysql-firewall_whitelist_enabled is turned on and a user it is not found in mysql_firewall_whitelist_users , the default behavior is that query is rejected
* lookup on mysql_firewall_whitelist_users happens for username@ip (specific IP) and username@'' (all IPs) . Match on network subnet is currently not implemented
* implemented SAVE MYSQL FIREWALL FROM RUNTIME
* fixed implementation of SAVE MYSQL FIREWALL TO DISK
* runtime configuration can be viewed in runtime_mysql_firewall_whitelist_rules and runtime_mysql_firewall_whitelist_users
This commit fixes the following bug:
if a client connection uses SSL and sends a query larger than 32KB, the query is never executed and the connection hang
Initial support for `SET SESSION TRANSACTION READ ONLY` or `READ WRITE`.
Extended `SET` parser to support also `SET SESSION TRANSACTION`.
Hostgroup Manager doesn't kill backend connections in case of error 1231.
`autocommit` is set at session level but also on MySQL client connection.
Added several debugging entries.
Several `handler_again___verify_backend_*` functions are disabled if `locked_on_hostgroup` is enabled.
Added 2 new status variables:
- queries_with_max_lag_ms__delayed
- queries_with_max_lag_ms__total_wait_time_us
Do not get replication lag from replicas if the value is 0
Fixed an error in the computation of max_lag_ms
This should fix a lot of issues related to failed parsing of SET statement.
This and the two 2 previous commits introduce several status variables, and a
new configuration variable: mysql-set_query_lock_on_hostgroup
Possible values for mysql-set_query_lock_on_hostgroup:
- 0 : legacy behavior , before 2.0.5
- 1 : (default) . SET statements that cannot be parsed correctly disable
both multiplexing AND routing. Attempting to route traffic while a
connection is linked to a specific backend connection will trigger
an error to be returned to the client
Issue #2120 : Send SESSION_TRACK_GTIDS to client
Issue #2121 : Track CLIENT_FOUND_ROWS required by the client
Issue #2125 : Track CLIENT_MULTI_STATEMENTS required by the client
Enhancements:
- added metrics rows_affected and rows_sent
- added global variable mysql-eventslog_default_log : if 1 , logging is enabled for every query unless explicitly disabled in mysql_query_rules.log . Default is 0
- added global variable mysql-eventslog_format : default is 1 (legacy format). A value of 2 enables logging in JSON format. Issue #871
Changing value at runtime causes the current file to be closed and a new one created
- fixed logging for prepared statements: till 2.0.5 only some percentage of prepared statements was correctly logged
Extended tables stats_mysql_query_digest and stats_mysql_query_digest_reset to also include sum_rows_affected and sum_rows_sent
Extended `eventslog_reader_sample.cpp` to support the new enhancements
MySQL_HostGroups_Manager::get_MyConn_from_pool() inserts connection
to mysrvc->ConnectionUsed, and this connections is also inserted
to My_Thread->cached_connections which leads to double deallocation.
Fixed https://github.com/sysown/proxysql/issues/1985.
After execute "PROXYSQL STOP", main() -> ProxySQL_Main_init_phase4___shutdown()
is called, then main() -> ProxySQL_Main_init_phase2___not_started() is called, this
function creates GloMTH and waits on GloVars.global.start_mutex, thus
ProxySQL_Main_init_phase3___start_all() -> ProxySQL_Main_init_MySQL_Threads_Handler_module()
isn't called.
Before "PROXYSQL START" is executed, admin module will crash on query to stats_mysql_processlist.
Similar cause to https://github.com/sysown/proxysql/issues/2037 and
https://github.com/sysown/proxysql/issues/2065.
Added new command `PROXYSQL INTERNAL SESSION` that clients can execute to
receive internal information about their own connection in JSON format.
Added JSON library.
Recompiled SQLite3 to support JSON.
Added new column `extended_info` in `stats_mysql_processlist`.
Added new mysql variable `mysql-show_processlist_extended` that determine the
content of `stats_mysql_processlist.extended_info`:
- 0 : no info
- 1 : JSON format
- 2 : JSON format with pretty printing
aws_aurora_replicas_skipped_during_query is a status variable for better
monitoring behavior due to replication lag in AWS Aurora
Also fixed lag computing in connection pool
Epoll thread does not poll for EPOLLOUT. If a throttled session with
pending data out is moved to epoll thread, the session will stay in
epoll thread indefinitely, causing a hang observed by client. Fix by not
moving throttled session to epoll thread.
Fix#1939
Temporary disable multiplexing when last_insert_id is returned in OK packet.
Multiplexing is disabled for mysql-auto_increment_delay_multiplex queries.
mysql-auto_increment_delay_multiplex ranges from 0 to 1000000 .
Default value is 5
When shunning a node, evaluate the different between
`mysql-monitor_ping_interval` and `mysql-shun_recovery_time_sec`, and postpone
in the future when the server needs to be recovered
- if clients uses mysql_native_password, if LDAP is enabled and if the user doesn't exist, switch to mysql_clear_password.
- if neither mysql_native_password or mysql_clear_password are used by the client:
- if LDAP is not enabled, always switch to mysql_native_password
- if LDAP is enabled:
- if the user exists, switch to mysql_native_password
- if the user doesn't exists, switch to mysql_clear_password
Added MySQL variable mysql-add_ldap_user_comment to determine if a comment with the original username needs to be added in the queries.
This commit also tracks the charset during the first handshake response.
Both KILL QUERY and KILL CONNECTION work
The only security check enforced is that the user sending the KILL
is the same user of the connection/query being killed.
The same applies also for:
- runtime_mysql_replication_hostgroups
- runtime_mysql_group_replication_hostgroups
- runtime_mysql_galera_hostgroups
Close#1435 and #1436
On a commodity hardware, performance improved from 25us to 7us
Performance can be improved more if replacing legacy algorithm of searching variables by name.
Although the legacy algorithm allow to add plugins in future.
Added 10 new metrics
Added 3 new global variables
* monitor_threads_min : minimum number of threads
* monitor_threads_max : maximum number of threads
* monitor_threads_queue_maxsize : maximum numbr of pending checks before starting new threads
Variable reset_connection_algorithm could either be:
1 = algorithm used up too version 1.4
2 = algorithm new since ProxySQL 2.0 (now default)
When reset_connection_algorithm = 2 , MySQL_Thread itself tries to reset connections instead of relying on connections purger HGCU_thread_run()