- Labelling has been simplified to group metrics which
are part of a common group.
- Help has been fixed to reflect this labelling change.
- Doxygen doc and comment marks has been added to signal
metrics grouping.
- Metrics now follow prometheus base unit convention.
- Many metrics now make use of labels instead of enconding
the info in the metric name.
- Almost all metrics now have a meaningful 'help' message.
Renamed variable mysql-monitor_replication_lag_retries to mysql-monitor_replication_lag_count to match other variables naming convention.
As it is a counter, it minimum value is changed to 1.
Maximum value sets to 10.
Use of thread storage variable mysql_thread___monitor_replication_lag_count instead of GloMTH->variables.monitor_replication_lag_retries to avoid dirty reads .
Issue #531 added "a desperate attempt to bring back online any shunned server" when no servers are available; however, no indication or notification of this is logged. Add a log message indicating that the hostgroup has no servers available and shunned hosts are being checked before the normal shun recovery time.
Changing the mapping logic in SESSION_TRACK_GTIDS in frontend and backend connections.
Up to now, frontend and backend `SESSION_TRACK_GTIDSs match.
This is now changed:
* backend connections are by default set to `mysql-default_session_track_gtids`
* if `mysql-default_session_track_gtids=OFF` (the default) , `session_track_gtids` is not changed on backend
* if the client asks for `session_track_gtids=OFF` , proxysql ignores it (it just acknowledge it)
* if the client asks for `session_track_gtids=OWN_GTID` , proxysql will apply it
* if the client asks for `session_track_gtids=ALL_GTIDS` , proxysql will switch to OWN_GTID and generate a warning
* if the backend doesn't support `session_track_gtids` (for example in MySQL 5.5 and MySQL 5.6), proxysql won't apply it. It knows checking server capabilities
This commit also deprecates function `MySQL_Session::handler_again___verify_backend__generic_variable()`
ProxySQL is now also able to know when proxysql binlog reader is perhaps in use tracking all gtid_port in mysql_servers.
This feature in future will be used to deprecate `mysql-default_session_track_gtids`
This commit prepares proxysql for further development related to SQLite3.
Specifically, it allows to either use built-in SQLite3 library, or to load it from a plugin.
- Added new helper function to simplify ''prometheus::counters' update
and removed code duplication using it.
- 'hostgroup_manager' now exposes the same method 'p_update_metrics' as
the other modules.
- GCC 4.8 and 5 have issues with uniform initialization syntax, for
this reason, all it's uses have been replaced with assignements.
- Per-module metrics initialization functions are no longer required,
they have been replaced with the new generic ones.
libsqlinjection generates a lot of false positives.
This commit introduces a new table: mysql_firewall_whitelist_sqli_fingerprints
This table can list fingerprints generated by libsqlinjection:
if the fingerprint is listed in this table, proxysql will consider it as
a false positive.
This commit also enables SQLi algorithm only if the query is not already
explicitly whitelisted.
Initial support for `SET SESSION TRANSACTION READ ONLY` or `READ WRITE`.
Extended `SET` parser to support also `SET SESSION TRANSACTION`.
Hostgroup Manager doesn't kill backend connections in case of error 1231.
`autocommit` is set at session level but also on MySQL client connection.
Added several debugging entries.
Several `handler_again___verify_backend_*` functions are disabled if `locked_on_hostgroup` is enabled.
Issue #2120 : Send SESSION_TRACK_GTIDS to client
Issue #2121 : Track CLIENT_FOUND_ROWS required by the client
Issue #2125 : Track CLIENT_MULTI_STATEMENTS required by the client
In `stats_mysql_free_connections`, exported:
* address of connection
* address of mysql struct
* mysql thread_id
In `stats_mysql_processlist`, exported:
* address of session
* address of data stream
* address of connection
* address of mysql struct
* mysql thread_id
* removed password from non-debug built
- generate_mysql_servers_table() is now less verbose
- update_galera_set_writer() is protected by a mutex
- update_galera_set_writer() set status to online
- fixed wording in monitor_galera_thread()
- corrected the computing of consecutive timeout in monitor_galera_thread()
aws_aurora_replicas_skipped_during_query is a status variable for better
monitoring behavior due to replication lag in AWS Aurora
Also fixed lag computing in connection pool
When shunning a node, evaluate the different between
`mysql-monitor_ping_interval` and `mysql-shun_recovery_time_sec`, and postpone
in the future when the server needs to be recovered
This functionality is enabled for:
- mysql_galera_hostgroups
- mysql_group_replication_hostgroups
If writer_is_also_reader=2 and there are servers in
backup_writer_hostgroup, only these servers will be used
in reader_hostgroup
Galera checks may first detect the status of a node and then decide if an action is needed.
This commit reduces the amount of messages logged in error log is no action will be performed.
This is to prevent cases in which only one node is left, but from its status it should be offline.
ProxySQL would normally set it offline and then back online.
This commit prevents it to change status twice.
The same applies also for:
- runtime_mysql_replication_hostgroups
- runtime_mysql_group_replication_hostgroups
- runtime_mysql_galera_hostgroups
Close#1435 and #1436
This commit also prevents shunned nodes to come back online if they are missing pings. Related to #1416
Because it reduces the number of checks, it may also be relevant to #1417
* is STMT_PREPARE failed when there is a previous status (maybe STMT_EXECUTE) disconnect the client immediately. Maybe this is related to #1366 , but I can't reproduce it yet
* purge prepared statements is also their server reference counter is 0
* initialize connections queue for purging connections
Statistics collected aboout GTID queries
Statistics displayed on HTTP server
Count number of GTID events per server
Online upgrade of all mysql_connections tables
Fixed path for libev
Handling of prepared statements changed a lot in 1.4 , as a lot of code was rewritten.
Old code was still present, and was possible to toggle it on and off based on PROXYSQL_STMT_V14 .
Because only the new code is maintained, all references to old code are now removed,
including PROXYSQL_STMT_V14
This commits adds a new variable: mysql-monitor_replication_lag_use_percona_heartbeat
This variable defines the percona heartbeat table used to check replication lag.
If set, replication lag is checked against the defined table, otherwise `SHOW SLAVE STATUS` is used.
To be set, the value should match the following regex:
```
`?([a-z\d_]+)`?\.`?([a-z\d_]+)`?
```
Variable name is `mysql-throttle_connections_per_sec_to_hostgroup` .
Curently it is a global variable and limits the number of new connections per
hostgroup, and not per specific node.
For example, if mysql-throttle_connections_per_sec_to_hostgroup=100, no more
than 100 new connections can be created on any hostgroup no matter the number
of servers in that hostgroup.
The default is very high (1000000) thus not changing default behaviour.
Tuning this variable allows to control and throttle connections spikes to the
backend servers.
Added also new status variable `Server_Connections_delayed`.
This is a counter of how many times Hostgroup Manager didn't return a
connection because the limit was reached. It is worth to note that a single
client request could make multiple requests, therefore this variable counts
the number of time a new connection wasn't created and not how many requests
were delayed.
If a user is configured with fast_forward, no connection should be taken from the connection pool.
Yet the Hostgroup Manager should know about such connection.
In `MySQL_Cluster` class added functions to sync with remote node, like:
- `pull_mysql_query_rules_from_peer()`
- `pull_mysql_servers_from_peer()`
- `pull_mysql_users_from_peer()`
- `pull_proxysql_servers_from_peer()`
Added 8 new global variables in Admin.
4 variables determine after how many different checks the remote configuration will be synced:
- cluster_mysql_query_rules_diffs_before_sync
- cluster_mysql_servers_diffs_before_sync
- cluster_mysql_users_diffs_before_sync
- cluster_proxysql_servers_diffs_before_sync
4 variables determine if after a remote sync the changes need to be written to disk:
- cluster_mysql_query_rules_save_to_disk
- cluster_mysql_servers_save_to_disk
- cluster_mysql_users_save_to_disk
- cluster_proxysql_servers_save_to_disk
Table `proxysql_servers` is now automatically loaded from disk to memory and into runtime at startup.
Added new Admin's command `LOAD PROXYSQL SERVERS FROM CONFIG` to load `proxysql_servers` from config file to memory (not runtime).
Internal structures with credentials in MySQL_Authentication moved from `unsorted_map` to `map` : this to ensure the right order when generating the checksum.
Config file supports both `address` and `hostname` for `mysql_servers` and `proxysql_servers` , #1091
`ProxySQL_Admin::load_proxysql_servers_to_runtime()` now has a lock or no lock option, to avoid deadlock
For now, Cluster module is quite verbose.
Extended class ProxySQL_Checksum_Value() in ProxySQL_Cluster module to support further metrics
Implemeted `SELECT GLOBAL_CHECKSUM()` and relative tracking of global checksums
Added variable `admin-cluster_check_status_frequency` to check peer's global status at regular intervals
This commit introduces:
2 new tables:
* `runtime_checksums_values` : stores checksums of configurations in runtime. For now for `mysql_query_rules`, `mysql_servers` and `mysql_users`
* `stats_proxysql_servers_checksums` : when clustering is enabled, it collects all metrics from `runtime_checksums_values` from all its peers
3 new global variables that defines it checksum needs to be generated during `LOAD ... TO RUNTIME`
* `admin-checksum_mysql_query_rules`
* `admin-checksum_mysql_servers`
* `admin-checksum_mysql_users`
ProxySQL Cluster connections now have timeouts:
* 1 second timeout for CONNECT and WRITE
* 60 seconds timeout for READ (useful for long poll)
Status of the server was not changed if:
* was OFFLINE_HARD
* was incorrectly present in myhgm.mysql_servers
Added also a mutex to serialize the read_only actions
If total mysql_servers.weight per hostgroup was higher than 64K, traffic was unbalanced
Conflicts:
include/gen_utils.h
lib/MySQL_HostGroups_Manager.cpp
Enforced max_writers
Enforced writer_is_also_reader
Increased the number of entries in mysql_server_group_replication_log to 100 per node
Lowered the minimum value for mysql-monitor_groupreplication_healthcheck_interval to 50ms
Lowered the minimum value for mysql-monitor_groupreplication_healthcheck_timeout to 50ms
Introduced a timeout in HGCU_thread_run()
Fixed a logical bug in MySQL_Protocol::process_pkt_COM_CHANGE_USER() that was causing to not authenticate user
The absence of the timeout and the logicl bug was causing a lot of connections to be dropped and not really reused.
But as now the connections are reused, they may present further bugs
* reduce build time by 25%;
* remove circular dependency on `cpp.h` and other headers.
* fixes issue with `my_global.h` from mariadbclient which
redefines `__attribute__`, that leads to broken STL and GCC
intrinsics if `<algorithm>` wasn't included prior including this header;
* fixes another issue with redefined `__attribute__` which leads to
`_conn_exchange_t` not being aligned to cache line size;
* removes `pthread_setstacksize` calls as `my_global.h` redefines `pthread_setstacksize`
to be `pthread_dummy(0)` which led to stack size being never adjusted;
* add missing include guards to some headers;
* remove unused proxysql_hash.h header.