libsqlinjection generates a lot of false positives.
This commit introduces a new table: mysql_firewall_whitelist_sqli_fingerprints
This table can list fingerprints generated by libsqlinjection:
if the fingerprint is listed in this table, proxysql will consider it as
a false positive.
This commit also enables SQLi algorithm only if the query is not already
explicitly whitelisted.
This commit fix the following bug:
If some session variables were set by the client before proxysql turned on
lock_hostgroup, these variables were not set.
This commit also makes proxysql aware that this query causes lock_hostgroup:
`SET SESSION information_schema_stats_expiry=0`
Library libjection is being used.
New variable mysql-automatic_detect_sqli defines if the feature is enabled or not.
If an SQL injection is detected:
- the client connection is immediately terminated
- an entry in the error log is reported
- ability to handle transactions
- in OK packets and resultsets, SERVER_STATUS_IN_TRANS flag is set if a transaction is running
- a persistent database file in datadir is used instead of in-memory database
- database file uses WAL (Write-Ahead Logging)
- WAL journal size is set to 64MB
- monitor user can always login to SQLite3 Server
- in case of database lock within a transaction, an error is immediately returned to client
It a SET statement wasn't parsed correctly, hostgroup was locked
on current hostgroup and not on the hostgroup specified in
mysql_query_rules.destination_hostgroup
Initial support for `SET SESSION TRANSACTION READ ONLY` or `READ WRITE`.
Extended `SET` parser to support also `SET SESSION TRANSACTION`.
Hostgroup Manager doesn't kill backend connections in case of error 1231.
`autocommit` is set at session level but also on MySQL client connection.
Added several debugging entries.
Several `handler_again___verify_backend_*` functions are disabled if `locked_on_hostgroup` is enabled.
- switched from SSLv23_server_method() to TLS_server_method
- enforced min proto version to TLS1_VERSION using SSL_CTX_set_min_proto_version()
- when inspecting session status, cipher used is exported
Added 2 new status variables:
- queries_with_max_lag_ms__delayed
- queries_with_max_lag_ms__total_wait_time_us
Do not get replication lag from replicas if the value is 0
Fixed an error in the computation of max_lag_ms
This should fix a lot of issues related to failed parsing of SET statement.
This and the two 2 previous commits introduce several status variables, and a
new configuration variable: mysql-set_query_lock_on_hostgroup
Possible values for mysql-set_query_lock_on_hostgroup:
- 0 : legacy behavior , before 2.0.5
- 1 : (default) . SET statements that cannot be parsed correctly disable
both multiplexing AND routing. Attempting to route traffic while a
connection is linked to a specific backend connection will trigger
an error to be returned to the client
Issue #2120 : Send SESSION_TRACK_GTIDS to client
Issue #2121 : Track CLIENT_FOUND_ROWS required by the client
Issue #2125 : Track CLIENT_MULTI_STATEMENTS required by the client
Enhancements:
- added metrics rows_affected and rows_sent
- added global variable mysql-eventslog_default_log : if 1 , logging is enabled for every query unless explicitly disabled in mysql_query_rules.log . Default is 0
- added global variable mysql-eventslog_format : default is 1 (legacy format). A value of 2 enables logging in JSON format. Issue #871
Changing value at runtime causes the current file to be closed and a new one created
- fixed logging for prepared statements: till 2.0.5 only some percentage of prepared statements was correctly logged
Extended tables stats_mysql_query_digest and stats_mysql_query_digest_reset to also include sum_rows_affected and sum_rows_sent
Extended `eventslog_reader_sample.cpp` to support the new enhancements
In `stats_mysql_free_connections`, exported:
* address of connection
* address of mysql struct
* mysql thread_id
In `stats_mysql_processlist`, exported:
* address of session
* address of data stream
* address of connection
* address of mysql struct
* mysql thread_id
* removed password from non-debug built
Added new command `PROXYSQL INTERNAL SESSION` that clients can execute to
receive internal information about their own connection in JSON format.
Added JSON library.
Recompiled SQLite3 to support JSON.
Added new column `extended_info` in `stats_mysql_processlist`.
Added new mysql variable `mysql-show_processlist_extended` that determine the
content of `stats_mysql_processlist.extended_info`:
- 0 : no info
- 1 : JSON format
- 2 : JSON format with pretty printing
`SERVER_STATUS_NO_BACKSLASH_ESCAPES` is now tracked.
`set sql_mode` is executed immediately if client executes `set sql_mode`
specifying `NO_BACKSLASH_ESCAPES`.
A backend connection with `SERVER_STATUS_NO_BACKSLASH_ESCAPES` enabled has multiplexing immediately disabled.
Temporary disable multiplexing when last_insert_id is returned in OK packet.
Multiplexing is disabled for mysql-auto_increment_delay_multiplex queries.
mysql-auto_increment_delay_multiplex ranges from 0 to 1000000 .
Default value is 5
- if clients uses mysql_native_password, if LDAP is enabled and if the user doesn't exist, switch to mysql_clear_password.
- if neither mysql_native_password or mysql_clear_password are used by the client:
- if LDAP is not enabled, always switch to mysql_native_password
- if LDAP is enabled:
- if the user exists, switch to mysql_native_password
- if the user doesn't exists, switch to mysql_clear_password
Added MySQL variable mysql-add_ldap_user_comment to determine if a comment with the original username needs to be added in the queries.
This commit also tracks the charset during the first handshake response.
If:
* client wants autocommit=0
* enforce_autocommit_on_reads=false
* there is no transaction
* this seems to be the first query, and a SELECT not FOR UPDATE
Action:
* switch back to autcommit=1
If:
* client wants autocommit=0
* enforce_autocommit_on_reads=false
* there is no transaction
* this seems to be the first query, and a SELECT not FOR UPDATE
Action:
* switch back to autcommit=1
Both KILL QUERY and KILL CONNECTION work
The only security check enforced is that the user sending the KILL
is the same user of the connection/query being killed.
Functions add() and lookup() in MySQL_LDAP_Authentication have support for backend_username.
Added mysql_ldap_mapping table.
Created Admin::init_ldap() to be called after LDAP initialization.
Added better LDAP caching.
LOAD LDAP MAPPING TO RUNTIME cleans part of the cache (association to backend user).
All queries will have a comment "proxysql-ldap-user=%s" to track original user
The current parser for SET in MySQL_Session is not able to parse multiple
variables SET commands like:
SET sql_mode='TRADITIONAL', NAMES utf8 COLLATE unicode_ci
This patch introduces a simple regex based parser for all variation of
simple variables.
This is not a generic SET parser, though.
Fixed STMT_SEND_LONG_DATA processing which was incorrectly reading data from the STMT_EXECUTE packet, causing corruption of any subsequent parameters.
Prior to this it was only safe to use STMT_SEND_LONG_DATA for the last parameter of the statement.
Also:
* Corrected some comment indentation.
Fixes: #1283
The fix for bug #1038 was to not return a connection to the connection pool if it has an error.
Although this is correct, it also has the side effect that connections coming
from connection pool and failing during the first query because the connection
was already broken, would be considered as possibly to run a transaction.
That is incorrect.
Now at connection level it is tracked if the transaction status is known or not.
Variable reset_connection_algorithm could either be:
1 = algorithm used up too version 1.4
2 = algorithm new since ProxySQL 2.0 (now default)
When reset_connection_algorithm = 2 , MySQL_Thread itself tries to reset connections instead of relying on connections purger HGCU_thread_run()
* is STMT_PREPARE failed when there is a previous status (maybe STMT_EXECUTE) disconnect the client immediately. Maybe this is related to #1366 , but I can't reproduce it yet
* purge prepared statements is also their server reference counter is 0
* initialize connections queue for purging connections
Statistics collected aboout GTID queries
Statistics displayed on HTTP server
Count number of GTID events per server
Online upgrade of all mysql_connections tables
Fixed path for libev
Handling of prepared statements changed a lot in 1.4 , as a lot of code was rewritten.
Old code was still present, and was possible to toggle it on and off based on PROXYSQL_STMT_V14 .
Because only the new code is maintained, all references to old code are now removed,
including PROXYSQL_STMT_V14
If variable mysql-verbose_query_error is set, "Error during query" message will be extended adding:
- username
- client IP
- schemaname
- digest of the original query (not the original query itself)
If autocommit_false_is_transaction=true (false by default), a connection
with autocommit=0 is treated as a transaction.
If forward_autocommit=true (false by default), the same behavior applies.
3 server errors were retried only once, and then destroying the client connection instead of returning error:
* case 1290: // read-only
* case 1047: // WSREP has not yet prepared node for application use
* case 1053: // Server shutdown in progress
This fix should also improve the handling of graceful shutdown (error 1053)
Introduced 2 new global variables:
* mysql-stats_time_backend_query (default true)
* mysql-stats_time_query_processor (default true)
For backward compatibility, they are both enabled by default
Added variable for SQLite3 Server
Added new command:
* LOAD SQLITESERVER VARIABLES FROM MEMORY / TO RUNTIME
* LOAD SQLITESERVER VARIABLES FROM DISK / TO MEMORY
* SAVE SQLITESERVER VARIABLES FROM RUNTIME / TO MEMORY
* SAVE SQLITESERVER VARIABLES FROM MEMORY / TO DISK
Connections to SQLite3 Server uses the same MySQL users in `mysql_users`
Fixed minor issues related to ClickHouse Serve
Fixed also some Makefile errors
If a user is configured with fast_forward, no connection should be taken from the connection pool.
Yet the Hostgroup Manager should know about such connection.
Extended class ProxySQL_Checksum_Value() in ProxySQL_Cluster module to support further metrics
Implemeted `SELECT GLOBAL_CHECKSUM()` and relative tracking of global checksums
Added variable `admin-cluster_check_status_frequency` to check peer's global status at regular intervals
Automatically reset mysql->insert_id when a connection is sent to connection pool.
If is not reset to 0 , when a backend connections is assigned to a client, this one can potentially read the last inserted id from the previous client.
Extending mysql_query_rules with a new field: OK_msg
If OK_msg is not NULL, an OK packet is sent to the client with an optional
message as specified in OK_msg itself.
If OK_msg is an not NULL empty string, an OK packet is sent to the client
without any message.
If both error_msg and OK_msg are present, error_msg is returned.
This commit includes also few minor bugs, mostly related to typo that would
prevent online upgrade of mysql_query_rules from early release of 1.4.0 .
If the users connection is rejected we shouldn't be incrementing
`num_connections_used` on the account_details object. This leads to
`free_users` dropping below zero and _all_ subsequent requests getting
rejected for that user.
Refs #940