* Added forced triggering of DNS Cache Update on MySQL Servers and ProxySQL Servers table update.
* Direct DNS cache update via socket (connected peer ip) if record is not available in cache.
* Used strdup instead of realloc.
* DNS lookup only once for ProxySQL Cluster Nodes.
- Compression flag is now propagated from client to backend connection
when the backend connection is being created for a 'fast_forward'
session. In case of client trying to connect without compression, to
a server configured with compression, we fallback to an uncompressed
connection.
- After a 'MySQL_Session' has obtained a connection for a 'fast_forward'
session, i.e, the one-to-one relationship between the client and
backend connection has been stablished, we completely disable the
compression 'STATUS_MYSQL_CONNECTION_COMPRESSION' flag from the client
'MySQL_Connection'.
- Correct the behavior for 'connect_retries_on_failure' for
'fast_forward' sessions to match regular sessions, reusing the
same implementation based on 'MySQL_Data_Stream::max_connect_time'.
- Change 'connect_timeout' for 'fast_forward' sessions to be the
highest of 'connect_timeout_server' and 'connect_timeout_server_max'.
- Add handling for 'COM_QUIT' packets for 'fast_forward' sessions which
have not yet received a backend connection.
This forces the re-evaluation of the query digest for each
'STMT_EXECUTE' for impossing the required flags over the current
backend 'MySQL_Connection'.
This commit contains an implementation rework and a fix:
- Impl for 'auto_increment_delay_multiplex_timeout_ms' has been
reworked in favor of reusing 'wait_until' to share logic with
previous 'connection_delay_multiplex_ms' variable.
- Fix previous 'connection_delay_multiplex_ms' behavior preventing
connection retaining when traffic unrelated to target hostgroup is
being received by the session.
If any scenario is found in which we may want to consider a non
'ASYNC_IDLE' connection for expiring, this precondition shall be
removed, and checks should be responsible for ensuring 'ASYNC_IDLE'
state in the connection.
Because of MySQL bug https://bugs.mysql.com/bug.php?id=107875 ,
autocommit=0 can "create a transaction" even if SERVER_STATUS_IN_TRANS
is not set. This means that a SAVEPOINT can be created in a "transaction"
created with autocommit=0 but without SERVER_STATUS_IN_TRANS .
This commit takes also care of scenarios when autocommit=0 was switched back
to autocommit=1 when a SELECT was executed while variable
mysql-enforce_autocommit_on_reads was false.
MySQL_Session::NumActiveTransactions() now accepts a flag to check
for savepoints.
PMC-10003: Retrieved a resultset while running a simple command using async_send_simple_command() .
async_send_simple_command() is used by ProxySQL to configure the connection, thus it
shouldn't retrieve any resultset.
A common issue for triggering this error is to have configure mysql-init_connect to
run a statement that returns a resultset.
* uninitialized variables variables.read_only in Clickhouse
* generate_proxysql_internal_session_json() may try to access variables not set
This commit also prevents watchdog to restart ProxySQL if running under valgrind.
In debug build, the spinning loops also have a minor usleep.
This to prevent valgrind to loop for long time
Track NO_BACKSLASH_ESCAPES in case it is changed:
- using SET sql_mode by the client
- directly on the backend (for example default sql_mode, or an
sql_mode set from another variable)
New variables supported:
- lc_messages
- lc_time_names
- long_query_time
- max_execution_time
- max_heap_table_size
- max_sort_length
- optimizer_prune_level
- optimizer_search_depth
- optimizer_switch
- sort_buffer_size
- sql_big_selects
- timestamp
- tmp_table_size
Note: sql_big_selects is still not fully supported when
combined with max_join_size.
In set testing TAP ,set large max_execution_time.
If max_execution_time is too small, the TAP test may fails because the
queries will timeout.
Better parsing of SET SQL_LOG_BIN .
SQL_LOG_BIN is now managed by the same functions that handle boolean variables
instead of the now deprecated MySQL_Session::handler_WCD_SS_MCQ_qpo_Parse_SQL_LOG_BIN()
For naming convention, enum value SQL_LOG_BIN was renamed to SQL_SQL_LOG_BIN
In MySQL_Thread we also deprecated match_regexes[0] , previously
used to parse `set sql_log_bin`.
Functions Variable::fill_server_internal_session() and
Variable::fill_client_internal_session() seems to handle sql_log_bin as a
special value. This exception was removed.
Fixed a bug in MySQL_Session::generate_proxysql_internal_session_json() where
when backend variables needed to be generated, fill_client_internal_session()
was incorrectly called instead of fill_server_internal_session() , leading to
incorrect values reported by `PROXYSQL INTERNAL SESSION`
- Provided a new function to partially initialize 'MySQL_ResultSet'.
- Replaced 'generate_pkt_row' in favor of 'generate_pkt_row3' for
resulset writing.
Deprecated variables:
- net_write_timeout
Renamed set_testing-t.csv to set_testing-t.csv.obsolete , and removed net_write_timeout from set_testing-t.csv
Variables added to ignore list:
- net_write_timeout
- net_buffer_length
- read_buffer_size
- read_rnd_buffer_size
Added support for quoting with backtick
Renamed `enum variable_name` into `enum mysql_variable_name` for better readibility
`enum mysql_variable_name` has variable names alphabetically ordered after SQL_NAME_LAST_LOW_WM
`mysql_tracked_variables` has variable names alphabetically ordered after SQL_NAME_LAST_LOW_WM
A lot of new variables added in `mysql_tracked_variables` , although not all tracked yet (TODO)
In `MySQL_Variables::MySQL_Variables()`:
- if `internal_variable_name` in `mysql_tracked_variables` is NULL , it will be automatically initialized
- it performs some sanity check between `enum mysql_variable_name` and `mysql_tracked_variables`
Testing:
- refactored code in set_testing-t.cpp and set_testing-multi-t.cpp
- added set_testing-240-t.cpp to test improvements in 2.4.0
- generate_set_session_csv to automatically create set_testing-240.csv used by set_testing-240-t
This commits partially revert 0dde4adb21
It also improve the TAP test reg_test_3493-USE_with_comment-t.cpp with a
variety of combinations, that helped find a bug in the tokenizer
mysql_query_digest_and_first_comment()
Add initial support for FR limiting the number of connection errors that
can be initiated from a particular address before deniying future
connections from that address. Feature is controlled by new introduced
variables:
- mysql-client_host_error_counts
- mysql-client_host_cache_size
Command PROXYSQL RELOAD TLS is able to load new key/cert files and
create a new SSL context.
Loading of SSL can happens in two stages:
* during bootstrap: if it fails, proxysql dies
* running PROXYSQL RELOAD TLS: if it fails, proxysql will keep the old SSL context
This new method should successfully handle errors like missing or corrupted files.
We are also adding two buffers used to store the key/cert, to be used by the web interface.
This commit add the support of ERR packet in binary protocol resultset,
for example due to MAX_EXECUTION_TIME while running a prepared statement.
It also changes how EOF packets are added: instead of writing the EOF in the
resultset buffer, the EOF packet is written in its own packet. This allows to
easily remove it if needed.
It also fix what seems (not sure) a bug in MariaDB client library, where if a
prepared statement is interrupted it cannot be executed again
Added function MySQL_Session::detected_broken_connection() that unifies the way broken connections are logged.
It also logs username, and last time used.
Added also a new macro proxy_error_inline()
'CurrentQuery.QueryParserArgs.first_comment' needs to hold it's own copy
of 'first_comment', otherwise, the 'first_comment' from global 'stmt_info'
will be freed by 'Query_Info::end' at the end of the life of 'CurrentQuery'.
All variables that are not known by ProxySQL should fail to be parsed.
A best effor should also be done for tracking all the variables present
in a 'SET' statement prior to returning the parsing failure.
If a backend connection was terminated in the middle of the execution of multiple
statements in a multi-statements command, the worker thread was entering an infinite loop.
Furthermore, this commit disables query retry in case of multi-statements if the first one
completed successfully
This fixes#3339
This commit is the port to 2.1 of 90a4922ed8
* MySQL_ResultSet() stores a pointer to the PS
* MySQL_ResultSet::init_with_stmt() doesn't need anymore the pointer to the PS
Furthermore, it isn't anymore an "init" but an end (we will rename it)
* Added MySQL_ResultSet::add_row(MYSQL_ROWS *rows) for PS only
* MySQL_Protocol::generate_pkt_row3() accepts an optional row length.
If passed, it will know the length of the row without computing it
* MyRS initialized during ASYNC_STMT_EXECUTE_STORE_RESULT_START
* Implemented throttling during ASYNC_STMT_EXECUTE_STORE_RESULT_START
* MySQL_Connection::process_rows_in_ASYNC_STMT_EXECUTE_STORE_RESULT_CONT() always
generates a heartbeat. For now unnecessary, and we will better tune it later
* test_ps_async-t.cpp tunes mysql-threshold_resultset_size to trigger buffering
More testing is required
Few special queries are handled separately in ProxySQL
1. select @@version_comment limit 1
2. SHOW WARNINGS
3. SELECT USER()
4. PROXYSQL INTERNAL SESSION
5. SELECT LAST_INSERT_ID()
6. SELECT CONNECTION_ID()
7. C code mysql_set_server_option(mysql, MYSQL_OPTION_MULTI_STATEMENTS_ON)
We now remove EOF1 for these and respond with OK packets intead of EOF2
Former max_transaction_time is now renamed to max_transaction_idle_time.
mysql-max_transaction_idle_time defines the maximum time a transaction can stay idle.
mysql-max_transaction_time defines the maximum time of a transaction since its start.
In handler___status_WAITING_CLIENT_DATA___STATE_SLEEP___MYSQL_COM_QUERY_qpo()
Changing the default of exit_after_SetParse from false to true.
If a variable isn't parsed or it needs to be forwarded, exit_after_SetParse is set to false
Changing the mapping logic in SESSION_TRACK_GTIDS in frontend and backend connections.
Up to now, frontend and backend `SESSION_TRACK_GTIDSs match.
This is now changed:
* backend connections are by default set to `mysql-default_session_track_gtids`
* if `mysql-default_session_track_gtids=OFF` (the default) , `session_track_gtids` is not changed on backend
* if the client asks for `session_track_gtids=OFF` , proxysql ignores it (it just acknowledge it)
* if the client asks for `session_track_gtids=OWN_GTID` , proxysql will apply it
* if the client asks for `session_track_gtids=ALL_GTIDS` , proxysql will switch to OWN_GTID and generate a warning
* if the backend doesn't support `session_track_gtids` (for example in MySQL 5.5 and MySQL 5.6), proxysql won't apply it. It knows checking server capabilities
This commit also deprecates function `MySQL_Session::handler_again___verify_backend__generic_variable()`
ProxySQL is now also able to know when proxysql binlog reader is perhaps in use tracking all gtid_port in mysql_servers.
This feature in future will be used to deprecate `mysql-default_session_track_gtids`
- Added extra doc about the 'status_flags' and 'server_status'.
- Added 'sql_log_bin0' and 'prepared_statement' to the 'conn.status'
from PROXYSQL SESSION INFO.
When a backend connection is reused for a different user, the data stream status is set to `STATE_MARIADB_QUERY`, but was not set back to `STATE_MARIADB_GENERIC` once the user was switched successfully.
This caused additional connection setup (like executing `init_connect`) to be skipped on the first query after switching users. If another query using the the same user was executed, this new query would end up performing this additional connection setup.
By setting the data stream status back to `STATE_MARIADB_GENERIC`, we can ensure connection setup happens correctly after switching users.
When a new backend connection was selected, the `session_track_gtids` config variable would be marked as set, but was never actually set on the backed connection.
This commit prepares proxysql for further development related to SQLite3.
Specifically, it allows to either use built-in SQLite3 library, or to load it from a plugin.
Adding the following methods:
* handler___status_WAITING_CLIENT_DATA___STATE_SLEEP___MYSQL_COM_STMT_RESET()
* handler___status_WAITING_CLIENT_DATA___STATE_SLEEP___MYSQL_COM_STMT_CLOSE()
* handler___status_WAITING_CLIENT_DATA___STATE_SLEEP___MYSQL_COM_STMT_SEND_LONG_DATA()
The two main blocks are:
* get_pkts_from_client() : handling the main loop around label __get_pkts_from_client
* handler() : handling the main loop around label handler_again
First commit to simplify MySQL_Session::handler()
It handles the following statuses:
* CHANGING_USER_SERVER
* CHANGING_AUTOCOMMIT
* SETTING_MULTI_STMT
* SETTING_SESSION_TRACK_GTIDS
* SETTING_SET_NAMES
From 1.3.0 till 2.0.x , when COM_STMT_PREPARE is executed and Query Processor is executed, metadata are stored in MySQL_STMT_Global_info().
These metadata include routing information.
Is a prepared statement is prepared on a specific hostgroup, it will always be executed on the same hostgroup.
This causes negative consequences, like the one reported in issue #2691 .
This commit:
* remove hostgroup_id from MySQL_STMT_Global_info() and other metadata
* Query Processor is called also during COM_STMT_EXECUTE
* removes hostgroup from table stats_mysql_prepared_statements_info
* Query Processor do not perform any query rewrite ruding COM_STMT_EXECUTE