131 Commits (208df329ea35f47c497df4cd7187131a98ef5b8a)

Author	SHA1	Message	Date
Javier Jaramago Fernández	66478b89ee	Added new stat 'new_req_conns_count' counting the num of 'create_new_connection' recieved	6 years ago
Javier Jaramago Fernández	4409044398	Added support for annotation 'create_new_connection' that prevents reusing backend connections	6 years ago
René Cannaò	00ce76e57e	Add support for SAVEPOINT #948	6 years ago
René Cannaò	7b021e027a	Do not cache routing metadata for COM_STMT_EXECUTE ... From 1.3.0 till 2.0.x , when COM_STMT_PREPARE is executed and Query Processor is executed, metadata are stored in MySQL_STMT_Global_info(). These metadata include routing information. Is a prepared statement is prepared on a specific hostgroup, it will always be executed on the same hostgroup. This causes negative consequences, like the one reported in issue #2691 . This commit: * remove hostgroup_id from MySQL_STMT_Global_info() and other metadata * Query Processor is called also during COM_STMT_EXECUTE * removes hostgroup from table stats_mysql_prepared_statements_info * Query Processor do not perform any query rewrite ruding COM_STMT_EXECUTE	6 years ago
René Cannaò	f9fdde3b99	Fast routing lookup without username	6 years ago
René Cannaò	9c2f0bf3ac	Adding tests to benchmark multiple lookups in mysql_query_rules_fast_routing	6 years ago
René Cannaò	d15f055b1b	Remove define PROXYSQL_QPRO_PTHREAD_MUTEX ... Removed also all old code that didn't use pthread mutex	6 years ago
René Cannaò	c4a0ea9d4d	Remove all code not related to FAST_ROUTING_NEW208 ... Removed also #define FAST_ROUTING_NEW208 This was new code introduced in 2.0.8 , and now is the default	6 years ago
val	a095ef00d3	use DIGEST_STATS_FAST_1 always	6 years ago
René Cannaò	b0af4d272d	Fix 2 bugs related to firewall ... * Query_Processor::find_firewall_whitelist_rule() wasn't correctly finding rules * SQL injection algorithm should only be triggered if the query isn't blocked already	6 years ago
René Cannaò	e40246f62a	Fixed a bug in firewall whitelist	6 years ago
Kir Shatrov	2f7bb1a1cf	Validate gtid input	6 years ago
Kir Shatrov	0922949b8c	Support min_gtid comment	6 years ago
René Cannaò	b8ecdaa708	Removing some comments	6 years ago
René Cannaò	e9dc7a9b60	Implementation of SQLI fingerprints whitelist ... libsqlinjection generates a lot of false positives. This commit introduces a new table: mysql_firewall_whitelist_sqli_fingerprints This table can list fingerprints generated by libsqlinjection: if the fingerprint is listed in this table, proxysql will consider it as a false positive. This commit also enables SQLi algorithm only if the query is not already explicitly whitelisted.	6 years ago
René Cannaò	485d313514	Further development of firewall whitelist. ... * introduced 2 new variables: - mysql-firewall_whitelist_enabled : defines if the firewall with whitelist algorithm is enabled or not - firewall_whitelist_errormsg : error message returned to the client if an error message is not already set * Query_Processor returns firewall_whitelist_mode to the calling MySQL_Session, but the logging on error log currently happens within the Query_Processor itself * if mysql-firewall_whitelist_enabled is turned on and a user it is not found in mysql_firewall_whitelist_users , the default behavior is that query is rejected * lookup on mysql_firewall_whitelist_users happens for username@ip (specific IP) and username@'' (all IPs) . Match on network subnet is currently not implemented * implemented SAVE MYSQL FIREWALL FROM RUNTIME * fixed implementation of SAVE MYSQL FIREWALL TO DISK * runtime configuration can be viewed in runtime_mysql_firewall_whitelist_rules and runtime_mysql_firewall_whitelist_users	6 years ago
René Cannaò	94f27716b7	Preparing SAVE MYSQL FIREWALL FROM RUNTIME	6 years ago
René Cannaò	6ee2a0cb19	Load mysql firewall whitelist rules to runtime	6 years ago
René Cannaò	ce4a4a2464	Code cleanup and speeding compile time	6 years ago
René Cannaò	80e92b144d	[WIP] Second commit for mysql firewall whitelist ... Improved also compiling time	6 years ago
René Cannaò	6bed334db1	stats_history.history_mysql_query_digest #2368 ... Implementation of new table stats_history.history_mysql_query_digest #2368 Implemented command `SAVE MYSQL DIGEST TO DISK`. Save to disk with really minimal memory footprint compared to querying `stats_mysql_query_digest` or `stats_mysql_query_digest_reset` . Implemented new admin variable `admin-stats_mysql_query_digest_to_disk` to automatically save to disk.	6 years ago
René Cannaò	f829ebcba0	Allows max_lag_ms to be 0	6 years ago
René Cannaò	a0ea0c118c	V2.0.8 fast routing (#2309) ... * Implementation of unit tests related to mysql_query_rules_fast_routing * Implementation of rules_fast_routing with khash * Adding memory usage for new fast routing algo	6 years ago
René Cannaò	5054998f7f	hostgroup sign in stats_mysql_query_digest #2285	6 years ago
René Cannaò	2fec51c0a9	A lot of changes to make query digest map faster ... Extended SQLite3_result to support a mutex for concurrent rows insert. Extensively rewritten get_query_digests() and get_query_digests_reset() to support multiple threads, and concurrently generate a resultset. Function get_query_digests_reset() defer the deletion of objects from query digest map after generating the resultset for Admin. This drastically reduce the amount of time that query digest map is locked. Rewritten get_query_digests_total_size() to make it multi-threaded. Added 3 new functions for purging: - purge_query_digests() : wrapper - purge_query_digests_sync() : synchronous purge of query digest map, either single-threaded or multi-threaded - purge_query_digests_async() : asynchronous purge of query digest map, single-threaded only If table `stats_mysql_query_digest_reset` is queried but not `stats_mysql_query_digest`, only the first one is populated. If table `stats_mysql_query_digest_reset` is queried and also `stats_mysql_query_digest`, the second one is populated as a copy from the first. Implemented new SQL `TRUNCATE [TABLE]` for `[stats.]stats_mysql_query_digest[_reset]`: this will purge the query digest map as fast as possible without creating any table in SQLite. For testing/benchmark: - modified ProxySQL_Test___GenerateRandomQueryInDigestTable() to generate more random data and randomly switch `mysql_thread___query_digests_normalize_digest_text` Extended command `PROXYSQLTEST` with the following tests: - 1 : generates entries in digest map - 2 : gets all the entries from the digest map, but without writing to DB, with multiple threads - 3 : gets all the entries from the digest map and reset, but without writing to DB, with multiple threads - 4 : purges the digest map, synchronously, in single thread - 5 : purges the digest map, synchronously, in multiple threads - 6 : purges the digest map, asynchronously, in single thread (this seems the faster) Replaced some sprintf() with new function my_itoa() . Modified class QP_query_digest_stats() to include username, schemaname and client_address if no longer than 24 bytes. This makes storage requirement for QP_query_digest_stats() bigger, but reduces a lot of calls to memory management. Modified QP_query_digest_stats::get_row() to drastically reduce the number of malloc/free and string copy. Overall, this commit has the following effects: - it increases the memory usage for query digest map. On test workload, this results in 10% more memory usage - drastically reduces the calls to memory management - populating `stats_mysql_query_digest_reset` is 3 times faster - the amount of time in which query digest map is locked is reduced by 95%	7 years ago
René Cannaò	46e11afab5	Add min_epoch_ms as a modifier for max_lag_ms ... Instead of specifying max_lag_ms, it is possible to specify min_epoch_ms	7 years ago
René Cannaò	ce757db70d	Merge pull request #2156 from sysown/v2.0.6-hg_locked ... V2.0.6 hg locked	7 years ago
René Cannaò	f2f3935fcd	drafting locked hostgroup	7 years ago
René Cannaò	0831740e45	Several enhancements on Query logger ... Enhancements: - added metrics rows_affected and rows_sent - added global variable mysql-eventslog_default_log : if 1 , logging is enabled for every query unless explicitly disabled in mysql_query_rules.log . Default is 0 - added global variable mysql-eventslog_format : default is 1 (legacy format). A value of 2 enables logging in JSON format. Issue #871 Changing value at runtime causes the current file to be closed and a new one created - fixed logging for prepared statements: till 2.0.5 only some percentage of prepared statements was correctly logged Extended tables stats_mysql_query_digest and stats_mysql_query_digest_reset to also include sum_rows_affected and sum_rows_sent Extended `eventslog_reader_sample.cpp` to support the new enhancements	7 years ago
René Cannaò	01023d6856	Merge branch 'v2.0.4' into v2.0.1-aurora	7 years ago
René Cannaò	af0a7100e2	Adding support for max_lag_ms in query	7 years ago
Carson Ip	1232a6382e	Fix minor typos	7 years ago
René Cannaò	9b14103236	Collect memory statistics about query rules	7 years ago
René Cannaò	4d3e45fc1b	Fixed bug in get_current_query_rules	7 years ago
René Cannaò	0a60cfda11	Merge branch 'v2.0.0-tb' into v2.0.0-merge-tb	7 years ago
René Cannaò	6f8c37eec9	Store digest_text on a different hash table ... This should drastically reduce memory usage where identical queries are executed an several schemas or by different users.	7 years ago
René Cannaò	c5cff7a921	Added client_address in stats_mysql_query_digest	7 years ago
René Cannaò	72724b22e0	Preparing for support of cache_timeout ... Adding column mysql_query_rules.cache_timeout . Functionality is not implemented yet, only the column is added.	7 years ago
René Cannaò	f666d40db2	Do not cache empty resultset, rule setting #1696	8 years ago
René Cannaò	b63df5aff2	Fix bug introduce in 6eeeb5c	8 years ago
René Cannaò	c15ee49797	Add support for wildcard in mysql_query_rules.client_addr #1450 ... Input validation: - client_addr not longer than INET6_ADDRSTRLEN - % allowed only at the end of client_addr Query rule itself remembers if there is a wildcard or not. If there is a wildcard compares string till the wildcard. If client_addr=='%' , it is a match all.	8 years ago
René Cannaò	13a57285b1	Trigger new routing algorithm only if hash table is not empty	8 years ago
René Cannaò	ce7d32421f	Reduce the size of delimiter	8 years ago
René Cannaò	97c81b4ef6	Initialize qr in Query_Processor	8 years ago
René Cannaò	b5d8c4d3ba	Added table mysql_query_rules_fast_routing ... This table is responsible for performing routing for very specific use case scenarios, like having tens of thousands of schemas	8 years ago
René Cannaò	46d7f9a14b	Add support for GTID consistent reads ... This patch includes a new column mysql_query_rules.gtid_from_hostgroup If defined, it is used for GTID consistent read	8 years ago
everpcpc	2ac60b0dbc	log before free	8 years ago
René Cannaò	78fce2b0d4	Improvement on stats_mysql_query_digest #1100 ... * record also unknown queries * remove parenthesis from the beginning of the query to understand the type of query	8 years ago
René Cannaò	fbd2f15efe	Memory Optimizations ... Try to allocate buffers in tokenizer() Optimized MySQL_ResultSet::buffer_to_PSarrayOut()	9 years ago
René Cannaò	a8846cc543	Incorrect number of columns in Query_Processor::get_current_query_rules() ... Crashing bug affecting only 1.4	9 years ago