110 Commits (c2f82b3d4e74ea632abcd4a12c19c0b20988570d)

Author	SHA1	Message	Date
Rene Cannao	2987242d4f	Fix cache_empty_result=0 not caching non-empty resultsets (issue #5248) ... The `cache_empty_result` field in query rules has three possible values: • -1: Use global setting (`query_cache_stores_empty_result`) • 0: Do NOT cache empty resultsets, but cache non-empty resultsets • 1: Always cache resultsets (both empty and non-empty) Previously, when `cache_empty_result` was set to 0, nothing was cached at all, even for non-empty resultsets. This prevented users from disabling caching for empty resultsets while still allowing caching of non-empty resultsets on a per-rule basis. Changes: 1. Modified caching logic in MySQL_Session.cpp and PgSQL_Session.cpp to add the condition `(qpo->cache_empty_result == 0 && MyRS->num_rows)` (MySQL) and `(qpo->cache_empty_result == 0 && num_rows)` (PgSQL) to allow caching when cache_empty_result=0 AND result has rows. 2. Added comprehensive Doxygen documentation in query_processor.h explaining the semantics of cache_empty_result values. 3. Updated Query_Processor.cpp with inline comments explaining the three possible values. Now when cache_empty_result is set to 0: - Empty resultsets (0 rows) are NOT cached - Non-empty resultsets (>0 rows) ARE cached - This matches the intended per-rule behavior described in issue #5248. Fixes: https://github.com/sysown/proxysql/issues/5248	2 months ago
Rahim Kanji	71cd99f408	If cache_empty_result is not defined in pgsql_query_rules or mysql_query_rules, pgsql-query_cache_stores_empty_result and mysql-query_cache_stores_empty_result value should be considered when determining whether to store empty result sets	1 year ago
Rahim Kanji	4e61120158	Fixed clang linking error	1 year ago
René Cannaò	d4bd93672e	Merge pull request #79 from sysown/v2.x_refactore_query_processor ... Refactored Query Processor [WIP]	1 year ago
Rahim Kanji	e723082a1b	Refactored Query Processor Architecture ... - Introduced separate query processors for MySQL and PostgreSQL. - Removed unnecessary GTID logic from `PgSQL_Session`. - Added a command counter for PostgreSQL queries. - Refactored `Query_Processor` to serve as a base class containing core functionalities applicable to both MySQL and PostgreSQL. Changes: - Removed GTID-related code from `PgSQL_Logger` and `PgSQL_Session`: - bool with_gtid; - char gtid_buf[128]; - int gtid_hid; - Added new PostgreSQL-specific tables: - `pgsql_query_rules`, `pgsql_firewall_whitelist_rules`, `pgsql_query_rules_fast_routing` - `runtime_pgsql_query_rules`, `runtime_pgsql_firewall_whitelist_rules`, `runtime_pgsql_query_rules_fast_routing` - Added PostgreSQL-related variables: pgsql_thread___commands_stats pgsql_thread___query_digests pgsql_thread___query_digests_lowercase pgsql_thread___query_digests_no_digits pgsql_thread___query_digests_replace_null pgsql_thread___query_digests_normalize_digest_text pgsql_thread___query_digests_track_hostname pgsql_thread___query_digests_keep_comment pgsql_thread___query_digests_max_digest_length pgsql_thread___query_digests_max_query_length pgsql_thread___query_digests_grouping_limit pgsql_thread___query_digests_groups_grouping_limit pgsql_thread___enable_load_data_local_infile pgsql_thread___auditlog_filename pgsql_thread___auditlog_filesize pgsql_thread___eventslog_filename pgsql_thread___eventslog_filesize pgsql_thread___eventslog_default_log pgsql_thread___eventslog_format pgsql_thread___firewall_whitelist_errormsg pgsql_thread___firewall_whitelist_enabled pgsql_thread___query_processor_iterations pgsql_thread___query_processor_regex	1 year ago
Javier Jaramago Fernández	25c6cf9dff	Fix 'nlohmann::json' type forwarding	1 year ago
Rahim Kanji	0f44307f30	More readable args	1 year ago
Rahim Kanji	5bf827d8e4	Removed type selector	2 years ago
René Cannaò	fde8ace076	Removed json.hpp from include files ... This seems to speedup compile time, and also lead to smaller binary	2 years ago
René Cannaò	de345ecaed	Merge remote-tracking branch 'public/v2.x' into v2.x_postgres_240422	2 years ago
René Cannaò	73f8ad0a35	Code split related to generate_proxysql_internal_session_json	2 years ago
René Cannaò	38c6c2ac7b	Merge branch 'v2.x-main' into v2.x_postgres_240422 ... Merged main ProxySQL code up to commit 959c629	2 years ago
René Cannaò	0a2cd03672	Refactoring to split Query_Processor.cpp into multiple files	2 years ago
Rahim Kanji	ebf20b6a60	* Added PostgreSQL configuration (port: 6034 Admin and 6035: PostgreSQL client connections) ... * Fixed few bugs * Added Load and Save commands for PostgreSQL tables	2 years ago
Rahim Kanji	e029d39007	Initial PostgreSQL commit ... * Added plain text and SCRAM authentication	2 years ago
Javier Jaramago Fernández	d3a34712df	Improve function resource acquisition readability ... - Improved readability for the locked map acquisition for function 'search_rules_fast_routing_dest_hg'. - Improved function documentation and added note about usage.	3 years ago
Javier Jaramago Fernández	4b5fe5888e	Fix race condition when reloading 'Query Rules Fast Routing'	3 years ago
René Cannaò	b5677e3ba9	Adding flagOUTs in mysql_query_rules.attributes ... in JSON format	3 years ago
Javier Jaramago Fernández	43372bcfc9	Optimize locking for 'search_rules_fast_routing_dest_hg' ... - Add parameter for conditionally locking inside function. - Move potential syscalls outside the 'pthread_rwlock_rdlock' section. - Reduce number of 'sprintf' calls. - Improved PROXYSQLTEST 11/15 with data randomly exceeding function default buffer size.	3 years ago
Javier Jaramago Fernández	8331858179	Remove thread variable for 'mysql-query_rules_fast_routing_algorithm' ... - Thread variable has been replaced for new variable in 'Query_Processor'. - Computation for 'mysql_query_rules_memory' is now performed by Admin during 'load_mysql_query_rules_to_runtime' instead of lazily by the MySQL_Threads.	3 years ago
Javier Jaramago Fernández	abccb39201	Add new variable 'mysql-query_rules_fast_routing_algorithm' ... - Implemented new variable 'query_rules_fast_routing_algorithm' that allows choosing between the classic per-thread hashmaps for 'query_rules_fast_routing' searches (1), and a new unique global hashmap (2). - Improved locking time on 'QueryProcessor::rwlock' during 'load_mysql_query_rules_to_runtime'. - Changed computation of 'mysql_query_rules_memory' to be lazily updated by threads themselves while building the per-threads maps. - Fixed 'ProxySQL_Admin::stats___memory_metrics' report of the memory metrics. - Improved 'PROXYSQLTEST 17' command, for being able to micro-benchmark and check 'query_rules_fast_routing_algorithm'.	3 years ago
Javier Sánchez Parra	0da59a6620	Honor reset and copy options in get_query_digests_reset_v2()	3 years ago
Javier Sánchez Parra	2c13dfdc52	Make PROXYSLQTEST 22 and 24 return the number of row affected	3 years ago
Javier Sánchez Parra	14ba7475d4	Create stats_mysql_query_digest statements directly from the digest_umap	3 years ago
Javier Sánchez Parra	6fb18bd2ac	Move get digest text code to its own function	3 years ago
Javier Sánchez Parra	28095e3109	Add auxiliary maps in get_query_digests_reset() to improve performance	3 years ago
Javier Sánchez Parra	263d645bad	Add auxiliary maps in get_query_digests() to improve performance ... Otherwise, all queries processed by ProxySQL have to wait for get_query_digest() to finish reading the maps. With this patch, ProxySQL stores query digests in the auxiliary maps while the main maps are being used by get_query_digests(). Once get_query_digests() finish reading the main maps, it merge the auxiliary maps in to the main maps and clear the content of the auxiliary maps.	3 years ago
René Cannaò	5e68075edf	Cluster: Speed up processing of MySQL Query Rules ... Multiple changes: - Query_Processor stored the resultset of query rules loaded to runtime (previously this was only for query rules fast routing) - Admin returns the stored resultset (query rules and fast routing) when queries by Cluster - In khash replaced the hashing function from the built-in __ac_X31_hash_string to CityHash32 - When Cluster is used, it calls load_mysql_query_rules_to_runtime() passing the resultsets retrieved by the remote peer - Increased SQLite cache_size to ~50MB: this seems to be a very small optimization and probably it will be reverted - pull_mysql_query_rules_from_peer() uses transactions to write to mysql_query_rules_fast_routing - (important change) pull_mysql_query_rules_from_peer() verifies the checksum of MySQL Query ules before loading them to runtime	4 years ago
René Cannaò	43c6acb218	Merge branch 'v2.x' into v2.1.2-cluster	4 years ago
André Cabaça	977bf0f1da	Removed dead code ... As reported in #3740 there was duplicated code. Originated from this merge #739	4 years ago
René Cannaò	f7bb4d201b	Add get_current_query_rules_fast_routing_count() ... Query_Processor::get_current_query_rules_fast_routing_count() now allows to quickly execute this query is ran on Admin: SELECT COUNT(*) FROM runtime_mysql_query_rules_fast_routing	5 years ago
Javier Jaramago Fernández	622a763920	Added new column 'attributes' to 'mysql_query_rules' table	5 years ago
Javier Jaramago Fernández	66478b89ee	Added new stat 'new_req_conns_count' counting the num of 'create_new_connection' recieved	6 years ago
Javier Jaramago Fernández	4409044398	Added support for annotation 'create_new_connection' that prevents reusing backend connections	6 years ago
René Cannaò	9c2f0bf3ac	Adding tests to benchmark multiple lookups in mysql_query_rules_fast_routing	6 years ago
René Cannaò	d15f055b1b	Remove define PROXYSQL_QPRO_PTHREAD_MUTEX ... Removed also all old code that didn't use pthread mutex	6 years ago
René Cannaò	c4a0ea9d4d	Remove all code not related to FAST_ROUTING_NEW208 ... Removed also #define FAST_ROUTING_NEW208 This was new code introduced in 2.0.8 , and now is the default	6 years ago
val	a095ef00d3	use DIGEST_STATS_FAST_1 always	6 years ago
René Cannaò	c4607d94fe	class Web_Interface_plugin friends Query_Processor	6 years ago
Kir Shatrov	3b031f3518	Prevent crash on index() returning NULL ... in handler___client_DSS_QUERY_SENT___server_DSS_NOT_INITIALIZED__get_connection	6 years ago
Kir Shatrov	2f7bb1a1cf	Validate gtid input	6 years ago
Kir Shatrov	0922949b8c	Support min_gtid comment	6 years ago
René Cannaò	e9dc7a9b60	Implementation of SQLI fingerprints whitelist ... libsqlinjection generates a lot of false positives. This commit introduces a new table: mysql_firewall_whitelist_sqli_fingerprints This table can list fingerprints generated by libsqlinjection: if the fingerprint is listed in this table, proxysql will consider it as a false positive. This commit also enables SQLi algorithm only if the query is not already explicitly whitelisted.	6 years ago
René Cannaò	485d313514	Further development of firewall whitelist. ... * introduced 2 new variables: - mysql-firewall_whitelist_enabled : defines if the firewall with whitelist algorithm is enabled or not - firewall_whitelist_errormsg : error message returned to the client if an error message is not already set * Query_Processor returns firewall_whitelist_mode to the calling MySQL_Session, but the logging on error log currently happens within the Query_Processor itself * if mysql-firewall_whitelist_enabled is turned on and a user it is not found in mysql_firewall_whitelist_users , the default behavior is that query is rejected * lookup on mysql_firewall_whitelist_users happens for username@ip (specific IP) and username@'' (all IPs) . Match on network subnet is currently not implemented * implemented SAVE MYSQL FIREWALL FROM RUNTIME * fixed implementation of SAVE MYSQL FIREWALL TO DISK * runtime configuration can be viewed in runtime_mysql_firewall_whitelist_rules and runtime_mysql_firewall_whitelist_users	6 years ago
René Cannaò	94f27716b7	Preparing SAVE MYSQL FIREWALL FROM RUNTIME	6 years ago
René Cannaò	6ee2a0cb19	Load mysql firewall whitelist rules to runtime	6 years ago
René Cannaò	4005746b1a	More on firewall whitelist	6 years ago
René Cannaò	ce4a4a2464	Code cleanup and speeding compile time	6 years ago
René Cannaò	80e92b144d	[WIP] Second commit for mysql firewall whitelist ... Improved also compiling time	6 years ago
René Cannaò	6bed334db1	stats_history.history_mysql_query_digest #2368 ... Implementation of new table stats_history.history_mysql_query_digest #2368 Implemented command `SAVE MYSQL DIGEST TO DISK`. Save to disk with really minimal memory footprint compared to querying `stats_mysql_query_digest` or `stats_mysql_query_digest_reset` . Implemented new admin variable `admin-stats_mysql_query_digest_to_disk` to automatically save to disk.	6 years ago