proxysql

Commit Graph

Author	SHA1	Message	Date
Rahim Kanji	c0f99c0e15	Refactor: Improved Prepared-Statement Cache Design (Lock-Free Hot Path) #5211 Concurrency and Memory Management * Lock-Free Ref Counting: Replaced global mutex-protected integer reference counts with `std::atomic<uint32_t>` within `PgSQL_STMT_Global_info`, eliminating lock contention during statement referencing. * Modern Ownership: Adopted std::shared_ptr<const PgSQL_STMT_Global_info> for global and local storage, providing automatic, thread-safe memory and lifecycle management. * Memory Optimization: Removed redundant auxiliary maps `global_id_to_stmt_names` and `map_stmt_id_to_info` from local and global statement managers respectively, reducing overall memory overhead. * Optimized Purging: Statement removal logic was simplified for efficiently identifying and cleaning up unused statements. Hot Path Performance (`BIND`, `DESCRIBE`, `EXECUTE`) * Bypassed Global Lookups: Local session maps now store the `shared_ptr` directly, removing the need to acquire the global lock and search the global map during hot path operations. * Direct Refcount Manipulation: Refcount modification functions now operate directly on the passed statement object, eliminating the overhead of searching the global map to find the object pointer based on statement id. Safety and Protocol Logic (`PARSE`) * Efficient Statement Reuse: Implemented a local fast path check for the unnamed statement (`""`), allowing immediate reuse of an identical query (same hash) upon re-parse, which bypasses global processing and locks. Cleanup * Cleaned up and class rename `PgSQL_STMT_Manager_v14` -> `PgSQL_STMT_Manager`.	3 months ago
Wazir Ahmed	5219d4be93	processlist: Prevent crash due to SQLITE_TOOBIG - Add new mysql/pgsql variable `processlist_max_query_length`. - Min: 1K - Max: 32M - Default: 2M - Truncate current query based on the configuration before inserting into `stats_*_processlist` tables. - Refactor/fix code related to other processlist configurations. 1. `session_idle_show_processlist` value was not updated in `ProxySQL_Admin.variables`. 2. Pass processlist config as an argument to `MySQL_Threads_Handler::SQL3_Processlist` instead of using thread-local variables. Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	4 months ago
Rahim Kanji	f3ea179384	Add backend PID/state to stats_pgsql_processlist and create pg_stat_activity-style view - Added `backend_pid` and `backend_state` columns to `stats_pgsql_processlist` to display PostgreSQL backend process ID and connection state. - Created `stats_pgsql_stat_activity` view on top of `stats_pgsql_processlist` with column aliases matching PostgreSQL's `pg_stat_activity` for consistency.	4 months ago
Rahim Kanji	ff50bb4ec1	Ensure PgSQL_Errors_statsvdestructor is invoked on delete #5112 Previously, deleting `PgSQL_Errors_stats` instances in TUs with only a forward declaration caused the destructor to be skipped, leaking member allocations. The fix ensures the full class definition is visible at delete sites.	6 months ago
Rahim Kanji	abd9fcb743	Add PgSQL prepared statement related stats	6 months ago
Rahim Kanji	38841bf189	Remove DESCRIBE statement cache option Remove the `DESCRIBE` (statement) cache option and related cache code.	6 months ago
Rahim Kanji	ce94a7080a	Merge branch v3.0_session_param_reset_connection into v3.0_extended_query_protocol	7 months ago
Rahim Kanji	703a5f35f5	Added stats_pgsql_prepared_statements_info table	7 months ago
Javier Jaramago Fernández	25d5681c61	Fix potential overflow for 'sqlite3_status' memory metrics The previous call have been replaced by 'sqlite3_status64' when querying 'SQLITE_STATUS_MEMORY_USED' to avoid potential integer overflows.	8 months ago
Javier Jaramago Fernández	6f2798abe0	Fix memory leak in 'ProxySQL_Admin::stats___mysql_global' - Closes #4976 The resource leak was associated to prepared statements (sqlite3_stmt). Since the detection of these leaks is tricky, because the resources held by 'SQLite3' never lose their references, a helper type ('stmt_unique_ptr') and a safer variation for 'prepare_v2' have been introduced. This helper type should automatically handle the lifetime of the statements, avoiding these kind of leaks in the future. The non-safe alternative has been flagged as 'deprecated', so it can be easily identified in later refactors or slowly removed during development.	8 months ago
René Cannaò	e17477e297	Merge pull request #4871 from sysown/v3.0-4803_20250313 Port to 3.0 of in-memory query logging	11 months ago
Javier Jaramago Fernández	844d64d03e	Display warning message on problematic OpenSSL versions - Closes #4830 Also adds a status variable exporting the OpenSSL version in use.	11 months ago
René Cannaò	82648cf101	Fix port in previous commit	12 months ago
René Cannaò	3475f8a75f	Port of PR #4822 to v3.0	12 months ago
René Cannaò	cb53db81ca	Export MySQL_Logger metrics for v3.0 Port of #4802	12 months ago
Javier Jaramago Fernández	2c7b9847f6	Optimize 'stats_mysql_global' generation with prepared statements Measurements ('perf') revealed that 'prepare' from 'execute' where taking roughtly half the execution time of the queries used for table generation.	12 months ago
Rahim Kanji	720441af20	Refactored and Optimized Query Cache with PgSQL Support * Added Query Cache Support for PgSQL: * Introduced new variables for PgSQL Query Cache management: * pgsql-query_cache_size_MB * pgsql-query_cache_soft_ttl_pct * pgsql-query_cache_stores_empty_result * pgsql-query_cache_handle_warnings (not yet utilized) * Overwriting Transaction Status: * Sending current transaction state to the client when a result is returned from the query cache. * This currently does not differentiate between normal and error transaction states—both are treated as active transaction states. * Cache Lifetime Management: Replaced manual reference counting with std::shared_ptr to efficiently manage the lifecycle of cache entries, ensuring proper memory deallocation and reducing complexity. * Unified Purging Logic: Refactored the query cache purging logic to use a single thread for both MySQL and PgSQL cache entries. * Code Cleanup: Removed unnecessary methods and data members to streamline the codebase and improve maintainability. * Performance Optimization: Improved memory management, reducing memory footprint and enhancing performance.	1 year ago
Rahim Kanji	0229daeb64	Added stats_pgsql_query_digest and stats_pgsql_query_digest_reset tables	1 year ago
René Cannaò	d4bd93672e	Merge pull request #79 from sysown/v2.x_refactore_query_processor Refactored Query Processor [WIP]	1 year ago
René Cannaò	4c1d1ec81a	Fix bug in issue 4646	1 year ago
René Cannaò	f39d8cac49	Port of commit `58fa5ee5ec`	1 year ago
Rahim Kanji	e723082a1b	Refactored Query Processor Architecture - Introduced separate query processors for MySQL and PostgreSQL. - Removed unnecessary GTID logic from `PgSQL_Session`. - Added a command counter for PostgreSQL queries. - Refactored `Query_Processor` to serve as a base class containing core functionalities applicable to both MySQL and PostgreSQL. Changes: - Removed GTID-related code from `PgSQL_Logger` and `PgSQL_Session`: - bool with_gtid; - char gtid_buf[128]; - int gtid_hid; - Added new PostgreSQL-specific tables: - `pgsql_query_rules`, `pgsql_firewall_whitelist_rules`, `pgsql_query_rules_fast_routing` - `runtime_pgsql_query_rules`, `runtime_pgsql_firewall_whitelist_rules`, `runtime_pgsql_query_rules_fast_routing` - Added PostgreSQL-related variables: pgsql_thread___commands_stats pgsql_thread___query_digests pgsql_thread___query_digests_lowercase pgsql_thread___query_digests_no_digits pgsql_thread___query_digests_replace_null pgsql_thread___query_digests_normalize_digest_text pgsql_thread___query_digests_track_hostname pgsql_thread___query_digests_keep_comment pgsql_thread___query_digests_max_digest_length pgsql_thread___query_digests_max_query_length pgsql_thread___query_digests_grouping_limit pgsql_thread___query_digests_groups_grouping_limit pgsql_thread___enable_load_data_local_infile pgsql_thread___auditlog_filename pgsql_thread___auditlog_filesize pgsql_thread___eventslog_filename pgsql_thread___eventslog_filesize pgsql_thread___eventslog_default_log pgsql_thread___eventslog_format pgsql_thread___firewall_whitelist_errormsg pgsql_thread___firewall_whitelist_enabled pgsql_thread___query_processor_iterations pgsql_thread___query_processor_regex	1 year ago
Rahim Kanji	3112a56da4	Added 'stats_pgsql_client_host_cache' and 'stats_pgsql_client_host_cache_reset' tables.	2 years ago
Rahim Kanji	e939cf088e	Added 'stats_pgsql_errors' and 'stats_pgsql_errors_reset' tables	2 years ago
Rahim Kanji	43d3ec82c8	Added stats_pgsql_processlist table * Implemented 'SHOW FULL PGSQL PROCESSLIST' command. * Implemented 'SHOW PGSQL PROCESSLIST' command.	2 years ago
Rahim Kanji	0b8df6df00	Added stats_pgsql_users table	2 years ago
Rahim Kanji	b4e1e1eea7	Added stats_pgsql_free_connections table	2 years ago
Rahim Kanji	9cfe5b1761	Added stats_pgsql_connection_pool & stats_pgsql_connection_pool_reset tables	2 years ago
Rahim Kanji	c35a6a24cf	Added stats_pgsql_global table	2 years ago
Rene Cannao	a1fd4af1a6	Splitting ProxySQL_Admin	2 years ago
Rahim Kanji	a2516654cf	Added PgSQL Thread variables. Using a few MySQL thread variables, particularly for query digest, query processor, and query cache. In the future, we plan to create a global variable for modules that are common to both MySQL and PGSQL	2 years ago
René Cannaò	d0da556c12	Minor optimization in headers	2 years ago
René Cannaò	7a50a68f28	Split of ProxySQL_Admin.cpp ProxySQL_Admin.cpp as become very big, thus it is time to split it in smaller files for better reasibility and processing. Still a work in progress.	2 years ago

33 Commits (28742554b5ec3aa92e69e81b6e8bf047e2cbbfae)