proxysql

Commit Graph

Author	SHA1	Message	Date
Wazir Ahmed	50de536534	MCP: Fix crash during server restarts Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	1 month ago
Rene Cannao	f2536f01d2	Merge v3.1-vec into v3.1-MCP2_QR Resolved merge conflict in lib/ProxySQL_Admin_Stats.cpp: - Kept v3.1-MCP2_QR version with proper NULL handling - Uses stmt_unique_ptr (RAII smart pointer) - Checks for NULL before calling atoll() on numeric fields - Discarded v3.1-vec version which lacked NULL checks The merged code correctly handles NULL values for: - run_id - count_star - first_seen - last_seen - sum_time - min_time - max_time	1 month ago
Rene Cannao	188aef90fe	fix: Use delete instead of free for SQLite3_result in load_mcp_query_rules_to_runtime Change free(resultset) to delete resultset in ProxySQL_Admin::load_mcp_query_rules_to_runtime. SQLite3_result is a C++ class allocated with new, so it must be deallocated with delete, not free(). Using free() causes undefined behavior and memory leaks. Addresses coderabbitai review comment.	1 month ago
Javier Jaramago Fernández	52142c4648	fix: Multiple issues with MCP query_(rules/digests) - Fixed invalid memory accesses for tables: + mcp_query_rules + stats_mcp_query_rules + stats_mcp_query_digests - Fixed inactive 'mcp_query_rules' being loaded to runtime. - Fixed hash computation in 'compute_mcp_digest'. - Fixed invalid escaping during 'stats_mcp_query_digests' gen. - Fixed digest generation for MCP arguments: + SQL queries are now preserved using 'mysql_query_digest_and_first_comment'. + TODO: Options for the tokenizer are right now hardcoded. - Added initial testing and testing plan for MCP query_(rules/digests). + TODO: Test finished on phase8. Timeouts destroy the MCP connection, leaving it unusable for subsequent queries this should be fixed for continuing testing. - TODO: There are several limitations to fix in 'validate_readonly_query'. This reflect in some query hacks in the testing. + 'SELECT' starting with comments (--) gets flagged as non-read. + 'SELECT' must have a 'SELECT .* FROM' structure. While common, simple testing queries many times lack this form.	1 month ago
Rene Cannao	709649232b	fix: Address AI code review concerns from PR #19 This commit addresses valid concerns raised by coding agents (Gemini, Copilot, CoderabbitAI): 1. Fix stats_mcp_query_digest naming conflict (ProxySQL_Admin.cpp): - Made reset and non-reset paths mutually exclusive using else block - Prevents both flags from being true, matching MySQL pattern - Ensures reset takes precedence over non-reset 2. Fix INSERT OR REPLACE sync issue (Admin_Handler.cpp): - Added DELETE before INSERT OR REPLACE in LOAD/SAVE MCP QUERY RULES - Prevents stale rules from persisting when syncing disk <-> memory - Ensures deleted source rows are also removed from target 3. Fix integer division truncation for timeout (Query_Tool_Handler.cpp): - Changed timeout_ms/1000 to (timeout_ms+999)/1000 for ceiling division - Ensures sub-second timeouts (e.g., 500ms) become at least 1 second - Prevents zero-second timeouts from causing unexpected behavior 4. Remove confusing comment (Discovery_Schema.cpp): - Simplified column count comment to be clear and accurate Note: The re_modifiers parsing code already correctly handles VARCHAR "CASELESS" to int conversion (lines 2414-2425), so that review comment was already addressed.	1 month ago
Rene Cannao	23aaf80cd1	fix: Address AI code review concerns for PR #19 Fixed 6 legitimate issues from AI code review: Critical fixes: - Implement stats___mcp_query_digest to populate table with actual data - Fix double trigger bug (stats_mcp_query_digest_reset substring matching) Important fixes: - Fix re_modifiers parsing (VARCHAR "CASELESS" -> int 1) - Fix TOCTOU race condition in reset path (use write lock from start) - Add column count validation before accessing row fields Documentation: - Add memory ownership documentation for evaluate_mcp_query_rules False positives ignored (Issue 2: Schema mismatch, Issue 6: SQL injection)	1 month ago
Rene Cannao	a1d9d2f1ba	docs: Add comprehensive documentation to MCP features Add detailed function-level documentation to all MCP query rules, query digest, static harvester, and catalog components. Static_Harvester.cpp: - Document all 18+ harvest functions (schemas, objects, columns, indexes, FKs, views) - Document lifecycle methods (init, close, connect, disconnect) - Document helper methods (is_time_type, is_id_like_name) - Document run management (start_run, finish_run, run_full_harvest) - Document statistics methods (get_harvest_stats) Query_Tool_Handler.cpp: - Document JSON helper functions (json_string, json_int, json_double) - Document digest tracking section with flow explanation MySQL_Catalog.cpp: - Document schema isolation architecture - Document CRUD operations (upsert, get, search, list, remove, merge) Discovery_Schema.cpp: - Document MCP query rules evaluation (evaluate_mcp_query_rules) - Document digest functions (compute_mcp_digest, fingerprint_mcp_args) - Document update/get functions for rules and digests ProxySQL_Admin_Stats.cpp: - Document stats collection functions ProxySQL_Admin.cpp: - Document load/save functions for query rules Admin_Handler.cpp: - Document MCP query rules command handlers include/ProxySQL_Admin_Tables_Definitions.h: - Add comments explaining table purposes	1 month ago
Rene Cannao	cc3cc25532	fix: Remove unused reset parameter from stats___mcp_query_rules() Change function signature from stats___mcp_query_rules(bool reset) to stats___mcp_query_rules() to match MySQL query rules pattern. The reset parameter was never used in the function body and MySQL's stats___mysql_query_rules() has no reset parameter.	1 month ago
Rene Cannao	f01fc79584	feat: Add runtime_mcp_query_rules table and fix stats_mcp_query_rules schema - Add ADMIN_SQLITE_TABLE_RUNTIME_MCP_QUERY_RULES schema (17 columns, same as mcp_query_rules) - Fix STATS_SQLITE_TABLE_MCP_QUERY_RULES to only have rule_id and hits columns - Add runtime_mcp_query_rules detection and refresh in ProxySQL_Admin - Implement save_mcp_query_rules_from_runtime(bool _runtime) for both config and runtime tables - Update get_mcp_query_rules() to return 17 columns (no hits) - get_stats_mcp_query_rules() returns 2 columns (rule_id, hits) Mirrors the MySQL query rules pattern: - mcp_query_rules: config table (17 cols) - runtime_mcp_query_rules: runtime state (17 cols) - stats_mcp_query_rules: hit counters (2 cols)	1 month ago
Rene Cannao	a816a756d4	feat: Add MCP query tool usage counters to stats schema Add stats_mcp_query_tools_counters and stats_mcp_query_tools_counters_reset tables to track MCP query tool usage statistics. - Added get_tool_usage_stats_resultset() method to Query_Tool_Handler - Defined table schemas in ProxySQL_Admin_Tables_Definitions.h - Registered tables in Admin_Bootstrap.cpp - Added pattern matching in ProxySQL_Admin.cpp - Added stats___mcp_query_tools_counters() in ProxySQL_Admin_Stats.cpp - Fixed friend declaration for track_tool_invocation() - Fixed Discovery_Schema.cpp log_llm_search() to use prepare_v2/finalize	1 month ago
Rene Cannao	1eb42c57d0	fix: Add GenAI variables to runtime_global_variables population Add flush_genai_variables___runtime_to_database() call to the central location where all modules populate runtime_global_variables table. This was missing, causing genai-* variables to not appear in runtime_global_variables.	1 month ago
Rene Cannao	119ca5003a	Fix compilation errors in debug build - Move explicit template instantiations for send_ok_msg_to_client and send_error_msg_to_client to after template definitions - Add missing closing brace for init_mcp_variables() - Fix missing #endif and closing brace for GloMCPH shutdown block	1 month ago
René Cannaò	313f637cf0	Merge branch 'v3.1-vec' into v3.1-MCP1 Signed-off-by: René Cannaò <rene.cannao@gmail.com>	1 month ago
Rene Cannao	5a85ef04f6	Fix MCP variables persistence and add DISK command support This commit fixes several issues with MCP (Model Context Protocol) variables not being properly persisted across storage layers and adds support for DISK commands. Changes: 1. lib/Admin_FlushVariables.cpp: - Fixed flush_mcp_variables___runtime_to_database() to properly insert variables into runtime_global_variables using db->execute() with formatted strings (matching admin pattern) - Fixed SQL format string to avoid double-prefix bug (qualified_name already contains "mcp-" prefix) - Fixed lock ordering by releasing outer wrlock before calling runtime_to_database with use_lock=true, then re-acquiring - Removed explicit BEGIN/COMMIT transactions to match admin pattern 2. lib/Admin_Handler.cpp: - Added MCP DISK command handlers that rewrite commands to SQL queries: * LOAD MCP VARIABLES FROM DISK -> INSERT OR REPLACE INTO main.global_variables * SAVE MCP VARIABLES TO DISK -> INSERT OR REPLACE INTO disk.global_variables * SAVE MCP VARIABLES FROM MEMORY/MEM -> INSERT OR REPLACE INTO disk.global_variables - Separated DISK command handlers from MEMORY/RUNTIME handlers 3. lib/ProxySQL_Admin.cpp: - Added flush_mcp_variables___runtime_to_database() call to stats section to ensure MCP variables are repopulated when runtime_global_variables is cleared and refreshed 4. tests/mcp_module-t.cpp: - Added verbose diagnostic output throughout tests - Added section headers and test numbers for clarity - Added variable value logging and error logging All 52 MCP module tests now pass.	1 month ago
Rene Cannao	81c53896bc	Fix MCP module TAP test failures - Add MCP variables to load_save_disk_commands map for LOAD/SAVE commands - Add MCP variable validation in is_valid_global_variable() for SET commands - Implement has_variable() method in MCP_Threads_Handler - Add CHECKSUM command handlers for MCP VARIABLES (DISK/MEMORY/MEM) Test results improved from 28 passed / 16 failed to 49 passed / 3 failed. Remaining 3 failures are test expectation issues (boolean representation).	1 month ago
Rene Cannao	87fff9e046	Add MCP (Model Context Protocol) module skeleton Add new MCP module supporting multiple MCP server endpoints over HTTPS with JSON-RPC 2.0 protocol skeleton. Each endpoint (/mcp/config, /mcp/observe, /mcp/query, /mcp/admin, /mcp/cache) is a distinct MCP server with its own authentication configuration. Features: - HTTPS server using existing ProxySQL TLS certificates - JSON-RPC 2.0 skeleton implementation (actual protocol TBD) - 5 MCP endpoints with per-endpoint auth configuration - LOAD/SAVE MCP VARIABLES admin commands - Configuration file support (mcp_variables section) Implementation follows GenAI module pattern: - MCP_Threads_Handler: Main module handler with variable management - ProxySQL_MCP_Server: HTTPS server wrapper using libhttpserver - MCP_JSONRPC_Resource: Base endpoint class with JSON-RPC skeleton	1 month ago
Rene Cannao	c476f56f97	Add initial GenAI module placeholder Implement a new GenAI module for ProxySQL with basic infrastructure: - GenAI_Threads_Handler class for managing GenAI module configuration - Support for genai- prefixed variables in global_variables table - Dummy variables: genai-var1 (string) and genai-var2 (integer) - Config file support via genai_variables section - Flush functions for runtime_to_database and database_to_runtime - Module lifecycle: initialization at startup, graceful shutdown - LOAD/SAVE GENAI VARIABLES admin command infrastructure Core functionality verified: - Config file loading works - Variables persist in global_variables table - Disk save/load via SQL works - Module initializes and shuts down properly Related files: - include/GenAI_Thread.h: New GenAI thread handler class - lib/GenAI_Thread.cpp: Implementation with dummy variables - lib/Admin_Handler.cpp: Added GENAI command vectors and handlers - lib/Admin_FlushVariables.cpp: Added genai flush functions - lib/ProxySQL_Admin.cpp: Added init_genai_variables() and load_save_disk_commands entry - include/proxysql_admin.h: Added function declarations - lib/Makefile: Added GenAI_Thread.oo to build - src/main.cpp: Added module initialization and cleanup - src/proxysql.cfg: Added genai_variables configuration section	2 months ago
Rene Cannao	efe0d4fe61	Add extensive doxygen documentation for vacuum_stats and stats_pgsql_stat_activity This commit documents: 1. The vacuum_stats() function's purpose, behavior, and the reason why stats_pgsql_stat_activity is excluded from bulk deletion operations 2. The fact that stats_pgsql_stat_activity is a SQL VIEW (not a table) and attempting DELETE on it would cause SQLite error: "cannot modify stats_pgsql_stat_activity because it is a view" The documentation explains: - Why TRUNCATE stats_mysql_query_digest triggers vacuum_stats(true) - Why both MySQL and PostgreSQL tables are cleared regardless of protocol - How the view is automatically cleared via its underlying table stats_pgsql_processlist - The importance of keeping the view excluded from deletion lists	2 months ago
Rahim Kanji	38befb845f	Fixed PROXYSQL STOP	4 months ago
Wazir Ahmed	5219d4be93	processlist: Prevent crash due to SQLITE_TOOBIG - Add new mysql/pgsql variable `processlist_max_query_length`. - Min: 1K - Max: 32M - Default: 2M - Truncate current query based on the configuration before inserting into `stats_*_processlist` tables. - Refactor/fix code related to other processlist configurations. 1. `session_idle_show_processlist` value was not updated in `ProxySQL_Admin.variables`. 2. Pass processlist config as an argument to `MySQL_Threads_Handler::SQL3_Processlist` instead of using thread-local variables. Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	4 months ago
René Cannaò	ecad6f9249	Merge pull request #5160 from sysown/v3.0_disable_pgsql_prometheus_5159 Separate Query Cache Metrics for MySQL and PostgreSQL & Disable Prometheus Metrics for PostgreSQL	4 months ago
René Cannaò	abafaccbc0	Merge pull request #5156 from sysown/v3.0_pgsql_stat_activity_issue_4682 Add backend_pid and backend_state to stats_pgsql_processlist and create stats_pgsql_stat_activity view	4 months ago
Rahim Kanji	5bfb6e216c	Few fixes	4 months ago
Rahim Kanji	e6d0338959	Turning off Prometheus metrics collection for PostgreSQL modules in ProxySQL	4 months ago
Rahim Kanji	fc4d7f76a0	Added 'SHOW FULL PGSQL ACTIVITY' and 'SHOW PGSQL ACTIVITY' handling	4 months ago
Rahim Kanji	f3ea179384	Add backend PID/state to stats_pgsql_processlist and create pg_stat_activity-style view - Added `backend_pid` and `backend_state` columns to `stats_pgsql_processlist` to display PostgreSQL backend process ID and connection state. - Created `stats_pgsql_stat_activity` view on top of `stats_pgsql_processlist` with column aliases matching PostgreSQL's `pg_stat_activity` for consistency.	4 months ago
Rahim Kanji	1e988c665e	Removed admin_handler_command_kill_connection	5 months ago
Rahim Kanji	703a5f35f5	Added stats_pgsql_prepared_statements_info table	7 months ago
Rahim Kanji	bc87d0d901	* For a Describe Portal message, peek at the next message in the extended query frame. If it is an Execute message, indicating that client is sending a Bind/Describe/Execute sequence skip processing the Describe Portal. This is because libpq automatically includes a Describe Portal during execution, and sending it again would be redundant. * Removed server_capabilities from PgSQL modules	7 months ago
René Cannaò	c48c3310a6	Check if database file exists before checking if it is writable Related to `bf3cca8639`	11 months ago
René Cannaò	bf3cca8639	exit if database file is not writable If database file (proxysql.db) is not writable, exit immediately instead of crashing because it is not writable. See issue #4893	11 months ago
René Cannaò	cb53db81ca	Export MySQL_Logger metrics for v3.0 Port of #4802	12 months ago
René Cannaò	2551d350e9	Port to 3.0 of in-memory query logging See #4713 for further details	12 months ago
Javier Jaramago Fernández	9e941854c3	Replace table based 'VACUUM' for SQLite3 'stats' database Introduced generic approach based on the potential benefits for 'VACUUM' over the 'stats' database instead of the previous selective, table based, approach.	12 months ago
René Cannaò	b9dcc9420a	PROXYSQL FLUSH LOGS forces debug database flush Also increased barching size from 31 to 100	1 year ago
René Cannaò	c2f1871adf	Make set_thread_name() optional	1 year ago
Rahim Kanji	6d614a4b99	Merge branch 'v3.0'	1 year ago
Rahim Kanji	110ce8d511	Fixed crash caused by sending single semicolon (;) on admin interface	1 year ago
Rahim Kanji	720441af20	Refactored and Optimized Query Cache with PgSQL Support * Added Query Cache Support for PgSQL: * Introduced new variables for PgSQL Query Cache management: * pgsql-query_cache_size_MB * pgsql-query_cache_soft_ttl_pct * pgsql-query_cache_stores_empty_result * pgsql-query_cache_handle_warnings (not yet utilized) * Overwriting Transaction Status: * Sending current transaction state to the client when a result is returned from the query cache. * This currently does not differentiate between normal and error transaction states—both are treated as active transaction states. * Cache Lifetime Management: Replaced manual reference counting with std::shared_ptr to efficiently manage the lifecycle of cache entries, ensuring proper memory deallocation and reducing complexity. * Unified Purging Logic: Refactored the query cache purging logic to use a single thread for both MySQL and PgSQL cache entries. * Code Cleanup: Removed unnecessary methods and data members to streamline the codebase and improve maintainability. * Performance Optimization: Improved memory management, reducing memory footprint and enhancing performance.	1 year ago
Rahim Kanji	0229daeb64	Added stats_pgsql_query_digest and stats_pgsql_query_digest_reset tables	1 year ago
René Cannaò	d4bd93672e	Merge pull request #79 from sysown/v2.x_refactore_query_processor Refactored Query Processor [WIP]	1 year ago
Rahim Kanji	e723082a1b	Refactored Query Processor Architecture - Introduced separate query processors for MySQL and PostgreSQL. - Removed unnecessary GTID logic from `PgSQL_Session`. - Added a command counter for PostgreSQL queries. - Refactored `Query_Processor` to serve as a base class containing core functionalities applicable to both MySQL and PostgreSQL. Changes: - Removed GTID-related code from `PgSQL_Logger` and `PgSQL_Session`: - bool with_gtid; - char gtid_buf[128]; - int gtid_hid; - Added new PostgreSQL-specific tables: - `pgsql_query_rules`, `pgsql_firewall_whitelist_rules`, `pgsql_query_rules_fast_routing` - `runtime_pgsql_query_rules`, `runtime_pgsql_firewall_whitelist_rules`, `runtime_pgsql_query_rules_fast_routing` - Added PostgreSQL-related variables: pgsql_thread___commands_stats pgsql_thread___query_digests pgsql_thread___query_digests_lowercase pgsql_thread___query_digests_no_digits pgsql_thread___query_digests_replace_null pgsql_thread___query_digests_normalize_digest_text pgsql_thread___query_digests_track_hostname pgsql_thread___query_digests_keep_comment pgsql_thread___query_digests_max_digest_length pgsql_thread___query_digests_max_query_length pgsql_thread___query_digests_grouping_limit pgsql_thread___query_digests_groups_grouping_limit pgsql_thread___enable_load_data_local_infile pgsql_thread___auditlog_filename pgsql_thread___auditlog_filesize pgsql_thread___eventslog_filename pgsql_thread___eventslog_filesize pgsql_thread___eventslog_default_log pgsql_thread___eventslog_format pgsql_thread___firewall_whitelist_errormsg pgsql_thread___firewall_whitelist_enabled pgsql_thread___query_processor_iterations pgsql_thread___query_processor_regex	1 year ago
René Cannaò	cbf4628ea3	Merge remote-tracking branch 'proxysql27/v2.7' into v2.x_postgres_merge27_20240917	1 year ago
Javier Jaramago Fernández	d516ca5e84	Merge branch 'v2.x_postgres' of https://github.com/sysown/proxysql_postgresql into v2.x-postgres_dual_password	1 year ago
René Cannaò	cf9d5fe21a	Merge pull request #4574 from sysown/set-thread-names Set thread names	1 year ago
Rahim Kanji	745449fcbc	Removed Client_Session.h and Client_Session.cpp	2 years ago
Javier Jaramago Fernández	bbda0a86eb	Add initial impl for dual-password support	2 years ago
Rahim Kanji	5bf827d8e4	Removed type selector	2 years ago
Rahim Kanji	3112a56da4	Added 'stats_pgsql_client_host_cache' and 'stats_pgsql_client_host_cache_reset' tables.	2 years ago
Rahim Kanji	e939cf088e	Added 'stats_pgsql_errors' and 'stats_pgsql_errors_reset' tables	2 years ago

1 2 3 4 5 ...

946 Commits (e4f4dc95cef878de7b2b7cfdfeb8e9ee8aeb0c46)