proxysql

Commit Graph

Author	SHA1	Message	Date
Rene Cannao	3fe8a48f70	Fix genai variable handling and add API key masking - Add has_variable() method to GenAI_Threads_Handler for variable validation - Add genai- prefix check in is_valid_global_variable() - Auto-initialize NL2SQL converter when genai-nl2sql_enabled is set to true at runtime - Make init_nl2sql() public to allow runtime initialization - Mask API keys in logs (show only first 2 chars, rest as 'x')	1 month ago
Rene Cannao	6ffb59b856	fix: Use db parameter instead of hardcoded admindb in GenAI database_to_runtime The flush_genai_variables___database_to_runtime() function was using hardcoded 'admindb' instead of the 'db' parameter passed to the function. This caused the function to always query from admindb, ignoring the actual database parameter. This fixes the issue where runtime_global_variables table was not being populated on startup because the query was always hitting the same database regardless of the parameter.	1 month ago
Rene Cannao	4018a0ad3b	fix: Follow MCP pattern for GenAI variables runtime table population Update flush_genai_variables___database_to_runtime() to match the MCP pattern exactly: - Add 'lock' parameter (default true) for flexibility - Use ProxySQL_Admin's wrlock()/wrunlock() instead of GloGATH's - Use consistent variable naming (var_name = name + 6 for 'genai-' prefix) - Follow exact same locking pattern as MCP variables This fixes the issue where runtime_global_variables table was not being populated on startup because the locking pattern was incorrect.	1 month ago
Rene Cannao	1ea67900ab	fix: Populate runtime_global_variables for GenAI variables on startup The flush_genai_variables___database_to_runtime() function was only setting internal state in GloGATH but not populating the runtime_global_variables table. This caused variables to appear in global_variables but not in runtime_global_variables after startup. Fix: Add call to flush_genai_variables___runtime_to_database() with runtime=true to populate the runtime table, following the same pattern used by MCP variables.	1 month ago
René Cannaò	313f637cf0	Merge branch 'v3.1-vec' into v3.1-MCP1 Signed-off-by: René Cannaò <rene.cannao@gmail.com>	1 month ago
Rene Cannao	991f0138d8	Reinitialize MySQL Tool Handler when MCP variables change When LOAD MCP VARIABLES TO RUNTIME is called and the MCP server is already running, the MySQL Tool Handler is now recreated with the current configuration values. This allows changing MySQL connection parameters without restarting ProxySQL. The reinitialization: 1. Deletes the old MySQL Tool Handler 2. Creates a new one with current mcp-mysql_* values 3. Initializes the new handler 4. Logs success or failure	1 month ago
Rene Cannao	60d4a7378c	Implement automatic MCP server start/stop and add environment variable support - Add automatic MCP HTTPS server start/stop based on mcp-enabled flag - Server starts when mcp-enabled=true and LOAD MCP VARIABLES TO RUNTIME - Server stops when mcp-enabled=false and LOAD MCP VARIABLES TO RUNTIME - Validates SSL certificates before starting - Added to both flush_mcp_variables___database_to_runtime() and flush_mcp_variables___runtime_to_database() functions - Update configure_mcp.sh to respect environment variables - MYSQL_HOST, MYSQL_PORT, MYSQL_USER, MYSQL_PASSWORD - TEST_DB_NAME (mapped to MYSQL_DATABASE) - MCP_PORT - Updated --help documentation with all supported variables	1 month ago
Rene Cannao	5a85ef04f6	Fix MCP variables persistence and add DISK command support This commit fixes several issues with MCP (Model Context Protocol) variables not being properly persisted across storage layers and adds support for DISK commands. Changes: 1. lib/Admin_FlushVariables.cpp: - Fixed flush_mcp_variables___runtime_to_database() to properly insert variables into runtime_global_variables using db->execute() with formatted strings (matching admin pattern) - Fixed SQL format string to avoid double-prefix bug (qualified_name already contains "mcp-" prefix) - Fixed lock ordering by releasing outer wrlock before calling runtime_to_database with use_lock=true, then re-acquiring - Removed explicit BEGIN/COMMIT transactions to match admin pattern 2. lib/Admin_Handler.cpp: - Added MCP DISK command handlers that rewrite commands to SQL queries: * LOAD MCP VARIABLES FROM DISK -> INSERT OR REPLACE INTO main.global_variables * SAVE MCP VARIABLES TO DISK -> INSERT OR REPLACE INTO disk.global_variables * SAVE MCP VARIABLES FROM MEMORY/MEM -> INSERT OR REPLACE INTO disk.global_variables - Separated DISK command handlers from MEMORY/RUNTIME handlers 3. lib/ProxySQL_Admin.cpp: - Added flush_mcp_variables___runtime_to_database() call to stats section to ensure MCP variables are repopulated when runtime_global_variables is cleared and refreshed 4. tests/mcp_module-t.cpp: - Added verbose diagnostic output throughout tests - Added section headers and test numbers for clarity - Added variable value logging and error logging All 52 MCP module tests now pass.	1 month ago
Rene Cannao	b70b07ead7	Skip checksum generation for MCP until feature is complete The checksum generation caused an assert failure because the MCP module was not yet added to the checksums_values struct. For now, we skip checksum generation for MCP until the feature is complete and stable. Changes: - Removed flush_GENERIC_variables__checksum__database_to_runtime() call - Kept flush_mcp_variables___runtime_to_database() to populate runtime_global_variables - Added comment explaining checksum is skipped until MCP is complete This allows ProxySQL to start without crashing while MCP is under development.	1 month ago
Rene Cannao	2e7109d894	Fix lock ordering in flush_mcp_variables___database_to_runtime The crash was caused by incorrect lock ordering. The admin version has: 1. wrlock() (acquire admin lock) 2. Process variables 3. checksum_mutex lock() (acquire checksum lock) 4. flush to runtime + generate checksum 5. checksum_mutex unlock() (release checksum lock) 6. wrunlock() (release admin lock) The MCP version had the wrong order with the checksum_mutex lock outside the wrlock/wrunlock region. This also added the missing 'lock' parameter that exists in the admin version but was missing in MCP. Changes: - Added 'lock' parameter to flush_mcp_variables___database_to_runtime() - Added conditional wrlock()/wrunlock() calls (if lock=true) - Moved checksum generation inside the wrlock/wrunlock region - Updated function signature in header file	1 month ago
Rene Cannao	2874c9ad54	Fix flush_mcp_variables___database_to_runtime to populate runtime_global_variables The MCP module's flush_mcp_variables___database_to_runtime() was missing the logic to populate runtime_global_variables table. This caused the table to remain empty even though global_variables was correctly populated. Following the same pattern as admin variables (line 268), this commit adds: 1. Call to flush_mcp_variables___runtime_to_database(admindb, ..., true) to populate runtime_global_variables 2. Checksum generation for cluster sync After this fix, both global_variables and runtime_global_variables will contain MCP variables after ProxySQL startup.	1 month ago
Rene Cannao	87fff9e046	Add MCP (Model Context Protocol) module skeleton Add new MCP module supporting multiple MCP server endpoints over HTTPS with JSON-RPC 2.0 protocol skeleton. Each endpoint (/mcp/config, /mcp/observe, /mcp/query, /mcp/admin, /mcp/cache) is a distinct MCP server with its own authentication configuration. Features: - HTTPS server using existing ProxySQL TLS certificates - JSON-RPC 2.0 skeleton implementation (actual protocol TBD) - 5 MCP endpoints with per-endpoint auth configuration - LOAD/SAVE MCP VARIABLES admin commands - Configuration file support (mcp_variables section) Implementation follows GenAI module pattern: - MCP_Threads_Handler: Main module handler with variable management - ProxySQL_MCP_Server: HTTPS server wrapper using libhttpserver - MCP_JSONRPC_Resource: Base endpoint class with JSON-RPC skeleton	1 month ago
Rene Cannao	c476f56f97	Add initial GenAI module placeholder Implement a new GenAI module for ProxySQL with basic infrastructure: - GenAI_Threads_Handler class for managing GenAI module configuration - Support for genai- prefixed variables in global_variables table - Dummy variables: genai-var1 (string) and genai-var2 (integer) - Config file support via genai_variables section - Flush functions for runtime_to_database and database_to_runtime - Module lifecycle: initialization at startup, graceful shutdown - LOAD/SAVE GENAI VARIABLES admin command infrastructure Core functionality verified: - Config file loading works - Variables persist in global_variables table - Disk save/load via SQL works - Module initializes and shuts down properly Related files: - include/GenAI_Thread.h: New GenAI thread handler class - lib/GenAI_Thread.cpp: Implementation with dummy variables - lib/Admin_Handler.cpp: Added GENAI command vectors and handlers - lib/Admin_FlushVariables.cpp: Added genai flush functions - lib/ProxySQL_Admin.cpp: Added init_genai_variables() and load_save_disk_commands entry - include/proxysql_admin.h: Added function declarations - lib/Makefile: Added GenAI_Thread.oo to build - src/main.cpp: Added module initialization and cleanup - src/proxysql.cfg: Added genai_variables configuration section	2 months ago
Rene Cannao	6ed82ef8cb	Fix crash in TCP keepalive warnings for issue #5212 - Fix crash by using get_variable_int() instead of get_variable_string() for boolean use_tcp_keepalive variable - use_tcp_keepalive is a boolean variable, not a string, so get_variable_int() returns 0/1 instead of a string - Fix syntax errors by removing duplicate code and fixing brace structure - Add comprehensive Doxygen documentation for both MySQL and PostgreSQL warnings Resolves assertion failure: "Not existing variable: use_tcp_keepalive" Resolves: #5212	3 months ago
Rene Cannao	cf454b8ef2	Add TCP keepalive warnings for issue #5212 - Add warnings in flush_mysql_variables___database_to_runtime() when mysql-use_tcp_keepalive=false - Add warnings in flush_pgsql_variables___database_to_runtime() when pgsql-use_tcp_keepalive=false - Include comprehensive Doxygen documentation explaining why disabling TCP keepalive is unsafe - Warn users about potential connection drops when ProxySQL is deployed behind network load balancers When TCP keepalive is disabled: - Load balancers may drop idle connections from connection pools - NAT devices may remove connection state - Cloud load balancers may terminate connections during idle periods - Results in sudden connection failures and "connection reset" errors Resolves: #5212	3 months ago
Wazir Ahmed	5219d4be93	processlist: Prevent crash due to SQLITE_TOOBIG - Add new mysql/pgsql variable `processlist_max_query_length`. - Min: 1K - Max: 32M - Default: 2M - Truncate current query based on the configuration before inserting into `stats_*_processlist` tables. - Refactor/fix code related to other processlist configurations. 1. `session_idle_show_processlist` value was not updated in `ProxySQL_Admin.variables`. 2. Pass processlist config as an argument to `MySQL_Threads_Handler::SQL3_Processlist` instead of using thread-local variables. Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	4 months ago
Rahim Kanji	bc6f95a1fb	Removed previous_default_client_encoding variable	8 months ago
Rahim Kanji	06e496b7c3	Added tracking of parameters: * client_encoding/names * datestyle * timezone/time zone * standard_conforming_strings	1 year ago
Rahim Kanji	720441af20	Refactored and Optimized Query Cache with PgSQL Support * Added Query Cache Support for PgSQL: * Introduced new variables for PgSQL Query Cache management: * pgsql-query_cache_size_MB * pgsql-query_cache_soft_ttl_pct * pgsql-query_cache_stores_empty_result * pgsql-query_cache_handle_warnings (not yet utilized) * Overwriting Transaction Status: * Sending current transaction state to the client when a result is returned from the query cache. * This currently does not differentiate between normal and error transaction states—both are treated as active transaction states. * Cache Lifetime Management: Replaced manual reference counting with std::shared_ptr to efficiently manage the lifecycle of cache entries, ensuring proper memory deallocation and reducing complexity. * Unified Purging Logic: Refactored the query cache purging logic to use a single thread for both MySQL and PgSQL cache entries. * Code Cleanup: Removed unnecessary methods and data members to streamline the codebase and improve maintainability. * Performance Optimization: Improved memory management, reducing memory footprint and enhancing performance.	1 year ago
Rahim Kanji	0229daeb64	Added stats_pgsql_query_digest and stats_pgsql_query_digest_reset tables	1 year ago
Rahim Kanji	e723082a1b	Refactored Query Processor Architecture - Introduced separate query processors for MySQL and PostgreSQL. - Removed unnecessary GTID logic from `PgSQL_Session`. - Added a command counter for PostgreSQL queries. - Refactored `Query_Processor` to serve as a base class containing core functionalities applicable to both MySQL and PostgreSQL. Changes: - Removed GTID-related code from `PgSQL_Logger` and `PgSQL_Session`: - bool with_gtid; - char gtid_buf[128]; - int gtid_hid; - Added new PostgreSQL-specific tables: - `pgsql_query_rules`, `pgsql_firewall_whitelist_rules`, `pgsql_query_rules_fast_routing` - `runtime_pgsql_query_rules`, `runtime_pgsql_firewall_whitelist_rules`, `runtime_pgsql_query_rules_fast_routing` - Added PostgreSQL-related variables: pgsql_thread___commands_stats pgsql_thread___query_digests pgsql_thread___query_digests_lowercase pgsql_thread___query_digests_no_digits pgsql_thread___query_digests_replace_null pgsql_thread___query_digests_normalize_digest_text pgsql_thread___query_digests_track_hostname pgsql_thread___query_digests_keep_comment pgsql_thread___query_digests_max_digest_length pgsql_thread___query_digests_max_query_length pgsql_thread___query_digests_grouping_limit pgsql_thread___query_digests_groups_grouping_limit pgsql_thread___enable_load_data_local_infile pgsql_thread___auditlog_filename pgsql_thread___auditlog_filesize pgsql_thread___eventslog_filename pgsql_thread___eventslog_filesize pgsql_thread___eventslog_default_log pgsql_thread___eventslog_format pgsql_thread___firewall_whitelist_errormsg pgsql_thread___firewall_whitelist_enabled pgsql_thread___query_processor_iterations pgsql_thread___query_processor_regex	1 year ago
Rahim Kanji	745449fcbc	Removed Client_Session.h and Client_Session.cpp	2 years ago
Rene Cannao	a1fd4af1a6	Splitting ProxySQL_Admin	2 years ago

23 Commits (3fe8a48f705bea9bb77c3255e54bbc1ace125ebb)