proxysql

Commit Graph

Author	SHA1	Message	Date
René Cannaò	ce9fdc9d7a	Merge branch 'v3.0' into v3.0.6-implement_FLUSH_STATS Signed-off-by: René Cannaò <rene@proxysql.com>	2 months ago
Rene Cannao	48bc7dd7bf	Merge v4.0 GenAI features into v3.0 with conditional compilation This commit merges the experimental v4.0 GenAI/MCP features into the stable v3.0 branch using conditional compilation. All v4.0 features are disabled by default and only enabled when PROXYSQLGENAI=1 is set at compile time. Changes: Build System: - Modified main Makefile to pass PROXYSQLGENAI flag to sub-makefiles - Modified deps/Makefile to conditionally build sqlite-vec and sqlite-rembed - Modified lib/Makefile to add PSQLGA flag and include GenAI object files - Modified src/Makefile to add PSQLGA flag and conditional linking Headers (wrapped with #ifdef PROXYSQLGENAI): - All 20 new GenAI header files in include/ - Modified cpp.h, proxysql_glovars.hpp, proxysql_admin.h - Modified ProxySQL_Admin_Tables_Definitions.h for GenAI/MCP tables Source Files: - All 22 new GenAI source files in lib/ wrapped with #ifdef PROXYSQLGENAI - Modified src/main.cpp for conditional global variables and init/shutdown - Modified Admin_Handler.cpp for conditional command handlers - Modified Admin_Bootstrap.cpp for conditional table registration - Modified Admin_FlushVariables.cpp for conditional variable flushing - Modified ProxySQL_Admin.cpp for conditional admin methods - Modified ProxySQL_Admin_Stats.cpp for conditional MCP stats functions - Modified proxy_sqlite3_symbols.cpp to always compile (needed by core) - Modified MySQL_Session.cpp for conditional GenAI function calls Test Files: - Renamed test_mcp_query_rules-t to test_mcp_query_rules-t.sh - Renamed test_mcp_rag_metrics-t to test_mcp_rag_metrics-t.sh - Modified anomaly_detection-t.cpp for conditional test execution Usage: # Build without GenAI (v3.0 mode - default) make clean && make build_deps -j$(nproc) && make build_lib -j$(nproc) && make build_src -j$(nproc) # Build with GenAI (v4.0 mode) make clean && PROXYSQLGENAI=1 make build_deps -j$(nproc) && PROXYSQLGENAI=1 make build_lib -j$(nproc) && PROXYSQLGENAI=1 make build_src -j$(nproc)	2 months ago
Miro Stauder	cc1d93d08b	implement admin command 'PROXYSQL FLUSH STATS' for DEBUG builds	2 months ago
Wazir Ahmed	50de536534	MCP: Fix crash during server restarts Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	2 months ago
Rene Cannao	b4f521c634	Merge v3.1-MCP2 into v3.1-vec Conflict resolution summary: Features preserved from v3.1-vec: - mcp_use_ssl variable (HTTP/HTTPS mode support) - RAG tool handler support - FTS (Full Text Search) functionality in MySQL_Tool_Handler Changes accepted from v3.1-MCP2: - Schema isolation for catalog (removed mcp_catalog_path/fts_path variables) - Schema parameter added to catalog functions - Comprehensive query rewriting improvements in Query_Tool_Handler - Format string fixes in vector_db_performance-t.cpp Key decisions: - catalog_path/fts_path: Removed per v3.1-MCP2 commit `35b0b224` (intentional) - MySQL_Catalog.cpp: Accepted v3.1-MCP2 version (schema isolation improvements) - Query_Tool_Handler.cpp: Accepted v3.1-MCP2 version (significant improvements) - Makefile: Kept v3.1-vec's EXCLUDE_TRACKING_VARIABLES (correct spelling) Build notes: - MySQL_Tool_Handler now requires schema parameter for catalog operations - Catalog path is hardcoded to datadir/mcp_catalog.db - FTS functionality preserved but may need updates for schema isolation	3 months ago
René Cannaò	75a62f334d	Merge branch 'v4.0' into v3.1-vec Signed-off-by: René Cannaò <rene@proxysql.com>	3 months ago
Rene Cannao	cc3cc25532	fix: Remove unused reset parameter from stats___mcp_query_rules() Change function signature from stats___mcp_query_rules(bool reset) to stats___mcp_query_rules() to match MySQL query rules pattern. The reset parameter was never used in the function body and MySQL's stats___mysql_query_rules() has no reset parameter.	3 months ago
Rene Cannao	f01fc79584	feat: Add runtime_mcp_query_rules table and fix stats_mcp_query_rules schema - Add ADMIN_SQLITE_TABLE_RUNTIME_MCP_QUERY_RULES schema (17 columns, same as mcp_query_rules) - Fix STATS_SQLITE_TABLE_MCP_QUERY_RULES to only have rule_id and hits columns - Add runtime_mcp_query_rules detection and refresh in ProxySQL_Admin - Implement save_mcp_query_rules_from_runtime(bool _runtime) for both config and runtime tables - Update get_mcp_query_rules() to return 17 columns (no hits) - get_stats_mcp_query_rules() returns 2 columns (rule_id, hits) Mirrors the MySQL query rules pattern: - mcp_query_rules: config table (17 cols) - runtime_mcp_query_rules: runtime state (17 cols) - stats_mcp_query_rules: hit counters (2 cols)	3 months ago
Rene Cannao	fb66af7c1b	feat: Expose MCP catalog database in ProxySQL Admin interface The MCP catalog database is now accessible as the 'mcp_catalog' schema from the ProxySQL Admin interface, enabling direct SQL queries against discovered schemas and LLM memories.	3 months ago
Rene Cannao	a816a756d4	feat: Add MCP query tool usage counters to stats schema Add stats_mcp_query_tools_counters and stats_mcp_query_tools_counters_reset tables to track MCP query tool usage statistics. - Added get_tool_usage_stats_resultset() method to Query_Tool_Handler - Defined table schemas in ProxySQL_Admin_Tables_Definitions.h - Registered tables in Admin_Bootstrap.cpp - Added pattern matching in ProxySQL_Admin.cpp - Added stats___mcp_query_tools_counters() in ProxySQL_Admin_Stats.cpp - Fixed friend declaration for track_tool_invocation() - Fixed Discovery_Schema.cpp log_llm_search() to use prepare_v2/finalize	3 months ago
Rene Cannao	4018a0ad3b	fix: Follow MCP pattern for GenAI variables runtime table population Update flush_genai_variables___database_to_runtime() to match the MCP pattern exactly: - Add 'lock' parameter (default true) for flexibility - Use ProxySQL_Admin's wrlock()/wrunlock() instead of GloGATH's - Use consistent variable naming (var_name = name + 6 for 'genai-' prefix) - Follow exact same locking pattern as MCP variables This fixes the issue where runtime_global_variables table was not being populated on startup because the locking pattern was incorrect.	3 months ago
René Cannaò	313f637cf0	Merge branch 'v3.1-vec' into v3.1-MCP1 Signed-off-by: René Cannaò <rene.cannao@gmail.com>	3 months ago
Rene Cannao	2e7109d894	Fix lock ordering in flush_mcp_variables___database_to_runtime The crash was caused by incorrect lock ordering. The admin version has: 1. wrlock() (acquire admin lock) 2. Process variables 3. checksum_mutex lock() (acquire checksum lock) 4. flush to runtime + generate checksum 5. checksum_mutex unlock() (release checksum lock) 6. wrunlock() (release admin lock) The MCP version had the wrong order with the checksum_mutex lock outside the wrlock/wrunlock region. This also added the missing 'lock' parameter that exists in the admin version but was missing in MCP. Changes: - Added 'lock' parameter to flush_mcp_variables___database_to_runtime() - Added conditional wrlock()/wrunlock() calls (if lock=true) - Moved checksum generation inside the wrlock/wrunlock region - Updated function signature in header file	3 months ago
Rene Cannao	87fff9e046	Add MCP (Model Context Protocol) module skeleton Add new MCP module supporting multiple MCP server endpoints over HTTPS with JSON-RPC 2.0 protocol skeleton. Each endpoint (/mcp/config, /mcp/observe, /mcp/query, /mcp/admin, /mcp/cache) is a distinct MCP server with its own authentication configuration. Features: - HTTPS server using existing ProxySQL TLS certificates - JSON-RPC 2.0 skeleton implementation (actual protocol TBD) - 5 MCP endpoints with per-endpoint auth configuration - LOAD/SAVE MCP VARIABLES admin commands - Configuration file support (mcp_variables section) Implementation follows GenAI module pattern: - MCP_Threads_Handler: Main module handler with variable management - ProxySQL_MCP_Server: HTTPS server wrapper using libhttpserver - MCP_JSONRPC_Resource: Base endpoint class with JSON-RPC skeleton	3 months ago
Rene Cannao	c476f56f97	Add initial GenAI module placeholder Implement a new GenAI module for ProxySQL with basic infrastructure: - GenAI_Threads_Handler class for managing GenAI module configuration - Support for genai- prefixed variables in global_variables table - Dummy variables: genai-var1 (string) and genai-var2 (integer) - Config file support via genai_variables section - Flush functions for runtime_to_database and database_to_runtime - Module lifecycle: initialization at startup, graceful shutdown - LOAD/SAVE GENAI VARIABLES admin command infrastructure Core functionality verified: - Config file loading works - Variables persist in global_variables table - Disk save/load via SQL works - Module initializes and shuts down properly Related files: - include/GenAI_Thread.h: New GenAI thread handler class - lib/GenAI_Thread.cpp: Implementation with dummy variables - lib/Admin_Handler.cpp: Added GENAI command vectors and handlers - lib/Admin_FlushVariables.cpp: Added genai flush functions - lib/ProxySQL_Admin.cpp: Added init_genai_variables() and load_save_disk_commands entry - include/proxysql_admin.h: Added function declarations - lib/Makefile: Added GenAI_Thread.oo to build - src/main.cpp: Added module initialization and cleanup - src/proxysql.cfg: Added genai_variables configuration section	3 months ago
Rene Cannao	442635b721	Add comprehensive documentation for SSL/TLS key logging feature This commit adds extensive documentation for the ssl_keylog_file feature (introduced in PR #4236), which enables TLS key logging for debugging encrypted traffic. ## Background The ssl_keylog_file variable (exposed as admin-ssl_keylog_file in SQL interface) allows ProxySQL to write TLS secrets to a file in NSS Key Log Format. These secrets can be used by tools like Wireshark and tshark to decrypt and analyze TLS traffic for debugging purposes. ## Changes ### Inline Documentation (Code) 1. include/proxysql_sslkeylog.h (+96 lines) - File-level documentation explaining the module purpose and security - Doxygen comments for all 5 public APIs - Thread-safety annotations - Parameter descriptions and return values 2. lib/proxysql_sslkeylog.cpp (+136 lines) - Implementation-level documentation - Algorithm explanations (double-checked locking, thread safety) - Reference to NSS Key Log Format specification 3. include/proxysql_admin.h (+19 lines) - Variable documentation for ssl_keylog_file - Path handling rules (absolute vs relative) - Security implications ### Developer Documentation (doc/ssl_keylog/ssl_keylog_developer_guide.md) Target audience: Developers working on ProxySQL codebase Contents: - Variable naming convention (SQL vs config file vs internal) - Architecture diagrams - Thread safety model (pthread rwlock) - NSS Key Log Format specification - Complete API reference for all public functions - Integration points in the codebase - Security considerations and code review checklist - Testing procedures ### User Documentation (doc/ssl_keylog/ssl_keylog_user_guide.md) Target audience: End users and system administrators Contents: - What is SSL key logging and when to use it - Variable naming: admin-ssl_keylog_file (SQL) vs ssl_keylog_file (config) - Step-by-step enable/disable instructions - Path resolution (absolute vs relative) - Log rotation procedures - Production workflow: tcpdump capture → offline analysis - Wireshark (GUI) integration tutorial - tshark (command-line) usage examples - Troubleshooting common issues - Security best practices - Quick reference card ## Key Features Documented 1. Variable Naming Convention - SQL interface: SET admin-ssl_keylog_file = '/path'; - Config file: ssl_keylog_file='/path' (in admin_variables section) - Internal code: ssl_keylog_file 2. Production Workflow - Capture traffic with tcpdump (no GUI on production server) - Transfer pcap + keylog to analysis system - Analyze offline with Wireshark (GUI) or tshark (CLI) 3. tshark Examples - Command-line analysis of encrypted traffic - Filter examples for debugging TLS issues - JSON export for automated analysis ## Security Notes The documentation emphasizes that: - Key log files contain cryptographic secrets that decrypt ALL TLS traffic - Access must be restricted (permissions 0600) - Only enable for debugging, never in production - Securely delete old key log files ## Files Modified - include/proxysql_admin.h - include/proxysql_sslkeylog.h - lib/proxysql_sslkeylog.cpp ## Files Added - doc/ssl_keylog/ssl_keylog_developer_guide.md - doc/ssl_keylog/ssl_keylog_user_guide.md	3 months ago
Wazir Ahmed	5219d4be93	processlist: Prevent crash due to SQLITE_TOOBIG - Add new mysql/pgsql variable `processlist_max_query_length`. - Min: 1K - Max: 32M - Default: 2M - Truncate current query based on the configuration before inserting into `stats_*_processlist` tables. - Refactor/fix code related to other processlist configurations. 1. `session_idle_show_processlist` value was not updated in `ProxySQL_Admin.variables`. 2. Pass processlist config as an argument to `MySQL_Threads_Handler::SQL3_Processlist` instead of using thread-local variables. Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	6 months ago
René Cannaò	b5fa8ca1ae	Merge pull request #5044 from sysown/v3.0_extended_query_protocol PostgreSQL's Extended Query Protocol Support in ProxySQL [WIP]	8 months ago
Rahim Kanji	344ba25777	Add Watchdog PROXYSQLTEST covering MySQL and PgSQL threads	9 months ago
Rahim Kanji	703a5f35f5	Added stats_pgsql_prepared_statements_info table	9 months ago
Javier Jaramago Fernández	b7b4864729	Fix table definition for 'pgsql_replication_hostgroups' - Reduce 'check_type' range to uniquely supported check ('read_only'). - Added table upgrade procedure. - Fixed missing space in table definition (display for '\G').	9 months ago
Rahim Kanji	3cff4f9dca	* Added sticky backend support for extended query frames: all messages in a single frame will now use the same backend connection. * Fixed few compilation warnings. * Removed AutoCommit relate code from PostgreSQL session.	9 months ago
René Cannaò	2551d350e9	Port to 3.0 of in-memory query logging See #4713 for further details	1 year ago
Rahim Kanji	0229daeb64	Added stats_pgsql_query_digest and stats_pgsql_query_digest_reset tables	2 years ago
Rahim Kanji	e723082a1b	Refactored Query Processor Architecture - Introduced separate query processors for MySQL and PostgreSQL. - Removed unnecessary GTID logic from `PgSQL_Session`. - Added a command counter for PostgreSQL queries. - Refactored `Query_Processor` to serve as a base class containing core functionalities applicable to both MySQL and PostgreSQL. Changes: - Removed GTID-related code from `PgSQL_Logger` and `PgSQL_Session`: - bool with_gtid; - char gtid_buf[128]; - int gtid_hid; - Added new PostgreSQL-specific tables: - `pgsql_query_rules`, `pgsql_firewall_whitelist_rules`, `pgsql_query_rules_fast_routing` - `runtime_pgsql_query_rules`, `runtime_pgsql_firewall_whitelist_rules`, `runtime_pgsql_query_rules_fast_routing` - Added PostgreSQL-related variables: pgsql_thread___commands_stats pgsql_thread___query_digests pgsql_thread___query_digests_lowercase pgsql_thread___query_digests_no_digits pgsql_thread___query_digests_replace_null pgsql_thread___query_digests_normalize_digest_text pgsql_thread___query_digests_track_hostname pgsql_thread___query_digests_keep_comment pgsql_thread___query_digests_max_digest_length pgsql_thread___query_digests_max_query_length pgsql_thread___query_digests_grouping_limit pgsql_thread___query_digests_groups_grouping_limit pgsql_thread___enable_load_data_local_infile pgsql_thread___auditlog_filename pgsql_thread___auditlog_filesize pgsql_thread___eventslog_filename pgsql_thread___eventslog_filesize pgsql_thread___eventslog_default_log pgsql_thread___eventslog_format pgsql_thread___firewall_whitelist_errormsg pgsql_thread___firewall_whitelist_enabled pgsql_thread___query_processor_iterations pgsql_thread___query_processor_regex	2 years ago
Rahim Kanji	5bf827d8e4	Removed type selector	2 years ago
Rahim Kanji	3112a56da4	Added 'stats_pgsql_client_host_cache' and 'stats_pgsql_client_host_cache_reset' tables.	2 years ago
Rahim Kanji	e939cf088e	Added 'stats_pgsql_errors' and 'stats_pgsql_errors_reset' tables	2 years ago
Rahim Kanji	43d3ec82c8	Added stats_pgsql_processlist table * Implemented 'SHOW FULL PGSQL PROCESSLIST' command. * Implemented 'SHOW PGSQL PROCESSLIST' command.	2 years ago
Rahim Kanji	0b8df6df00	Added stats_pgsql_users table	2 years ago
Rahim Kanji	b4e1e1eea7	Added stats_pgsql_free_connections table	2 years ago
Rahim Kanji	9cfe5b1761	Added stats_pgsql_connection_pool & stats_pgsql_connection_pool_reset tables	2 years ago
Rahim Kanji	c35a6a24cf	Added stats_pgsql_global table	2 years ago
Rene Cannao	465eaef87e	Merge branch 'v2.x_postgres' into v2.x_240630	2 years ago
Miro Stauder	da61f23eb3	Merge branch 'v2.x' into v2.6.x-freebsd_fixes	2 years ago
René Cannaò	10a7caf601	Further cleanup on servers tables for PgSQL	2 years ago
René Cannaò	d0da556c12	Minor optimization in headers	2 years ago
René Cannaò	de345ecaed	Merge remote-tracking branch 'public/v2.x' into v2.x_postgres_240422	2 years ago
René Cannaò	38c6c2ac7b	Merge branch 'v2.x-main' into v2.x_postgres_240422 Merged main ProxySQL code up to commit `959c629`	2 years ago
René Cannaò	d3a90a8ef6	Some refactoring of ProxySQL_Admin Related to flush_XXXXX_variables___database_to_runtime(). Added three new functions then used by the various main functions(): * flush_GENERIC_variables__retrieve__database_to_runtime() * flush_GENERIC_variables__checksum__database_to_runtime() * flush_GENERIC_variables__process__database_to_runtime()	2 years ago
René Cannaò	8a525bf9f6	Merge pull request #4518 from sysown/v2.x-4510 Move Admin 'HTTP_Server' and 'RESTAPI' init to end of phase3 - Closes #4510	2 years ago
Javier Jaramago Fernández	e91bcd2c81	Move Admin 'HTTP_Server' and 'RESTAPI' init to end of phase3 - Closes #4510 Prevents invalid accesses that this modules can perform on other yet non-initialized modules during boot process.	2 years ago
René Cannaò	7a50a68f28	Split of ProxySQL_Admin.cpp ProxySQL_Admin.cpp as become very big, thus it is time to split it in smaller files for better reasibility and processing. Still a work in progress.	2 years ago
René Cannaò	0a2cd03672	Refactoring to split Query_Processor.cpp into multiple files	2 years ago
René Cannaò	42178f8dd9	Use of function pointer for SIGUSR1 handler	2 years ago
Miro Stauder	9ac76349fe	refactor FLUSH LOGS and signal handler	2 years ago
Rahim Kanji	936e27139b	Merge branch 'v2.x_PostgreSQL_Merge' into v2.x_postgres # Conflicts: # include/MySQL_Session.h # lib/Makefile # lib/MySQL_Session.cpp # lib/MySQL_Thread.cpp # lib/ProxySQL_Admin.cpp # lib/ProxySQL_Poll.cpp	2 years ago
Rahim Kanji	ebf20b6a60	* Added PostgreSQL configuration (port: 6034 Admin and 6035: PostgreSQL client connections) * Fixed few bugs * Added Load and Save commands for PostgreSQL tables	2 years ago
Rahim Kanji	e029d39007	Initial PostgreSQL commit * Added plain text and SCRAM authentication	2 years ago
René Cannaò	8a4246d1e9	Merge pull request #4458 from sysown/v2.x_Servers_SSL_Params Introducing mysql_servers_ssl_params	2 years ago

1 2 3 4 5

240 Commits (db5f1ea498f736d4ff733bbd41ec649d073cf932)