proxysql

Commit Graph

Author	SHA1	Message	Date
Rene Cannao	48bc7dd7bf	Merge v4.0 GenAI features into v3.0 with conditional compilation This commit merges the experimental v4.0 GenAI/MCP features into the stable v3.0 branch using conditional compilation. All v4.0 features are disabled by default and only enabled when PROXYSQLGENAI=1 is set at compile time. Changes: Build System: - Modified main Makefile to pass PROXYSQLGENAI flag to sub-makefiles - Modified deps/Makefile to conditionally build sqlite-vec and sqlite-rembed - Modified lib/Makefile to add PSQLGA flag and include GenAI object files - Modified src/Makefile to add PSQLGA flag and conditional linking Headers (wrapped with #ifdef PROXYSQLGENAI): - All 20 new GenAI header files in include/ - Modified cpp.h, proxysql_glovars.hpp, proxysql_admin.h - Modified ProxySQL_Admin_Tables_Definitions.h for GenAI/MCP tables Source Files: - All 22 new GenAI source files in lib/ wrapped with #ifdef PROXYSQLGENAI - Modified src/main.cpp for conditional global variables and init/shutdown - Modified Admin_Handler.cpp for conditional command handlers - Modified Admin_Bootstrap.cpp for conditional table registration - Modified Admin_FlushVariables.cpp for conditional variable flushing - Modified ProxySQL_Admin.cpp for conditional admin methods - Modified ProxySQL_Admin_Stats.cpp for conditional MCP stats functions - Modified proxy_sqlite3_symbols.cpp to always compile (needed by core) - Modified MySQL_Session.cpp for conditional GenAI function calls Test Files: - Renamed test_mcp_query_rules-t to test_mcp_query_rules-t.sh - Renamed test_mcp_rag_metrics-t to test_mcp_rag_metrics-t.sh - Modified anomaly_detection-t.cpp for conditional test execution Usage: # Build without GenAI (v3.0 mode - default) make clean && make build_deps -j$(nproc) && make build_lib -j$(nproc) && make build_src -j$(nproc) # Build with GenAI (v4.0 mode) make clean && PROXYSQLGENAI=1 make build_deps -j$(nproc) && PROXYSQLGENAI=1 make build_lib -j$(nproc) && PROXYSQLGENAI=1 make build_src -j$(nproc)	2 months ago
Javier Jaramago Fernández	fec97bbabe	fix: Fix GENAI variable loading during 'ProxySQL_Admin::init'	2 months ago
Wazir Ahmed	22c4e94d53	AI: Fix sqlite-vec extension loading Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	2 months ago
René Cannaò	af28598b23	Merge pull request #19 from ProxySQL/v3.1-MCP2_QR feat: Add MCP query rules and digest statistics [WIP]	2 months ago
Rene Cannao	0db022a179	Apply fixes	2 months ago
Rene Cannao	2dfd61a958	Replace remaining direct sqlite3_* calls with proxy_sqlite3_* equivalents (address code-review)	2 months ago
Rene Cannao	7e6f9f0ab3	fix: Add MCP query rules LOAD/SAVE command handlers - Add separate MCP QUERY RULES command block in Admin_Handler - Fix string length comparison (21 chars for "SAVE/LOAD MCP QUERY RULES ") - Add handlers for: - LOAD MCP QUERY RULES TO RUNTIME - SAVE MCP QUERY RULES TO DISK - SAVE MCP QUERY RULES TO MEMORY / FROM RUNTIME - Register mcp_query_rules in disk database (tables_defs_config) Previously MCP commands were incorrectly nested inside MYSQL/PGSQL block and could not be reached. Now they have their own conditional block.	2 months ago
Rene Cannao	f01fc79584	feat: Add runtime_mcp_query_rules table and fix stats_mcp_query_rules schema - Add ADMIN_SQLITE_TABLE_RUNTIME_MCP_QUERY_RULES schema (17 columns, same as mcp_query_rules) - Fix STATS_SQLITE_TABLE_MCP_QUERY_RULES to only have rule_id and hits columns - Add runtime_mcp_query_rules detection and refresh in ProxySQL_Admin - Implement save_mcp_query_rules_from_runtime(bool _runtime) for both config and runtime tables - Update get_mcp_query_rules() to return 17 columns (no hits) - get_stats_mcp_query_rules() returns 2 columns (rule_id, hits) Mirrors the MySQL query rules pattern: - mcp_query_rules: config table (17 cols) - runtime_mcp_query_rules: runtime state (17 cols) - stats_mcp_query_rules: hit counters (2 cols)	2 months ago
Rene Cannao	fb66af7c1b	feat: Expose MCP catalog database in ProxySQL Admin interface The MCP catalog database is now accessible as the 'mcp_catalog' schema from the ProxySQL Admin interface, enabling direct SQL queries against discovered schemas and LLM memories.	2 months ago
Rene Cannao	a816a756d4	feat: Add MCP query tool usage counters to stats schema Add stats_mcp_query_tools_counters and stats_mcp_query_tools_counters_reset tables to track MCP query tool usage statistics. - Added get_tool_usage_stats_resultset() method to Query_Tool_Handler - Defined table schemas in ProxySQL_Admin_Tables_Definitions.h - Registered tables in Admin_Bootstrap.cpp - Added pattern matching in ProxySQL_Admin.cpp - Added stats___mcp_query_tools_counters() in ProxySQL_Admin_Stats.cpp - Fixed friend declaration for track_tool_invocation() - Fixed Discovery_Schema.cpp log_llm_search() to use prepare_v2/finalize	2 months ago
Rene Cannao	ef07831780	Add MCP module to admin bootstrap and SHOW MCP VARIABLES command The MCP module was not being loaded because: 1. The admin bootstrap process was not calling flush_mcp_variables___database_to_runtime - Added the call after flush_sqliteserver_variables___database_to_runtime 2. There was no SHOW MCP VARIABLES command handler - Added the handler in Admin_Handler.cpp, following the same pattern as SHOW MYSQL VARIABLES and SHOW PGSQL VARIABLES Now after this change: - MCP variables (mcp-enabled, mcp-port, mcp-mysql_hosts, etc.) will be automatically inserted into global_variables table during ProxySQL startup - Users can run "SHOW MCP VARIABLES" to list all MCP configuration variables - The configure_mcp.sh script will work correctly Note: Requires rebuilding ProxySQL for changes to take effect.	3 months ago
Rene Cannao	01d654692d	Integrate sqlite-rembed for text embedding generation Add support for sqlite-rembed Rust SQLite extension to enable text embedding generation from remote AI APIs (OpenAI, Nomic, Ollama, Cohere, etc.) within ProxySQL's SQLite3 Server. Changes: 1. Build system integration for Rust static library compilation - Rust toolchain detection in deps/Makefile - Static library target: sqlite3/libsqlite_rembed.a - Linking integration in lib/Makefile and src/Makefile 2. Extension auto-registration in Admin_Bootstrap.cpp - Declare sqlite3_rembed_init() extern C function - Register via sqlite3_auto_extension() after sqlite-vec 3. Documentation updates - doc/sqlite-rembed-integration.md: comprehensive integration guide - doc/SQLite3-Server.md: usage examples and provider list 4. Source code inclusion - deps/sqlite3/sqlite-rembed-source/: upstream sqlite-rembed v0.0.1-alpha.9 The integration follows the same pattern as sqlite-vec (static linking with auto-registration). Provides rembed() function and temp.rembed_clients virtual table for embedding generation. Build requires Rust toolchain (cargo, rustc) and clang/libclang-dev.	3 months ago
Rene Cannao	d55947b49f	Add comprehensive documentation for sqlite-vec integration This commit adds extensive documentation for the sqlite-vec vector search extension integration in ProxySQL, including: ## README Documentation ### deps/sqlite3/README.md - Overview of sqlite-vec and its vector search capabilities - Integration method using static linking - Directory structure explanation - Compilation flags and build process details - Usage examples for all ProxySQL databases - Benefits and verification instructions ### deps/sqlite3/sqlite-vec-source/README.md - Complete sqlite-vec documentation - Source files explanation - Integration specifics for ProxySQL - Licensing information - Standalone building instructions - Performance considerations ## Doxygen Code Documentation ### lib/Admin_Bootstrap.cpp - Added comprehensive doxygen comments for sqlite-vec integration - Documented sqlite3_vec_init function declaration - Added section documentation for SQLite database initialization - Detailed documentation for each database instance: * Admin: Configuration analytics and vector operations * Stats: Performance metrics and similarity analysis * Config: Configuration optimization with vectors * Monitor: Anomaly detection and pattern recognition * Stats Disk: Historical trend analysis - Included usage examples and cross-references - Explained auto-extension mechanism and integration benefits The documentation provides developers with a complete reference for understanding, using, and maintaining the sqlite-vec integration in ProxySQL's SQLite databases. Technical Details: - Static linking implementation - Virtual table mechanism - JSON vector format support - Auto-extension registration - Multi-database integration - Performance optimizations	3 months ago
Rene Cannao	fbd0d9732b	Add sqlite-vec static extension for vector search in ProxySQL This commit integrates sqlite-vec (https://github.com/asg017/sqlite-vec) as a statically linked extension, enabling vector search capabilities in all ProxySQL SQLite databases (admin, stats, config, monitor). Changes: 1. Added sqlite-vec source files to deps/sqlite3/sqlite-vec-source/ - sqlite-vec.c: main extension source - sqlite-vec.h: header for static linking - sqlite-vec.h.tmpl: template header 2. Modified deps/Makefile: - Added target sqlite3/sqlite3/vec.o that copies sources and compiles with flags -DSQLITE_CORE -DSQLITE_VEC_STATIC - Made sqlite3 target depend on vec.o 3. Modified lib/Makefile: - Added $(SQLITE3_LDIR)/vec.o to libproxysql.a prerequisites - Included vec.o in the static library archive 4. Modified lib/Admin_Bootstrap.cpp: - Added extern "C" declaration for sqlite3_vec_init - Enabled load extension support for all databases: - admindb, statsdb, configdb, monitordb, statsdb_disk - Registered sqlite3_vec_init as auto-extension at database open (replacing commented sqlite3_json_init) 5. Updated top-level Makefile: - Made GIT_VERSION fallback to git describe --always when tags missing Result: - Vector search functions (vec0 virtual tables, vector operations) are available in all ProxySQL SQLite databases without runtime dependencies - No separate shared library required; fully embedded in proxysql binary - Extension automatically loaded at database initialization	3 months ago
René Cannaò	3b4369bba9	Merge pull request #5113 from wazir-ahmed/load-from-config-validation Add validation for `LOAD * FROM CONFIG`	5 months ago
Rahim Kanji	0292b780e9	Fixed typo	6 months ago
Rahim Kanji	f3ea179384	Add backend PID/state to stats_pgsql_processlist and create pg_stat_activity-style view - Added `backend_pid` and `backend_state` columns to `stats_pgsql_processlist` to display PostgreSQL backend process ID and connection state. - Created `stats_pgsql_stat_activity` view on top of `stats_pgsql_processlist` with column aliases matching PostgreSQL's `pg_stat_activity` for consistency.	6 months ago
Wazir Ahmed	2b72e723e8	Add INFO log when values from configuration file are ignored Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	6 months ago
Wazir Ahmed	0a336de048	Add validation for `LOAD * FROM CONFIG` - Add validation methods for `mysql_users`, `pgsql_users`, `mysql_servers`, `pgsql_servers` and `proxysql_servers` - Check for duplicates and mandatory fields - Return descriptive error messages to clients when validation fails Signed-off-by: Wazir Ahmed <wazir@proxysql.com> Co-authored-by: takaidohigasi <takaidohigasi@gmail.com>	7 months ago
René Cannaò	b5fa8ca1ae	Merge pull request #5044 from sysown/v3.0_extended_query_protocol PostgreSQL's Extended Query Protocol Support in ProxySQL [WIP]	7 months ago
Rahim Kanji	703a5f35f5	Added stats_pgsql_prepared_statements_info table	8 months ago
Javier Jaramago Fernández	b7b4864729	Fix table definition for 'pgsql_replication_hostgroups' - Reduce 'check_type' range to uniquely supported check ('read_only'). - Added table upgrade procedure. - Fixed missing space in table definition (display for '\G').	9 months ago
René Cannaò	c48c3310a6	Check if database file exists before checking if it is writable Related to `bf3cca8639`	12 months ago
René Cannaò	bf3cca8639	exit if database file is not writable If database file (proxysql.db) is not writable, exit immediately instead of crashing because it is not writable. See issue #4893	1 year ago
René Cannaò	2551d350e9	Port to 3.0 of in-memory query logging See #4713 for further details	1 year ago
Rahim Kanji	720441af20	Refactored and Optimized Query Cache with PgSQL Support * Added Query Cache Support for PgSQL: * Introduced new variables for PgSQL Query Cache management: * pgsql-query_cache_size_MB * pgsql-query_cache_soft_ttl_pct * pgsql-query_cache_stores_empty_result * pgsql-query_cache_handle_warnings (not yet utilized) * Overwriting Transaction Status: * Sending current transaction state to the client when a result is returned from the query cache. * This currently does not differentiate between normal and error transaction states—both are treated as active transaction states. * Cache Lifetime Management: Replaced manual reference counting with std::shared_ptr to efficiently manage the lifecycle of cache entries, ensuring proper memory deallocation and reducing complexity. * Unified Purging Logic: Refactored the query cache purging logic to use a single thread for both MySQL and PgSQL cache entries. * Code Cleanup: Removed unnecessary methods and data members to streamline the codebase and improve maintainability. * Performance Optimization: Improved memory management, reducing memory footprint and enhancing performance.	1 year ago
Rahim Kanji	0229daeb64	Added stats_pgsql_query_digest and stats_pgsql_query_digest_reset tables	2 years ago
Rahim Kanji	e723082a1b	Refactored Query Processor Architecture - Introduced separate query processors for MySQL and PostgreSQL. - Removed unnecessary GTID logic from `PgSQL_Session`. - Added a command counter for PostgreSQL queries. - Refactored `Query_Processor` to serve as a base class containing core functionalities applicable to both MySQL and PostgreSQL. Changes: - Removed GTID-related code from `PgSQL_Logger` and `PgSQL_Session`: - bool with_gtid; - char gtid_buf[128]; - int gtid_hid; - Added new PostgreSQL-specific tables: - `pgsql_query_rules`, `pgsql_firewall_whitelist_rules`, `pgsql_query_rules_fast_routing` - `runtime_pgsql_query_rules`, `runtime_pgsql_firewall_whitelist_rules`, `runtime_pgsql_query_rules_fast_routing` - Added PostgreSQL-related variables: pgsql_thread___commands_stats pgsql_thread___query_digests pgsql_thread___query_digests_lowercase pgsql_thread___query_digests_no_digits pgsql_thread___query_digests_replace_null pgsql_thread___query_digests_normalize_digest_text pgsql_thread___query_digests_track_hostname pgsql_thread___query_digests_keep_comment pgsql_thread___query_digests_max_digest_length pgsql_thread___query_digests_max_query_length pgsql_thread___query_digests_grouping_limit pgsql_thread___query_digests_groups_grouping_limit pgsql_thread___enable_load_data_local_infile pgsql_thread___auditlog_filename pgsql_thread___auditlog_filesize pgsql_thread___eventslog_filename pgsql_thread___eventslog_filesize pgsql_thread___eventslog_default_log pgsql_thread___eventslog_format pgsql_thread___firewall_whitelist_errormsg pgsql_thread___firewall_whitelist_enabled pgsql_thread___query_processor_iterations pgsql_thread___query_processor_regex	2 years ago
Rahim Kanji	745449fcbc	Removed Client_Session.h and Client_Session.cpp	2 years ago
Rahim Kanji	3112a56da4	Added 'stats_pgsql_client_host_cache' and 'stats_pgsql_client_host_cache_reset' tables.	2 years ago
Rahim Kanji	e939cf088e	Added 'stats_pgsql_errors' and 'stats_pgsql_errors_reset' tables	2 years ago
Rahim Kanji	43d3ec82c8	Added stats_pgsql_processlist table * Implemented 'SHOW FULL PGSQL PROCESSLIST' command. * Implemented 'SHOW PGSQL PROCESSLIST' command.	2 years ago
Rahim Kanji	0b8df6df00	Added stats_pgsql_users table	2 years ago
Rahim Kanji	b4e1e1eea7	Added stats_pgsql_free_connections table	2 years ago
Rahim Kanji	9cfe5b1761	Added stats_pgsql_connection_pool & stats_pgsql_connection_pool_reset tables	2 years ago
Rahim Kanji	c35a6a24cf	Added stats_pgsql_global table	2 years ago
Rahim Kanji	5a57f7f6ca	Added missing tables	2 years ago
Rene Cannao	a1fd4af1a6	Splitting ProxySQL_Admin	2 years ago

38 Commits (afd6dffb05241468c7aebc689f5abbbfbc7d83e2)