proxysql

Commit Graph

Author	SHA1	Message	Date
René Cannaò	a360dc22ae	Merge pull request #5308 from sysown/v3.0_mac Support for ProxySQL compilation on macOS (Darwin/Apple Silicon)	5 months ago
René Cannaò	48f1b3cfef	Merge pull request #5095 from Gonlo2/fix-overflow-if-first-server-is-invalid Fix int overflow if the first server is invalid	5 months ago
René Cannaò	de0a9b9c27	Merge pull request #5093 from Gonlo2/fix-usage-deleted-con Avoid usage of deleted connection	5 months ago
René Cannaò	77fe582801	Merge pull request #5317 from sysown/v3.0.6-implement_FLUSH_STATS implement admin command 'PROXYSQL FLUSH STATS'	5 months ago
René Cannaò	ce9fdc9d7a	Merge branch 'v3.0' into v3.0.6-implement_FLUSH_STATS Signed-off-by: René Cannaò <rene@proxysql.com>	5 months ago
Rene Cannao	e3026cbc6f	Fix wrong index in connection cleanup loops (MySQL and PgSQL) This commit fixes a bug where connection cleanup loops were removing the wrong connection from the pool. The loops checked each connection by index (i), but when an expired connection was found, they removed index 0 instead of index i. This caused: - Fresh connections at index 0 to be incorrectly deleted - Expired connections to remain in the pool Fixed files: - lib/Base_HostGroups_Manager.cpp:2603 (MySQL) - lib/MySQL_HostGroups_Manager.cpp:2939 (MySQL) - lib/PgSQL_HostGroups_Manager.cpp:2782 (PgSQL) The fix changes `remove(0)` to `remove(i)` to remove the correct connection. Related: #5094 (fixes similar issue in drop_all_idle_connections)	5 months ago
René Cannaò	0524363322	Merge pull request #5339 from sysown/v3.0-merge-v4.0-genai Merge v4.0 GenAI features into v3.0 with conditional compilation	5 months ago
René Cannaò	27f13ca18d	Merge pull request #5096 from Gonlo2/fix-close-ssl-con Avoid send close in ssl connections	5 months ago
Rene Cannao	79756e78d7	Fix missing #endif for PROXYSQLGENAI guard in Admin_Handler.cpp	5 months ago
Rene Cannao	1af768f932	Fix critical issues in GenAI code (PR #5339 ) This commit addresses critical issues identified in the PR review: 1. Build system redundancy (src/Makefile): - Remove redundant PROXYSQLGENAI conditionals in linking commands - The flag doesn't affect linking, so simplify to PROXYSQLCLICKHOUSE only 2. Missing PROXYSQLGENAI guard (lib/Admin_Handler.cpp): - Add #ifdef PROXYSQLGENAI around MCP VARIABLES DISK commands - Ensures MCP commands are only available when GenAI is enabled 3. Broken retry logic (lib/LLM_Clients.cpp): - Remove misleading is_retryable_error() checks that used stale values - last_curl_code and last_http_code were never updated - Simplify to retry on empty responses only (documented limitation) - Fix thread-safety: use thread_local std::mt19937 instead of rand() 4. Resource leak (lib/MySQL_Tool_Handler.cpp): - Clean up previously created connections on init failure - Both mysql_init and mysql_real_connect error paths now clean up 5. Race condition (lib/Query_Tool_Handler.cpp): - Add missing pool_lock in find_connection() - Prevents race condition when accessing connection_pool Fixes identified by automated PR review agents.	5 months ago
Rene Cannao	b965fc6df4	Fix PROXYSQLGENAI build - resolve circular includes and missing headers This commit fixes compilation errors when building with PROXYSQLGENAI=1: Header file fixes: - Add missing #endif /* PROXYSQLGENAI */ before header guards in multiple headers - Remove #include cpp.h from GenAI headers to avoid circular dependencies Source file fixes: - Add #include proxysql.h to GenAI .cpp files that were missing it - Add #include Static_Harvester.h to Query_Tool_Handler.cpp for forward decl Build system fixes: - Remove vec.o from libproxysql.a (it's linked separately in src/Makefile) - Prevents duplicate symbol errors during linking Test files: - Rename MCP test files with .sh suffix to prevent make clean deletion Both build modes now work: - make build_lib_debug (without GenAI) - PROXYSQLGENAI=1 make build_lib_debug (with GenAI)	5 months ago
Rene Cannao	48bc7dd7bf	Merge v4.0 GenAI features into v3.0 with conditional compilation This commit merges the experimental v4.0 GenAI/MCP features into the stable v3.0 branch using conditional compilation. All v4.0 features are disabled by default and only enabled when PROXYSQLGENAI=1 is set at compile time. Changes: Build System: - Modified main Makefile to pass PROXYSQLGENAI flag to sub-makefiles - Modified deps/Makefile to conditionally build sqlite-vec and sqlite-rembed - Modified lib/Makefile to add PSQLGA flag and include GenAI object files - Modified src/Makefile to add PSQLGA flag and conditional linking Headers (wrapped with #ifdef PROXYSQLGENAI): - All 20 new GenAI header files in include/ - Modified cpp.h, proxysql_glovars.hpp, proxysql_admin.h - Modified ProxySQL_Admin_Tables_Definitions.h for GenAI/MCP tables Source Files: - All 22 new GenAI source files in lib/ wrapped with #ifdef PROXYSQLGENAI - Modified src/main.cpp for conditional global variables and init/shutdown - Modified Admin_Handler.cpp for conditional command handlers - Modified Admin_Bootstrap.cpp for conditional table registration - Modified Admin_FlushVariables.cpp for conditional variable flushing - Modified ProxySQL_Admin.cpp for conditional admin methods - Modified ProxySQL_Admin_Stats.cpp for conditional MCP stats functions - Modified proxy_sqlite3_symbols.cpp to always compile (needed by core) - Modified MySQL_Session.cpp for conditional GenAI function calls Test Files: - Renamed test_mcp_query_rules-t to test_mcp_query_rules-t.sh - Renamed test_mcp_rag_metrics-t to test_mcp_rag_metrics-t.sh - Modified anomaly_detection-t.cpp for conditional test execution Usage: # Build without GenAI (v3.0 mode - default) make clean && make build_deps -j$(nproc) && make build_lib -j$(nproc) && make build_src -j$(nproc) # Build with GenAI (v4.0 mode) make clean && PROXYSQLGENAI=1 make build_deps -j$(nproc) && PROXYSQLGENAI=1 make build_lib -j$(nproc) && PROXYSQLGENAI=1 make build_src -j$(nproc)	5 months ago
René Cannaò	e00617be11	Merge pull request #5333 from wazir-ahmed/v4.0-config-embedding-model RAG: Fix query embedding and vector search	5 months ago
Wazir Ahmed	10026891ce	RAG: Convert vector similarity search into a subquery - `sqlite-vec` requires that `knn` queries (using the `MATCH` operator for vector similarity search) must have a `LIMIT` clause at the same query level as the `MATCH` clause. - Execute `knn` queries as a subquery and then do `JOIN`s with `rag_chunks` and `rag_documents`. Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	5 months ago
Javier Jaramago Fernández	e351a0df74	feat: Add leading comments (--) detection for 'run_sql_readonly' Read only queries are no longer flagged as non-readonly when starting with double dash comments.	5 months ago
Miro Stauder	cc1d93d08b	implement admin command 'PROXYSQL FLUSH STATS' for DEBUG builds	5 months ago
Wazir Ahmed	8d022a0fcc	RAG: Add 'model' request parameter for embedding service - `model` is a required parameter for embedding request in OpenAI API specification. - https://platform.openai.com/docs/api-reference/embeddings Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	5 months ago
Javier Jaramago Fernández	fec97bbabe	fix: Fix GENAI variable loading during 'ProxySQL_Admin::init'	5 months ago
Javier Jaramago Fernández	a5ef787c7e	feat: Improve logging for MCP and RAG tools - Add new column 'endpoint' for 'stats_mcp_query_tools_counters'. - Add new table 'rag_search_log' to log 'log_rag_search_fts' operations.	5 months ago
Wazir Ahmed	e4f4dc95ce	RAG: bm25 and MATCH do not work with table alias Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	5 months ago
Wazir Ahmed	c52c621b27	MCP: Add mcp-rag_endpoint_auth config Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	5 months ago
Wazir Ahmed	eb495f42ee	AI: Fix vector_db table creation Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	5 months ago
Wazir Ahmed	22c4e94d53	AI: Fix sqlite-vec extension loading Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	5 months ago
Wazir Ahmed	7167f95247	AI: Enable extensions for vector_db Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	5 months ago
Wazir Ahmed	4031f85394	AI: Fix vector_db initialization Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	5 months ago
Wazir Ahmed	cecb975f66	fix: LOAD GENAI TO RUNTIME does not initialize the module Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	5 months ago
Wazir Ahmed	50de536534	MCP: Fix crash during server restarts Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	5 months ago
Rene Cannao	42a67ebaf2	Merge rahim/v4.0_rag_ingest into v4.0_rag_ingest_2 Merge changes from PR #5318 (RAG ingestion feature) into the new development branch v4.0_rag_ingest_2. Changes include: - RAG ingestion tool (rag_ingest) with chunking and embeddings - MySQL_Catalog fixes for NULL pointer handling - MCP server updates for RAG tools - Comprehensive documentation and test scripts	5 months ago
Rene Cannao	38e5e8e56b	Fix critical issues from coderabbitai review - Fix NULL pointer dereference in rag_ingest.cpp: use str_or_empty() helper for all sqlite3_column_text results assigned to std::string - Fix NULL tags/links crash in MySQL_Catalog.cpp: add null guards before assigning sqlite3_column_text results to std::string - Fix missing curl_global_cleanup on error path in rag_ingest.cpp - Fix std::out_of_range exception in rag_ingest.cpp: wrap std::stoll calls in try-catch blocks, fall back to string comparison on overflow	5 months ago
Rene Cannao	9ba3df0ce7	Address AI code review feedback from PR #5318 - Fix Makefile: Use $(CXXFLAGS) directly for consistency with build philosophy - Fix MySQL_Catalog: Return proper error JSON instead of empty array on missing query	5 months ago
Rahim Kanji	d28444a02e	Merge remote-tracking branch 'v4.0' into v4.0_rag_ingest	5 months ago
Rene Cannao	3ccfa2bcc1	Address AI code review feedback for PR #5313 This commit addresses concerns raised by AI code reviewers (gemini-code-assist, Copilot, coderabbitai) on the initial security fixes. Critical fixes: - Fix lock.release() → lock.unlock() in GenAI_Thread.cpp worker_loop (lock.release() detaches without unlocking, causing deadlock) - Add missing early return after schema validation failure in Query_Tool_Handler.cpp Code quality improvements: - Improve escape_string() memory management in MySQL_Tool_Handler.cpp: - Use std::string instead of new[]/delete[] for buffer management - Check return value of mysql_real_escape_string() for errors - Remove redundant validation checks in validate_sql_identifier functions (character class loop already rejects unsafe characters) - Add backslash escaping to escape_string_literal() for defense-in-depth - Improve column list validation in MySQL_Tool_Handler sample_rows(): - Replace blacklist approach with proper column identifier parsing - Allow qualified identifiers (table.column) - Allow AS aliases (column AS alias) - No longer rejects legitimate column names containing "JOIN" These changes improve robustness while maintaining the security posture of the original SQL injection fixes.	5 months ago
Rene Cannao	c914feb230	Fix security issues identified in PR #5312 code review This commit addresses critical and important security vulnerabilities found during comprehensive code review of the Gen AI features merge. Critical fixes: - SQL injection vulnerabilities in MySQL_Tool_Handler.cpp: - Added validate_sql_identifier() for schema/table validation - Added escape_string() for MySQL string escaping using mysql_real_escape_string - Fixed list_tables(), describe_table(), sample_rows(), sample_distinct() - SQL injection vulnerabilities in Query_Tool_Handler.cpp: - Added validate_sql_identifier_sqlite() for identifier validation - Added escape_string_literal() for SQLite string escaping - Fixed list_tables tool and catalog.get_relationships function - Use-after-free race condition in GenAI_Thread: - Changed shutdown_ from int to std::atomic<int> for proper memory ordering - Added additional shutdown check in worker_loop after popping request Important fixes: - Buffer overflow risks from sprintf usage: - Converted all sprintf() calls to snprintf() in GenAI_Thread.cpp - Converted sprintf() to snprintf() in MySQL_Session.cpp - Worker loop shutdown race condition: - Added shutdown check after popping request from queue - Properly clean up client_fd when shutdown is detected These fixes ensure: 1. All user input is properly validated before use in SQL queries 2. String values are properly escaped using database-specific escaping 3. Thread-safe shutdown with proper memory ordering guarantees 4. Bounds-safe string formatting to prevent buffer overflows	5 months ago
Rene Cannao	03e58146ea	fix: Re-enable SQLite3DB::LoadPlugin() with allow_load_plugin flag This addresses an issue from PR #22 where LoadPlugin() was completely disabled. The function performs necessary initialization even when no plugin is loaded (initializes built-in sqlite3 function pointers). Changes: - Added `const bool allow_load_plugin = false` flag in LoadPlugin() - Modified `if (plugin_name)` to `if (plugin_name && allow_load_plugin == true)` - Re-enabled the LoadPlugin() call in LoadPlugins() The plugin loading code remains disabled (allow_load_plugin=false) while the function pointer initialization from built-in SQLite3 now works correctly. TODO: Revisit plugin loading safety mechanism to allow actual plugin loading.	5 months ago
Rene Cannao	a3afde3472	fix: Address copilot review concerns for Discovery_Schema.cpp - Fix comment mismatch: Changed _2 suffix to match actual function name (mysql_query_digest_and_first_comment, not _2) - Make get_def_mysql_opts() static to avoid symbol pollution - Fix NULL first_comment parameter to prevent potential segfault - Pass valid char* pointer instead of NULL - Free first_comment if allocated by the function	5 months ago
Rene Cannao	7b6966b9c2	fix: Complete JSON escaping in fingerprint_mcp_args Address coderabbitai review - implement full JSON escaping for SQL digest: - Handle backslash (\) and double quote (") - Handle control characters: newline (\n), carriage return (\r), tab (\t) - Handle other control characters (U+0000 through U+001F) with \uXXXX escapes This ensures digest_text in stats_mcp_query_digest is always valid JSON, preventing parsing errors for consumers of this data.	5 months ago
Rene Cannao	f2536f01d2	Merge v3.1-vec into v3.1-MCP2_QR Resolved merge conflict in lib/ProxySQL_Admin_Stats.cpp: - Kept v3.1-MCP2_QR version with proper NULL handling - Uses stmt_unique_ptr (RAII smart pointer) - Checks for NULL before calling atoll() on numeric fields - Discarded v3.1-vec version which lacked NULL checks The merged code correctly handles NULL values for: - run_id - count_star - first_seen - last_seen - sum_time - min_time - max_time	5 months ago
Rene Cannao	5d4318b547	fix: Address coderabbitai review concerns for PR #27 - Fix Discovery_Schema.cpp fingerprint output JSON consistency - Quote placeholders (e.g., "?" instead of ?) for valid JSON - Fix test_mcp_query_rules_block.sh exec_admin_silent function - Add -B and -N flags for batch mode with no headers - Remove unused YELLOW variable from test_phase1_crud.sh - Fix test_phase2_load_save.sh runtime DELETE to be non-fatal - Remove unused INITIAL_COUNT from test_phase3_runtime.sh - Fix test_phase3_runtime.sh to verify exact ID ordering - Fix test_phase4_stats.sh read-only test assertion - Make INSERT/DELETE non-fatal and fix assertion logic - Remove unused DIGEST_TEXT_CHECK from test_phase5_digest.sh - Fix test_phase8_eval_timeout.sh non-timeout response handling - Return failure for non-timeout responses - Remove stray exit 1 from test_phase8_eval_timeout.sh	5 months ago
Rene Cannao	ffe5690360	fix: Address coderabbitai review - use-after-free, missing responses, SQL injection Fix issues identified by coderabbitai review: 1. Admin_Handler.cpp: Fix typo in strncasecmp for LOAD MCP QUERY RULES - Line 2365 had "LOAD MCP RULES FROM DISK" instead of "LOAD MCP QUERY RULES FROM DISK" 2. Admin_Handler.cpp: Fix use-after-free and missing client response - Removed l_free(ql,q) which freed q before caller used it - Added send_error_msg_to_client calls on all error paths - Added send_ok_msg_to_client call on success path - Changed return value from true to false (to match handler pattern) - Applied to both LOAD MCP QUERY RULES and SAVE MCP QUERY RULES handlers 3. ProxySQL_Admin_Stats.cpp: Fix sprintf SQL injection in stats___mcp_query_rules - Replaced sprintf with prepared statement using positional parameters - Changed from char query with malloc/free to sqlite3_stmt with prepare_v2 - Both columns now bound as parameters (?1, ?2)	5 months ago
Rene Cannao	5ece563514	fix: Correct SQL prepared statement API usage and template variable access Fix compilation errors in the SQL injection fixes: 1. ProxySQL_Admin_Stats.cpp: Use public statsdb->prepare_v2() API - Changed from direct proxy_sqlite3_prepare_v2() calls with statsdb->db - statsdb->db is private, must use public prepare_v2(query, &stmt) method 2. Admin_Handler.cpp: Add SPA cast for template function access - Added ProxySQL_Admin SPA=(ProxySQL_Admin )pa; declaration - Changed all admindb->execute to SPA->admindb->execute - Removed unused 'error' and 'success' variables The build now completes successfully.	5 months ago
Rene Cannao	9f07e9631e	fix: Use prepared statements in ProxySQL_Admin_Stats to prevent SQL injection Fix two SQL injection vulnerabilities identified by coderabbitai in ProxySQL_Admin_Stats.cpp by converting sprintf/snprintf interpolation to SQLite prepared statements. 1. stats___mcp_query_digest (lines 2581-2636): - Changed from sprintf with format string to prepared statement - digest_text field now properly bound as parameter (was unsafe) - All 10 columns now use positional parameters (?1-?10) 2. stats___mcp_query_tools_counters (lines 1587-1635): - Changed from snprintf with unescaped fields to prepared statement - Fixed incorrect table name logic (was appending _reset incorrectly) - All 8 columns now use positional parameters (?1-?8) These changes prevent SQL injection when resultset fields contain quotes, backslashes, or other SQL metacharacters.	5 months ago
Rene Cannao	3bcee22700	fix: Execute MCP query rules DELETE+INSERT as explicit transaction Fix multi-statement execution in Admin_Handler.cpp for MCP query rules. The previous code built a single SQL string with "DELETE ...; INSERT ..." but SQLite only executed the first statement. Changed to execute statements as an explicit transaction: 1. BEGIN 2. DELETE FROM target_table 3. INSERT OR REPLACE INTO target_table SELECT * FROM source_table 4. COMMIT (or ROLLBACK on error) Applied to both: - LOAD MCP QUERY RULES FROM DISK/TO MEMORY - SAVE MCP QUERY RULES TO DISK Addresses coderabbitai review comment.	5 months ago
Rene Cannao	188aef90fe	fix: Use delete instead of free for SQLite3_result in load_mcp_query_rules_to_runtime Change free(resultset) to delete resultset in ProxySQL_Admin::load_mcp_query_rules_to_runtime. SQLite3_result is a C++ class allocated with new, so it must be deallocated with delete, not free(). Using free() causes undefined behavior and memory leaks. Addresses coderabbitai review comment.	5 months ago
Rene Cannao	bbc04974f1	fix: Fix mysql_query failure path and affected_rows race condition Fix two issues in Query_Tool_Handler's execute_query functions: 1. mysql_query() failure path now returns immediately after return_connection() instead of continuing to process on bad state. 2. Capture affected_rows BEFORE return_connection() to avoid race condition. Previously, mysql_affected_rows() was called after return_connection(), potentially accessing a stale connection. Apply fixes to both execute_query() and execute_query_with_schema(). Addresses coderabbitai review comments.	5 months ago
Rene Cannao	b3edc6524b	fix: Escape SQL strings in harvest_view_definitions Add escape_sql_string() helper function that doubles single quotes to prevent SQL injection when strings are used in SQL string concatenation. Update harvest_view_definitions to use this function for view_def, schema_name, and view_name. This prevents SQL injection in the UPDATE statement that stores view definitions in the catalog. Addresses coderabbitai review comment.	5 months ago
Rene Cannao	6835713f11	fix: Correct column indexes in build_quick_profiles Fix off-by-one errors in row->fields indices. The SELECT clause returns: object_id(0), schema_name(1), object_name(2), object_type(3), engine(4), table_rows_est(5), data_length(6), index_length(7), has_primary_key(8), has_foreign_keys(9), has_time_column(10) But the code was reading from fields[3..9] instead of [4..10]. Added comment documenting the correct column order. Addresses coderabbitai review comment.	5 months ago
Rene Cannao	e9abee625b	fix: Execute prepared statement in execute_parameterized_query The execute_parameterized_query function in RAG_Tool_Handler was creating a prepared statement and binding parameters, but then executing the raw query string instead of the prepared statement. This completely bypassed the parameter binding, making the function useless for preventing SQL injection. Changed to use vector_db->execute_prepared(stmt, ...) to execute the bound statement instead of the raw query, so the bound parameters are actually used. Addresses coderabbitai review comment.	5 months ago
Rene Cannao	6305537ba8	fix: Use delete instead of free for SQLite3_result deallocation Change free(resultset) to delete resultset in Query_Tool_Handler (extract_schema_name function). SQLite3_result is a C++ class allocated with new, so it must be deallocated with delete, not free(). Using free() causes mixed allocator UB. Addresses coderabbitai review comment.	5 months ago
Rene Cannao	5e121399aa	fix: Add AFTER UPDATE trigger to keep catalog_fts index in sync for upserts Add catalog_au AFTER UPDATE trigger in MySQL_Catalog that mirrors the delete+insert pattern used in catalog_ad/catalog_ai. This keeps the FTS index current when upserts occur (INSERT OR REPLACE ... ON CONFLICT ... DO UPDATE), since the UPDATE doesn't trigger INSERT/DELETE triggers. The trigger first deletes the old entry from catalog_fts then inserts the new entry, ensuring the full-text search index stays synchronized with the catalog table. Addresses coderabbitai review comment.	5 months ago
Rene Cannao	5dd5dbe6b7	fix: Add missing assert(proxy_sqlite3_bind_blob) in sqlite3db.cpp Add assert(proxy_sqlite3_bind_blob) to the assertion block in SQLite3DB::LoadPlugin to ensure the symbol is provided by plugins. Without this, if a plugin fails to provide the symbol, the code will crash at runtime with no safety check. proxy_sqlite3_bind_blob is actively used in Anomaly_Detector.cpp:765 to bind embeddings. Addresses coderabbitai review comment.	5 months ago

1 2 3 4 5 ...

4792 Commits (0e605f41704652b47a76c19a930e581cb0fc8583)