proxysql

Commit Graph

Author	SHA1	Message	Date
Rene Cannao	b965fc6df4	Fix PROXYSQLGENAI build - resolve circular includes and missing headers This commit fixes compilation errors when building with PROXYSQLGENAI=1: Header file fixes: - Add missing #endif /* PROXYSQLGENAI */ before header guards in multiple headers - Remove #include cpp.h from GenAI headers to avoid circular dependencies Source file fixes: - Add #include proxysql.h to GenAI .cpp files that were missing it - Add #include Static_Harvester.h to Query_Tool_Handler.cpp for forward decl Build system fixes: - Remove vec.o from libproxysql.a (it's linked separately in src/Makefile) - Prevents duplicate symbol errors during linking Test files: - Rename MCP test files with .sh suffix to prevent make clean deletion Both build modes now work: - make build_lib_debug (without GenAI) - PROXYSQLGENAI=1 make build_lib_debug (with GenAI)	1 month ago
Rene Cannao	48bc7dd7bf	Merge v4.0 GenAI features into v3.0 with conditional compilation This commit merges the experimental v4.0 GenAI/MCP features into the stable v3.0 branch using conditional compilation. All v4.0 features are disabled by default and only enabled when PROXYSQLGENAI=1 is set at compile time. Changes: Build System: - Modified main Makefile to pass PROXYSQLGENAI flag to sub-makefiles - Modified deps/Makefile to conditionally build sqlite-vec and sqlite-rembed - Modified lib/Makefile to add PSQLGA flag and include GenAI object files - Modified src/Makefile to add PSQLGA flag and conditional linking Headers (wrapped with #ifdef PROXYSQLGENAI): - All 20 new GenAI header files in include/ - Modified cpp.h, proxysql_glovars.hpp, proxysql_admin.h - Modified ProxySQL_Admin_Tables_Definitions.h for GenAI/MCP tables Source Files: - All 22 new GenAI source files in lib/ wrapped with #ifdef PROXYSQLGENAI - Modified src/main.cpp for conditional global variables and init/shutdown - Modified Admin_Handler.cpp for conditional command handlers - Modified Admin_Bootstrap.cpp for conditional table registration - Modified Admin_FlushVariables.cpp for conditional variable flushing - Modified ProxySQL_Admin.cpp for conditional admin methods - Modified ProxySQL_Admin_Stats.cpp for conditional MCP stats functions - Modified proxy_sqlite3_symbols.cpp to always compile (needed by core) - Modified MySQL_Session.cpp for conditional GenAI function calls Test Files: - Renamed test_mcp_query_rules-t to test_mcp_query_rules-t.sh - Renamed test_mcp_rag_metrics-t to test_mcp_rag_metrics-t.sh - Modified anomaly_detection-t.cpp for conditional test execution Usage: # Build without GenAI (v3.0 mode - default) make clean && make build_deps -j$(nproc) && make build_lib -j$(nproc) && make build_src -j$(nproc) # Build with GenAI (v4.0 mode) make clean && PROXYSQLGENAI=1 make build_deps -j$(nproc) && PROXYSQLGENAI=1 make build_lib -j$(nproc) && PROXYSQLGENAI=1 make build_src -j$(nproc)	1 month ago
René Cannaò	e00617be11	Merge pull request #5333 from wazir-ahmed/v4.0-config-embedding-model RAG: Fix query embedding and vector search	2 months ago
Wazir Ahmed	10026891ce	RAG: Convert vector similarity search into a subquery - `sqlite-vec` requires that `knn` queries (using the `MATCH` operator for vector similarity search) must have a `LIMIT` clause at the same query level as the `MATCH` clause. - Execute `knn` queries as a subquery and then do `JOIN`s with `rag_chunks` and `rag_documents`. Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	2 months ago
Javier Jaramago Fernández	a5ef787c7e	feat: Improve logging for MCP and RAG tools - Add new column 'endpoint' for 'stats_mcp_query_tools_counters'. - Add new table 'rag_search_log' to log 'log_rag_search_fts' operations.	2 months ago
Wazir Ahmed	e4f4dc95ce	RAG: bm25 and MATCH do not work with table alias Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	2 months ago
Rene Cannao	e9abee625b	fix: Execute prepared statement in execute_parameterized_query The execute_parameterized_query function in RAG_Tool_Handler was creating a prepared statement and binding parameters, but then executing the raw query string instead of the prepared statement. This completely bypassed the parameter binding, making the function useless for preventing SQL injection. Changed to use vector_db->execute_prepared(stmt, ...) to execute the bound statement instead of the raw query, so the bound parameters are actually used. Addresses coderabbitai review comment.	2 months ago
Rene Cannao	bd6d34f52b	fix: Address SQL injection vulnerabilities from PR #26 review - lib/MySQL_Catalog.cpp: Convert search/list/remove to use SQLite prepared statements instead of string concatenation for user parameters - lib/RAG_Tool_Handler.cpp: Add escape_fts_query() function to properly escape single quotes in FTS5 MATCH clauses; update all FTS and vector MATCH queries to use escaped values - lib/Static_Harvester.cpp: Add is_valid_schema_name() validation function to ensure schema names only contain safe characters (alphanumeric, underscore, dollar sign) before using in INFORMATION_SCHEMA queries - lib/Query_Tool_Handler.cpp: Add clarifying comments to validate_readonly_query explaining the blacklist (quick exit) + whitelist (allowed query types) approach - Remove backup file lib/Anomaly_Detector.cpp.bak Addresses gemini-code-assist review comments from PR #26.	2 months ago
Rene Cannao	0db022a179	Apply fixes	2 months ago
Rene Cannao	2dfd61a958	Replace remaining direct sqlite3_* calls with proxy_sqlite3_* equivalents (address code-review)	2 months ago
Rene Cannao	5d08deca7d	Fix AI agent review issues - Address SQL injection vulnerabilities by adding input validation and escaping - Fix configuration variable handling in get_variable and set_variable methods for RAG variables - Make embedding dimension configurable for rag_vec_chunks table - Remove code duplication in SQL filter building logic by creating consolidated build_sql_filters function - Update all search tools (FTS, vector, hybrid) to use consolidated filter building	2 months ago
Rene Cannao	ad166c6b8a	docs: Add comprehensive Doxygen documentation for RAG subsystem - Enhanced inline Doxygen comments in RAG_Tool_Handler.h and RAG_Tool_Handler.cpp - Added detailed parameter descriptions, return values, and cross-references - Created Doxyfile for documentation generation - Added documentation summary and guidelines - Documented all RAG tools with their schemas and usage patterns - Added security and performance considerations documentation The RAG subsystem is now fully documented with comprehensive Doxygen comments that provide clear guidance for developers working with the codebase.	2 months ago
Rene Cannao	55715ecc4b	feat: Complete RAG implementation according to blueprint specifications - Fully implemented rag.search_hybrid tool with both fuse and fts_then_vec modes - Added complete filter support across all search tools (source_ids, source_names, doc_ids, post_type_ids, tags_any, tags_all, created_after, created_before, min_score) - Implemented proper score normalization (higher is better) for all search modes - Updated all tool schemas to match blueprint specifications exactly - Added metadata inclusion in search results - Implemented Reciprocal Rank Fusion (RRF) scoring for hybrid search - Enhanced error handling and input validation - Added debug information for hybrid search ranking - Updated documentation and created completion summary This completes the v0 RAG implementation according to the blueprint requirements.	2 months ago
Rene Cannao	1dc5eb6581	fix: Fix RAG implementation compilation issues - Use public GenAI_Thread embed_documents() method instead of private LLM_Bridge get_text_embedding() - Fix signedness comparison warning in validate_query_length() - Fix JSON ternary operator type mismatch - Remove unused variables to eliminate warnings - Add GloGATH extern declaration	2 months ago
Rene Cannao	3daaa5c592	feat: Implement RAG (Retrieval-Augmented Generation) subsystem Adds a complete RAG subsystem to ProxySQL with: - RAG_Tool_Handler implementing all MCP tools for retrieval operations - Database schema with FTS and vector support - FTS, vector, and hybrid search capabilities - Fetch and refetch tools for document/chunk retrieval - Admin tools for monitoring - Configuration variables for RAG parameters - Comprehensive documentation and test scripts Implements v0 deliverables from RAG blueprint: - SQLite schema initialization - Source registry management - MCP tools: search_fts, search_vector, search_hybrid, get_chunks, get_docs, fetch_from_source, admin.stats - Unit/integration tests and examples	2 months ago

15 Commits (e762f91cda174ea3e6ab85bbcbdbc457280edf7f)