18 Commits (200fdecb8d870db39abe622f3ff875cd1abb0948)

Author	SHA1	Message	Date
Rene Cannao	48bc7dd7bf	Merge v4.0 GenAI features into v3.0 with conditional compilation ... This commit merges the experimental v4.0 GenAI/MCP features into the stable v3.0 branch using conditional compilation. All v4.0 features are disabled by default and only enabled when PROXYSQLGENAI=1 is set at compile time. Changes: Build System: - Modified main Makefile to pass PROXYSQLGENAI flag to sub-makefiles - Modified deps/Makefile to conditionally build sqlite-vec and sqlite-rembed - Modified lib/Makefile to add PSQLGA flag and include GenAI object files - Modified src/Makefile to add PSQLGA flag and conditional linking Headers (wrapped with #ifdef PROXYSQLGENAI): - All 20 new GenAI header files in include/ - Modified cpp.h, proxysql_glovars.hpp, proxysql_admin.h - Modified ProxySQL_Admin_Tables_Definitions.h for GenAI/MCP tables Source Files: - All 22 new GenAI source files in lib/ wrapped with #ifdef PROXYSQLGENAI - Modified src/main.cpp for conditional global variables and init/shutdown - Modified Admin_Handler.cpp for conditional command handlers - Modified Admin_Bootstrap.cpp for conditional table registration - Modified Admin_FlushVariables.cpp for conditional variable flushing - Modified ProxySQL_Admin.cpp for conditional admin methods - Modified ProxySQL_Admin_Stats.cpp for conditional MCP stats functions - Modified proxy_sqlite3_symbols.cpp to always compile (needed by core) - Modified MySQL_Session.cpp for conditional GenAI function calls Test Files: - Renamed test_mcp_query_rules-t to test_mcp_query_rules-t.sh - Renamed test_mcp_rag_metrics-t to test_mcp_rag_metrics-t.sh - Modified anomaly_detection-t.cpp for conditional test execution Usage: # Build without GenAI (v3.0 mode - default) make clean && make build_deps -j$(nproc) && make build_lib -j$(nproc) && make build_src -j$(nproc) # Build with GenAI (v4.0 mode) make clean && PROXYSQLGENAI=1 make build_deps -j$(nproc) && PROXYSQLGENAI=1 make build_lib -j$(nproc) && PROXYSQLGENAI=1 make build_src -j$(nproc)	2 weeks ago
Wazir Ahmed	eb495f42ee	AI: Fix vector_db table creation ... Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	4 weeks ago
Wazir Ahmed	7167f95247	AI: Enable extensions for vector_db ... Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	4 weeks ago
Wazir Ahmed	4031f85394	AI: Fix vector_db initialization ... Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	4 weeks ago
Rene Cannao	5d08deca7d	Fix AI agent review issues ... - Address SQL injection vulnerabilities by adding input validation and escaping - Fix configuration variable handling in get_variable and set_variable methods for RAG variables - Make embedding dimension configurable for rag_vec_chunks table - Remove code duplication in SQL filter building logic by creating consolidated build_sql_filters function - Update all search tools (FTS, vector, hybrid) to use consolidated filter building	1 month ago
Rene Cannao	3daaa5c592	feat: Implement RAG (Retrieval-Augmented Generation) subsystem ... Adds a complete RAG subsystem to ProxySQL with: - RAG_Tool_Handler implementing all MCP tools for retrieval operations - Database schema with FTS and vector support - FTS, vector, and hybrid search capabilities - Fetch and refetch tools for document/chunk retrieval - Admin tools for monitoring - Configuration variables for RAG parameters - Comprehensive documentation and test scripts Implements v0 deliverables from RAG blueprint: - SQLite schema initialization - Source registry management - MCP tools: search_fts, search_vector, search_hybrid, get_chunks, get_docs, fetch_from_source, admin.stats - Unit/integration tests and examples	1 month ago
Rene Cannao	8a395b9b47	style: Add spaces around commas in SQL CREATE TABLE statements ... Format column definitions in CREATE TABLE IF NOT EXISTS statements to have a space before and after each comma (e.g., " , "). This allows ProxySQL Admin to properly display multi-line table schemas. Modified files: - Discovery_Schema.cpp - MySQL_Catalog.cpp - AI_Features_Manager.cpp	1 month ago
Rene Cannao	a3f0bade4e	feat: Convert NL2SQL to generic LLM bridge ... - Rename NL2SQL_Converter to LLM_Bridge for generic prompt processing - Update MySQL protocol handler from /* NL2SQL: */ to /* LLM: */ - Remove SQL-specific fields (sql_query, confidence, tables_used) - Add GENAI_OP_LLM operation type to GenAI module - Rename all genai_nl2sql_* variables to genai_llm_* - Update AI_Features_Manager to use LLM_Bridge - Deprecate ai_nl2sql_convert MCP tool with error message - LLM bridge now handles any prompt type via MySQL protocol This enables generic LLM access (summarization, code generation, translation, analysis) while preserving infrastructure for future NL2SQL implementation via Web UI + external agents.	1 month ago
Rene Cannao	51fd51e3f5	fix: Add missing GenAI_Thread.h include and fix variables reference ... Added missing #include for GenAI_Thread.h in AI_Features_Manager.cpp to resolve the compilation error in debug mode. Also fixed the remaining reference to variables.ai_features_enabled which should now use GloGATH->variables.genai_enabled. This fixes the "make debug" build failure.	1 month ago
Rene Cannao	527bfed297	fix: Migrate AI variables to GenAI module for proper architecture ... This commit fixes a serious design flaw where AI configuration variables were not integrated with the ProxySQL admin interface. All ai_* variables have been migrated to the GenAI module as genai-* variables. Changes: - Added 21 new genai_* variables to GenAI_Thread.h structure - Implemented get/set functions for all new variables in GenAI_Thread.cpp - Removed internal variables struct from AI_Features_Manager - AI_Features_Manager now reads from GloGATH instead of internal state - Updated documentation to reference genai-* variables - Fixed debug.cpp assertion for PROXY_DEBUG_NL2SQL and PROXY_DEBUG_ANOMALY Variable mapping: - ai_nl2sql_enabled → genai-nl2sql_enabled - ai_anomaly_detection_enabled → genai-anomaly_enabled - ai_features_enabled → genai-enabled - All other ai_* variables follow the same pattern The flush functions automatically handle all variables in the genai_thread_variables_names array, so database persistence works correctly without additional changes. Related to: https://github.com/ProxySQL/proxysql-vec/pull/13	1 month ago
Rene Cannao	2888ee3f45	Fix gemini-code-assist recommendations and implement comprehensive anomaly detection tests ... - Fix retry logic to use is_retryable_error function for proper HTTP error handling - Add exception handling to get_json_int function with try-catch around std::stoi - Improve validate_numeric_range to use strtol instead of atoi for better error reporting - Fix Chinese characters in documentation (non-zero -> non-zero) - Replace placeholder tests with actual comprehensive tests for anomaly detection functionality - Create new standalone unit test anomaly_detector_unit-t.cpp with 29 tests covering: * SQL injection pattern detection (12 tests) * Query normalization (8 tests) * Risk scoring calculations (5 tests) * Configuration validation (4 tests) - All tests pass successfully, providing meaningful validation of core anomaly detection logic Thanks to gemini-code-assist for the thorough code review and recommendations.	1 month ago
Rene Cannao	ae4200dbc0	Enhance AI features with improved validation, memory safety, error handling, and performance monitoring ... - Rename validate_provider_name to validate_provider_format for clarity - Add null checks and error handling for all strdup() operations - Enhance error messages with more context and HTTP status codes - Implement performance monitoring with timing metrics for LLM calls and cache operations - Add comprehensive test coverage for edge cases, retry scenarios, and performance - Extend status variables to track performance metrics - Update MySQL session to report timing information to AI manager	1 month ago
Rene Cannao	49092e9c8d	test: Add unit tests for AI configuration validation ... This commit adds comprehensive unit tests for the AI configuration validation functions used in AI_Features_Manager. Changes: - Add test/tap/tests/ai_validation-t.cpp with 61 unit tests - Test URL format validation (validate_url_format) - Test API key format validation (validate_api_key_format) - Test numeric range validation (validate_numeric_range) - Test provider name validation (validate_provider_name) - Test edge cases and boundary conditions The test file is self-contained with its own copies of the validation functions to avoid complex linking dependencies on libproxysql. Test Categories: - URL validation: 15 tests (http://, https:// protocols) - API key validation: 14 tests (OpenAI, Anthropic formats) - Numeric range: 13 tests (min/max boundaries) - Provider name: 8 tests (openai, anthropic) - Edge cases: 11 tests (NULL handling, long values) All 61 tests pass successfully. Part of: Phase 4 of NL2SQL improvement plan	1 month ago
Rene Cannao	40b2608c2d	feat: Add configuration validation to AI_Features_Manager ... Add comprehensive validation for AI features configuration variables to prevent invalid states and improve error messages. Changes: - Add validate_url_format(): Checks for http:// or https:// prefix and host part - Add validate_api_key_format(): Validates API key format, checks for whitespace, minimum length, and incomplete key patterns (sk- with <20 chars, sk-ant- with <25 chars) - Add validate_numeric_range(): Validates numeric values are within min/max range - Add validate_provider_name(): Ensures provider is 'openai' or 'anthropic' - Update set_variable() to call validation functions before setting values Validated variables: - ai_nl2sql_provider: Must be 'openai' or 'anthropic' - ai_nl2sql_provider_url: Must have http:// or https:// prefix - ai_nl2sql_provider_key: No whitespace, minimum 10 chars - ai_nl2sql_cache_similarity_threshold: Range [0, 100] - ai_nl2sql_timeout_ms: Range [1000, 300000] (1 second to 5 minutes) - ai_nl2sql_max_cloud_requests_per_hour: Range [1, 10000] - ai_anomaly_similarity_threshold: Range [0, 100] - ai_anomaly_risk_threshold: Range [0, 100] - ai_anomaly_rate_limit: Range [1, 10000] - ai_vector_dimension: Range [128, 4096] This prevents misconfigurations and provides clear error messages to users when invalid values are provided. Fixes compilation issue by moving validation helper functions before set_variable() to resolve forward declaration errors.	1 month ago
Rene Cannao	897d306d2d	Refactor: Simplify NL2SQL to use only generic providers ... Remove Ollama-specific provider code and use only generic OpenAI-compatible and Anthropic-compatible providers. Ollama is now used via its OpenAI-compatible endpoint at /v1/chat/completions. Changes: - Remove LOCAL_OLLAMA from ModelProvider enum - Remove ai_nl2sql_ollama_model and ai_nl2sql_ollama_url variables - Remove call_ollama() function from LLM_Clients.cpp - Update default configuration to use OpenAI provider with Ollama URL - Update all documentation to reflect generic-only approach Configuration: - ai_nl2sql_provider: 'openai' or 'anthropic' (default: 'openai') - ai_nl2sql_provider_url: endpoint URL (default: Ollama OpenAI-compatible) - ai_nl2sql_provider_model: model name - ai_nl2sql_provider_key: API key (optional for local endpoints) This simplifies the codebase by removing a separate code path for Ollama and aligns with the goal of avoiding provider-specific variables.	1 month ago
Rene Cannao	1c7cd8c2b1	fix: Correct PROXY_DEBUG constant from AI_GENERIC to GENAI	1 month ago
Rene Cannao	fec7d64093	feat: Implement NL2SQL vector cache with GenAI embedding generation ... Implemented semantic caching for NL2SQL using sqlite-vec and GenAI module: Changes to lib/AI_Features_Manager.cpp: - Create virtual vec0 tables for similarity search: * nl2sql_cache_vec for NL2SQL cache * anomaly_patterns_vec for threat patterns * query_history_vec for query history Changes to include/NL2SQL_Converter.h: - Add get_query_embedding() method declaration Changes to lib/NL2SQL_Converter.cpp: - Add GenAI_Thread.h include and GloGATH extern - Implement get_query_embedding() - calls GloGATH->embed_documents() - Implement check_vector_cache() - sqlite-vec KNN search with cosine distance - Implement store_in_vector_cache() - stores embedding and updates vec table - Implement clear_cache() - deletes from both main and vec tables - Implement get_cache_stats() - returns cache entry/hit counts - Add vector_to_json() helper for sqlite-vec MATCH queries Features: - Uses GenAI module (llama-server) for embedding generation - Cosine similarity search via sqlite-vec vec_distance_cosine() - Configurable similarity threshold (ai_nl2sql_cache_similarity_threshold) - Automatic hit counting and timestamp tracking	1 month ago
Rene Cannao	d9346fe64d	feat: Add AI features manager foundation ... - Add AI_Features_Manager coordinator class - Add AI_Vector_Storage interface (stub) - Add Anomaly_Detector class (stub for Phase 3) - Update includes and main initialization	1 month ago