12 Commits (mysqlx-plugin-impl)

Author	SHA1	Message	Date
Rene Cannao	1d2f16b506	test: add 4 more unit test files with 198 assertions (Tier 1) ... Expand coverage for charset lookups, GenAI Query/LLM/MCP components. New test files (test/tap/tests/unit/): Core: - charset_find_unit-t (62 tests): all 4 proxysql_find_charset functions — edge cases for NULL returns, utf8mb3→utf8 translation, case-insensitive matching, collation preference GenAI/MCP (PROXYSQLGENAI=1): - genai_query_handler_unit-t (65 tests): validate_sql_identifier, escape_string_literal, strip_leading_comments, json_string/ json_int/json_double safe extraction, is_pgsql_protocol, is_runtime_online_status, uppercase_or_unknown - genai_llm_clients_unit-t (35 tests): is_retryable_error HTTP status codes (408/429/5xx) and CURL error codes, plus LLM_WriteCallback data accumulation - genai_mcp_endpoint_unit-t (36 tests): validate_bearer_token extraction/comparison, create_jsonrpc_response/error JSON-RPC 2.0 formatting Minimal production changes for testability: - include/Query_Tool_Handler.h: friend class + extern declarations for 8 pure helper functions - lib/Query_Tool_Handler.cpp: remove static from 8 file-local pure functions (no behavioral change) - include/MCP_Endpoint.h: friend class + new static validate_bearer_token() extracted from authenticate_request() - lib/MCP_Endpoint.cpp: extract token validation into static method - lib/LLM_Clients.cpp: remove static from is_retryable_error() and rename WriteCallback to LLM_WriteCallback (avoid collisions) Running totals: 15 test files, 718 assertions, all passing.	2 months ago
Rene Cannao	3ccf0982e6	AI: Fix retry logic bug and synchronize TAP tests ... - Fixed a bug in LLM_Bridge (LLM_Clients.cpp) where negative max_retries would prevent the initial API call from being made. - Improved numeric range validation in TAP tests by replacing atoi() with strtol() to correctly reject non-numeric suffixes (e.g., "50abc"). - Adjusted API key format validation in tests to match actual test data lengths for OpenAI and Anthropic prefixes. - Enhanced URL validation to correctly reject hosts starting with colons. - Updated test plans and added missing test cases to achieve full synchronization between planned and executed tests in: - ai_llm_retry_scenarios-t - ai_error_handling_edge_cases-t - ai_validation-t	3 months ago
Rene Cannao	1af768f932	Fix critical issues in GenAI code (PR #5339) ... This commit addresses critical issues identified in the PR review: 1. Build system redundancy (src/Makefile): - Remove redundant PROXYSQLGENAI conditionals in linking commands - The flag doesn't affect linking, so simplify to PROXYSQLCLICKHOUSE only 2. Missing PROXYSQLGENAI guard (lib/Admin_Handler.cpp): - Add #ifdef PROXYSQLGENAI around MCP VARIABLES DISK commands - Ensures MCP commands are only available when GenAI is enabled 3. Broken retry logic (lib/LLM_Clients.cpp): - Remove misleading is_retryable_error() checks that used stale values - last_curl_code and last_http_code were never updated - Simplify to retry on empty responses only (documented limitation) - Fix thread-safety: use thread_local std::mt19937 instead of rand() 4. Resource leak (lib/MySQL_Tool_Handler.cpp): - Clean up previously created connections on init failure - Both mysql_init and mysql_real_connect error paths now clean up 5. Race condition (lib/Query_Tool_Handler.cpp): - Add missing pool_lock in find_connection() - Prevents race condition when accessing connection_pool Fixes identified by automated PR review agents.	3 months ago
Rene Cannao	48bc7dd7bf	Merge v4.0 GenAI features into v3.0 with conditional compilation ... This commit merges the experimental v4.0 GenAI/MCP features into the stable v3.0 branch using conditional compilation. All v4.0 features are disabled by default and only enabled when PROXYSQLGENAI=1 is set at compile time. Changes: Build System: - Modified main Makefile to pass PROXYSQLGENAI flag to sub-makefiles - Modified deps/Makefile to conditionally build sqlite-vec and sqlite-rembed - Modified lib/Makefile to add PSQLGA flag and include GenAI object files - Modified src/Makefile to add PSQLGA flag and conditional linking Headers (wrapped with #ifdef PROXYSQLGENAI): - All 20 new GenAI header files in include/ - Modified cpp.h, proxysql_glovars.hpp, proxysql_admin.h - Modified ProxySQL_Admin_Tables_Definitions.h for GenAI/MCP tables Source Files: - All 22 new GenAI source files in lib/ wrapped with #ifdef PROXYSQLGENAI - Modified src/main.cpp for conditional global variables and init/shutdown - Modified Admin_Handler.cpp for conditional command handlers - Modified Admin_Bootstrap.cpp for conditional table registration - Modified Admin_FlushVariables.cpp for conditional variable flushing - Modified ProxySQL_Admin.cpp for conditional admin methods - Modified ProxySQL_Admin_Stats.cpp for conditional MCP stats functions - Modified proxy_sqlite3_symbols.cpp to always compile (needed by core) - Modified MySQL_Session.cpp for conditional GenAI function calls Test Files: - Renamed test_mcp_query_rules-t to test_mcp_query_rules-t.sh - Renamed test_mcp_rag_metrics-t to test_mcp_rag_metrics-t.sh - Modified anomaly_detection-t.cpp for conditional test execution Usage: # Build without GenAI (v3.0 mode - default) make clean && make build_deps -j$(nproc) && make build_lib -j$(nproc) && make build_src -j$(nproc) # Build with GenAI (v4.0 mode) make clean && PROXYSQLGENAI=1 make build_deps -j$(nproc) && PROXYSQLGENAI=1 make build_lib -j$(nproc) && PROXYSQLGENAI=1 make build_src -j$(nproc)	3 months ago
Rene Cannao	a3f0bade4e	feat: Convert NL2SQL to generic LLM bridge ... - Rename NL2SQL_Converter to LLM_Bridge for generic prompt processing - Update MySQL protocol handler from /* NL2SQL: */ to /* LLM: */ - Remove SQL-specific fields (sql_query, confidence, tables_used) - Add GENAI_OP_LLM operation type to GenAI module - Rename all genai_nl2sql_* variables to genai_llm_* - Update AI_Features_Manager to use LLM_Bridge - Deprecate ai_nl2sql_convert MCP tool with error message - LLM bridge now handles any prompt type via MySQL protocol This enables generic LLM access (summarization, code generation, translation, analysis) while preserving infrastructure for future NL2SQL implementation via Web UI + external agents.	4 months ago
Rene Cannao	2888ee3f45	Fix gemini-code-assist recommendations and implement comprehensive anomaly detection tests ... - Fix retry logic to use is_retryable_error function for proper HTTP error handling - Add exception handling to get_json_int function with try-catch around std::stoi - Improve validate_numeric_range to use strtol instead of atoi for better error reporting - Fix Chinese characters in documentation (non-zero -> non-zero) - Replace placeholder tests with actual comprehensive tests for anomaly detection functionality - Create new standalone unit test anomaly_detector_unit-t.cpp with 29 tests covering: * SQL injection pattern detection (12 tests) * Query normalization (8 tests) * Risk scoring calculations (5 tests) * Configuration validation (4 tests) - All tests pass successfully, providing meaningful validation of core anomaly detection logic Thanks to gemini-code-assist for the thorough code review and recommendations.	4 months ago
Rene Cannao	ae4200dbc0	Enhance AI features with improved validation, memory safety, error handling, and performance monitoring ... - Rename validate_provider_name to validate_provider_format for clarity - Add null checks and error handling for all strdup() operations - Enhance error messages with more context and HTTP status codes - Implement performance monitoring with timing metrics for LLM calls and cache operations - Add comprehensive test coverage for edge cases, retry scenarios, and performance - Extend status variables to track performance metrics - Update MySQL session to report timing information to AI manager	4 months ago
Rene Cannao	8f38b8a577	feat: Add exponential backoff retry for transient LLM failures ... This commit adds configurable retry logic with exponential backoff for NL2SQL LLM API calls. Changes: - Add retry configuration to NL2SQLRequest (max_retries, retry_backoff_ms, retry_multiplier, retry_max_backoff_ms) - Add is_retryable_error() to identify retryable HTTP/CURL errors - Add sleep_with_jitter() for exponential backoff with 10% jitter - Add call_generic_openai_with_retry() wrapper - Add call_generic_anthropic_with_retry() wrapper - Update NL2SQL_Converter::convert() to use retry wrappers Default retry behavior: - 3 retries with 1000ms initial backoff - 2.0x multiplier, 30000ms max backoff - Retries on empty responses (transient failures) Part of: Phase 3 of NL2SQL improvement plan	4 months ago
Rene Cannao	d0dc36ac0b	feat: Add structured logging with timing and request IDs ... Add comprehensive structured logging for NL2SQL LLM API calls with request correlation, timing metrics, and detailed error context. Changes: - Add request_id field to NL2SQLRequest with UUID-like auto-generation - Add structured logging macros: * LOG_LLM_REQUEST: Logs URL, model, prompt length with request ID * LOG_LLM_RESPONSE: Logs HTTP status, duration_ms, response preview * LOG_LLM_ERROR: Logs error phase, message, and status code - Update call_generic_openai() signature to accept req_id parameter - Update call_generic_anthropic() signature to accept req_id parameter - Add timing metrics to both LLM call functions using clock_gettime() - Replace existing debug logging with structured logging macros - Update convert() to pass request_id to LLM calls Request IDs are generated as UUID-like strings (e.g., "12345678-9abc-def0-1234-567890abcdef") and are included in all log messages for correlation. This allows tracking a single NL2SQL request through all log lines from request to response. Timing is measured using CLOCK_MONOTONIC for accurate duration tracking of LLM API calls, reported in milliseconds. This provides much better debugging capability when troubleshooting NL2SQL issues, as administrators can now: - Correlate all log lines for a single request - See exact timing of LLM API calls - Identify which phase of processing failed - Track request/response metrics Fixes #2 - Add Structured Logging	4 months ago
Rene Cannao	897d306d2d	Refactor: Simplify NL2SQL to use only generic providers ... Remove Ollama-specific provider code and use only generic OpenAI-compatible and Anthropic-compatible providers. Ollama is now used via its OpenAI-compatible endpoint at /v1/chat/completions. Changes: - Remove LOCAL_OLLAMA from ModelProvider enum - Remove ai_nl2sql_ollama_model and ai_nl2sql_ollama_url variables - Remove call_ollama() function from LLM_Clients.cpp - Update default configuration to use OpenAI provider with Ollama URL - Update all documentation to reflect generic-only approach Configuration: - ai_nl2sql_provider: 'openai' or 'anthropic' (default: 'openai') - ai_nl2sql_provider_url: endpoint URL (default: Ollama OpenAI-compatible) - ai_nl2sql_provider_model: model name - ai_nl2sql_provider_key: API key (optional for local endpoints) This simplifies the codebase by removing a separate code path for Ollama and aligns with the goal of avoiding provider-specific variables.	4 months ago
Rene Cannao	4f45c25945	docs: Add comprehensive doxygen comments to NL2SQL headers and LLM_Clients ... - Add file-level doxygen documentation with @file, @brief, @date, @version - Add detailed class and method documentation with @param, @return, @note, @see - Document data structures (NL2SQLRequest, NL2SQLResult, ModelProvider) - Add section comments and inline documentation for implementation files - Document all three LLM provider APIs (Ollama, OpenAI, Anthropic)	4 months ago
Rene Cannao	147a059781	feat: Add NL2SQL converter with hybrid LLM support ... - Add NL2SQL_Converter with prompt building and model selection - Add LLM clients for Ollama, OpenAI, Anthropic APIs - Update Makefile for new source files	4 months ago