31 Commits (499896e44bd6f7e91b64a2a93083bc4dda3d9ce7)

Author	SHA1	Message	Date
Rene Cannao	acaa100dc8	fix: address PR review feedback from coding agents ... Actionable fixes from CodeRabbit / agent reviews on PR #5549: 1. include/Stats_Tool_Handler.h: add #include <cstdint> — uint64_t used in parse_digest_filter() without direct include 2. lib/MCP_Endpoint.cpp: use strncasecmp() for Bearer token prefix check — RFC 7235 Section 2.1 says auth-scheme is case-insensitive. This is a real bug fix in production code. 3. test/tap/tests/unit/sqlite3db_unit-t.cpp: guard result->rows_count dereference with null check to avoid segfault if execute_statement returns null 4. test/tap/tests/unit/genai_mcp_endpoint_unit-t.cpp: remove memset() calls on MCP_Threads_Handler that leaked constructor's strdup("") allocations for auth fields 5. test/tap/groups/check_groups.py: use os.walk() instead of hardcoded directory list so new test subdirectories are discovered 6. test/tap/tests/unit/admin_disk_upgrade_unit-t.cpp: add comment documenting intentional calloc pattern (ProxySQL_Admin constructor has too many daemon deps for unit tests)	1 week ago
Rene Cannao	1d2f16b506	test: add 4 more unit test files with 198 assertions (Tier 1) ... Expand coverage for charset lookups, GenAI Query/LLM/MCP components. New test files (test/tap/tests/unit/): Core: - charset_find_unit-t (62 tests): all 4 proxysql_find_charset functions — edge cases for NULL returns, utf8mb3→utf8 translation, case-insensitive matching, collation preference GenAI/MCP (PROXYSQLGENAI=1): - genai_query_handler_unit-t (65 tests): validate_sql_identifier, escape_string_literal, strip_leading_comments, json_string/ json_int/json_double safe extraction, is_pgsql_protocol, is_runtime_online_status, uppercase_or_unknown - genai_llm_clients_unit-t (35 tests): is_retryable_error HTTP status codes (408/429/5xx) and CURL error codes, plus LLM_WriteCallback data accumulation - genai_mcp_endpoint_unit-t (36 tests): validate_bearer_token extraction/comparison, create_jsonrpc_response/error JSON-RPC 2.0 formatting Minimal production changes for testability: - include/Query_Tool_Handler.h: friend class + extern declarations for 8 pure helper functions - lib/Query_Tool_Handler.cpp: remove static from 8 file-local pure functions (no behavioral change) - include/MCP_Endpoint.h: friend class + new static validate_bearer_token() extracted from authenticate_request() - lib/MCP_Endpoint.cpp: extract token validation into static method - lib/LLM_Clients.cpp: remove static from is_retryable_error() and rename WriteCallback to LLM_WriteCallback (avoid collisions) Running totals: 15 test files, 718 assertions, all passing.	2 weeks ago
Rene Cannao	33fce1ca35	MCP stats: address review findings and add detailed docs	2 months ago
René Cannaò	762afaa44f	Merge branch 'v3.0' into v4.0-mcp-stats ... Signed-off-by: René Cannaò <rene@proxysql.com>	2 months ago
Rene Cannao	f4bc1943fb	mcp query diagnostics: restore strict ONLINE requirement and explain target non-executable failures ... This change does two things: 1) Reverts executable-target policy to strict ONLINE backend status - Query target resolution in Query_Tool_Handler is restored to require UPPER(status)='ONLINE' in runtime_mysql_servers/runtime_pgsql_servers. - This matches expected semantics: non-ONLINE backends should not be considered executable MCP targets. 2) Replaces generic non-executable errors with actionable diagnostics - Added Query_Tool_Handler::format_target_unavailable_error(target_id). - All query-tool paths that previously returned the generic: 'Target is not executable by this handler' now return a reasoned error with context, including: - target_id - protocol - hostgroup_id - auth_profile_id - concrete reason (for example: empty db_username in auth profile, or no ONLINE backend in runtime_*_servers) - Internal logs in get_connection()/get_pgsql_connection() now emit the same detailed reason. 3) Improves MCP endpoint failure logging with SQL/target context - In MCP_JSONRPC_Resource::handle_tools_call(), failed tool calls now log: - endpoint/tool/error - full arguments payload (existing) - additional parsed details when present: target_id, schema, sql (trimmed for safety) - This makes it explicit what SQL was requested when a tool call fails before execution. 4) Protocol normalization retained for robustness - MCP_Threads_Handler::load_target_auth_map() lowercases protocol values when loading runtime profile joins. - This avoids protocol-casing drift causing inconsistent routing behavior. Build validation: - Successfully recompiled modified objects: - lib/obj/Query_Tool_Handler.oo - lib/obj/MCP_Endpoint.oo - lib/obj/MCP_Thread.oo Resulting behavior expected on next test run: - If a target is non-executable, the response and logs will state exactly why (ONLINE/backend/auth reason), instead of a generic error. - Failure logs will also include the attempted SQL text (for tool calls that carry sql arguments), clarifying whether a backend query was actually executed or blocked before execution.	2 months ago
Wazir Ahmed	ae92ae265f	Merge branch 'v3.0' into v4.0-mcp-stats	2 months ago
Wazir Ahmed	643b322f29	MCP: Add stats endpoint and tools ... Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	2 months ago
Rene Cannao	b965fc6df4	Fix PROXYSQLGENAI build - resolve circular includes and missing headers ... This commit fixes compilation errors when building with PROXYSQLGENAI=1: Header file fixes: - Add missing #endif /* PROXYSQLGENAI */ before header guards in multiple headers - Remove #include cpp.h from GenAI headers to avoid circular dependencies Source file fixes: - Add #include proxysql.h to GenAI .cpp files that were missing it - Add #include Static_Harvester.h to Query_Tool_Handler.cpp for forward decl Build system fixes: - Remove vec.o from libproxysql.a (it's linked separately in src/Makefile) - Prevents duplicate symbol errors during linking Test files: - Rename MCP test files with .sh suffix to prevent make clean deletion Both build modes now work: - make build_lib_debug (without GenAI) - PROXYSQLGENAI=1 make build_lib_debug (with GenAI)	2 months ago
Rene Cannao	48bc7dd7bf	Merge v4.0 GenAI features into v3.0 with conditional compilation ... This commit merges the experimental v4.0 GenAI/MCP features into the stable v3.0 branch using conditional compilation. All v4.0 features are disabled by default and only enabled when PROXYSQLGENAI=1 is set at compile time. Changes: Build System: - Modified main Makefile to pass PROXYSQLGENAI flag to sub-makefiles - Modified deps/Makefile to conditionally build sqlite-vec and sqlite-rembed - Modified lib/Makefile to add PSQLGA flag and include GenAI object files - Modified src/Makefile to add PSQLGA flag and conditional linking Headers (wrapped with #ifdef PROXYSQLGENAI): - All 20 new GenAI header files in include/ - Modified cpp.h, proxysql_glovars.hpp, proxysql_admin.h - Modified ProxySQL_Admin_Tables_Definitions.h for GenAI/MCP tables Source Files: - All 22 new GenAI source files in lib/ wrapped with #ifdef PROXYSQLGENAI - Modified src/main.cpp for conditional global variables and init/shutdown - Modified Admin_Handler.cpp for conditional command handlers - Modified Admin_Bootstrap.cpp for conditional table registration - Modified Admin_FlushVariables.cpp for conditional variable flushing - Modified ProxySQL_Admin.cpp for conditional admin methods - Modified ProxySQL_Admin_Stats.cpp for conditional MCP stats functions - Modified proxy_sqlite3_symbols.cpp to always compile (needed by core) - Modified MySQL_Session.cpp for conditional GenAI function calls Test Files: - Renamed test_mcp_query_rules-t to test_mcp_query_rules-t.sh - Renamed test_mcp_rag_metrics-t to test_mcp_rag_metrics-t.sh - Modified anomaly_detection-t.cpp for conditional test execution Usage: # Build without GenAI (v3.0 mode - default) make clean && make build_deps -j$(nproc) && make build_lib -j$(nproc) && make build_src -j$(nproc) # Build with GenAI (v4.0 mode) make clean && PROXYSQLGENAI=1 make build_deps -j$(nproc) && PROXYSQLGENAI=1 make build_lib -j$(nproc) && PROXYSQLGENAI=1 make build_src -j$(nproc)	2 months ago
Wazir Ahmed	c52c621b27	MCP: Add mcp-rag_endpoint_auth config ... Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	2 months ago
Rene Cannao	b4f521c634	Merge v3.1-MCP2 into v3.1-vec ... Conflict resolution summary: **Features preserved from v3.1-vec:** - mcp_use_ssl variable (HTTP/HTTPS mode support) - RAG tool handler support - FTS (Full Text Search) functionality in MySQL_Tool_Handler **Changes accepted from v3.1-MCP2:** - Schema isolation for catalog (removed mcp_catalog_path/fts_path variables) - Schema parameter added to catalog functions - Comprehensive query rewriting improvements in Query_Tool_Handler - Format string fixes in vector_db_performance-t.cpp **Key decisions:** - catalog_path/fts_path: Removed per v3.1-MCP2 commit 35b0b224 (intentional) - MySQL_Catalog.cpp: Accepted v3.1-MCP2 version (schema isolation improvements) - Query_Tool_Handler.cpp: Accepted v3.1-MCP2 version (significant improvements) - Makefile: Kept v3.1-vec's EXCLUDE_TRACKING_VARIABLES (correct spelling) **Build notes:** - MySQL_Tool_Handler now requires schema parameter for catalog operations - Catalog path is hardcoded to datadir/mcp_catalog.db - FTS functionality preserved but may need updates for schema isolation	3 months ago
Wazir Ahmed	e450f1b30f	MCP: Handle DELETE method ... - Respond with 405 Method Not Allowed, when clients send DELETE request for session termination. Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	3 months ago
Wazir Ahmed	68a41d6db8	MCP: Add handler for prompts and resources ... Issue ----- - ProxySQL only supports the `tools` feature of the MCP protocol and does not support features such as `prompts` and `resources`. - Although ProxySQL expresses this in its `initialize` response, (server capabilities list contains only the `tools` object), clients such as Warp Terminal ignore it and continue to send requests for methods such `prompts/list` and `resources/list`. - Any response other than `HTTP 200 OK` is treated as an error and client fails to initialize. Fix --- - Handle prompt and resource list requests by returning an empty array. Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	3 months ago
Wazir Ahmed	2f38def403	MCP: Handle client notifications properly ... - Fix incorrect method name for notification - Handle all notification messages in a generic way - Respond with HTTP 202 Accepted (no response body) Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	3 months ago
Wazir Ahmed	155a77f969	MCP: Bump protocolVersion to 2025-06-18 ... - Version 2024-11-05 only supports HTTP_SSE as transport. - ProxySQL's MCP implemenation aligns more with the StreamableHTTP transport specified in version 2025-06-18. - Support for SSE in StreamableHTTP transport is optional. Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	3 months ago
Rahim Kanji	bf429f0a52	Fixed multiple issues	3 months ago
Rahim Kanji	5b8bb1952e	Merge remote-tracking branch 'wqv3.1-vec' into v3.1_mcp-http-ssl-toggle	3 months ago
Rahim Kanji	7564306e18	Handledwq "notifications/initialized" method	3 months ago
Rahim Kanji	4a858521c9	Fix JSON-RPC ID type ... Change id parameter from string to json& to support JSON-RPC 2.0 spec (id can be string, number, or null)	3 months ago
Rahim Kanji	a15be695e0	Add GET/OPTIONS handlers for MCP HTTP transport ... - Add render_GET() returning 405 Method Not Allowed - Add render_OPTIONS()	3 months ago
Rene Cannao	757cdaff15	fix: Improve error logging and fix llm.domain_set_members ... 1. Fix error logging to catch ALL tool failures, not just those with both success and result fields. Previously, error responses like {"success": false, "error": "..."} without a result field were silently ignored. 2. Fix llm.domain_set_members to accept both array and JSON string formats for the members parameter. Some clients send it as a JSON string, others as a native array. 3. Add detailed error logging for llm.domain_set_members failures, including what was actually received.	3 months ago
Rene Cannao	d962caea7e	feat: Improve MCP error logging with request payloads ... Exception handlers now log the full request payload that caused the error, making debugging much easier. Changes: - Move req_body/req_path declarations outside try block so catch handlers can access them - Log request payload in all exception handlers (parse errors, std::exception, and catch-all) - Log tool arguments when tool execution fails Previously, exceptions would only log the error message without context, making it impossible to reproduce the issue. Now the full payload is logged.	3 months ago
Rene Cannao	53ecda7730	fix: Add comprehensive error handling and logging for MCP tools ... - Add try-catch around handle_jsonrpc_request to catch unexpected exceptions - Add detailed logging for tool execution success/failure - Add proper SQLite error checking in create_agent_run with error messages - Fix json_int/json_double to handle both numbers and numeric strings The json_int function was throwing exceptions when receiving numeric strings (e.g., "14" instead of 14) from clients, causing 500 errors. Now it handles both formats gracefully. Also added logging so tool failures are visible in logs instead of being silent 500 errors.	3 months ago
Rene Cannao	1b7335acfe	Fix two-phase discovery documentation and scripts ... - Add mcp_config.example.json for Claude Code MCP configuration - Fix MCP bridge path in example config (../../proxysql_mcp_stdio_bridge.py) - Update Two_Phase_Discovery_Implementation.md with correct Phase 1/Phase 2 usage - Fix Two_Phase_Discovery_Implementation.md DELETE FROM fts_objects to scope to run_id - Update README.md with two-phase discovery section and multi-agent legacy note - Create static_harvest.sh bash wrapper for Phase 1 - Create two_phase_discovery.py orchestration script with prompts - Add --run-id parameter to skip auto-fetch - Fix RUN_ID placeholder mismatch (<USE_THE_PROVIDED_RUN_ID>) - Fix catalog path default to mcp_catalog.db - Add test_catalog.sh to verify catalog tools work - Fix Discovery_Schema.cpp FTS5 syntax (missing space) - Remove invalid CREATE INDEX on FTS virtual tables - Add MCP tool call logging to track tool usage - Fix Static_Harvester::get_harvest_stats() to accept run_id parameter - Fix DELETE FROM fts_objects to only delete for specific run_id - Update system prompts to say DO NOT call discovery.run_static - Update user prompts to say Phase 1 is already complete - Add --mcp-only flag to restrict Claude Code to MCP tools only - Make FTS table failures non-fatal (check if table exists first) - Add comprehensive documentation for both discovery approaches	3 months ago
Rene Cannao	606fe2e93c	Fix: Address code review feedback from gemini-code-assist ... Python bridge (scripts/mcp/proxysql_mcp_stdio_bridge.py): - Make log file path configurable via PROXYSQL_MCP_BRIDGE_LOG env var - Add httpx.RequestError exception handling for network issues - Fix asyncio.CancelledError not being re-raised (HIGH priority) - Replace deprecated asyncio.get_event_loop() with get_running_loop() C++ server (lib/MCP_Endpoint.cpp): - Refactor handle_tools_call() to reduce code duplication - Handle string responses directly without calling .dump() - Single shared wrapping block for all response types Per review: https://github.com/ProxySQL/proxysql-vec/pull/11	3 months ago
Rene Cannao	49e964bb02	Fix: Make ProxySQL MCP server return MCP-compliant tool responses ... The ProxySQL MCP server now wraps tool results in the correct MCP format: - result.content: array of content items (type: "text", text: "...") - result.isError: boolean Per MCP spec: https://modelcontextprotocol.io/specification/2025-11-25/server/tools Also simplified the bridge to pass through results directly since the server now returns the correct format.	3 months ago
Rene Cannao	6d83ff1680	Fix: unwrap ProxySQL response format in MCP tools and fix config syntax ... - Unwrap ProxySQL's {"success": ..., "result": ...} wrapper in tool responses for MCP protocol compliance - Fix proxysql.cfg missing closing brace for mcp_variables section	3 months ago
Rene Cannao	ced10dd054	Implement per-endpoint authentication for MCP endpoints ... This commit implements Phase 2 of the MCP multi-endpoint architecture: per-endpoint Bearer token authentication. ## Changes ### lib/MCP_Endpoint.cpp - Implemented `authenticate_request()` method with: - Per-endpoint token validation (mcp-{endpoint}_endpoint_auth) - Bearer token support via Authorization header - Query parameter fallback (?token=xxx) for simple testing - No authentication when token is not configured (backward compatible) - Proper 401 Unauthorized response on auth failure - Token whitespace trimming - Debug logging for troubleshooting ### doc/MCP/Architecture.md - Updated Per-Endpoint Authentication section with complete implementation - Marked Phase 3 authentication task as completed (✅) - Added authentication implementation code example ## Authentication Flow 1. Client sends request with Bearer token: - Header: `Authorization: Bearer <token>` - Or query param: `?token=<token>` 2. Server validates against endpoint-specific variable: - `/mcp/config` → `mcp-config_endpoint_auth` - `/mcp/observe` → `mcp-observe_endpoint_auth` - `/mcp/query` → `mcp-query_endpoint_auth` - `/mcp/admin` → `mcp-admin_endpoint_auth` - `/mcp/cache` → `mcp-cache_endpoint_auth` 3. Returns 401 Unauthorized if: - Auth is required but not provided - Token doesn't match expected value 4. Allows request if: - No auth token configured (backward compatible) - Token matches expected value ## Testing ```bash # Set auth token for /mcp/query endpoint mysql -h 127.0.0.1 -P 6032 -u admin -padmin \ -e "SET mcp-query_endpoint_auth='my-secret-token'; LOAD MCP VARIABLES TO RUNTIME;" # Test with Bearer token curl -k -X POST https://127.0.0.1:6071/mcp/query \ -H "Content-Type: application/json" \ -H "Authorization: Bearer my-secret-token" \ -d '{"jsonrpc":"2.0","method":"tools/list","id":1}' # Test with query parameter curl -k -X POST "https://127.0.0.1:6071/mcp/query?token=my-secret-token" \ -H "Content-Type: application/json" \ -d '{"jsonrpc":"2.0","method":"tools/list","id":1}' ``` ## Status ✅ Authentication fully implemented and functional ⚠️ Testing with running ProxySQL instance still needed Co-authored-by: Claude <claude@anthropic.com>	3 months ago
Rene Cannao	c86a048d9c	Implement MCP multi-endpoint architecture with dedicated tool handlers ... This commit implements Option 1 (Multiple Tool Handlers) for the MCP module, where each of the 5 endpoints has its own dedicated tool handler with specific tools. ## Architecture Changes - Created MCP_Tool_Handler base class interface for all tool handlers - Each endpoint now has its own dedicated tool handler: - /mcp/config → Config_Tool_Handler (configuration management) - /mcp/query → Query_Tool_Handler (database exploration) - /mcp/admin → Admin_Tool_Handler (administrative operations) - /mcp/cache → Cache_Tool_Handler (cache management) - /mcp/observe → Observe_Tool_Handler (monitoring & metrics) ## New Files Base Interface: - include/MCP_Tool_Handler.h - Base class for all tool handlers Tool Handlers: - include/Config_Tool_Handler.h, lib/Config_Tool_Handler.cpp - include/Query_Tool_Handler.h, lib/Query_Tool_Handler.cpp - include/Admin_Tool_Handler.h, lib/Admin_Tool_Handler.cpp - include/Cache_Tool_Handler.h, lib/Cache_Tool_Handler.cpp - include/Observe_Tool_Handler.h, lib/Observe_Tool_Handler.cpp Documentation: - doc/MCP/Architecture.md - Comprehensive architecture documentation ## Modified Files - include/MCP_Thread.h, lib/MCP_Thread.cpp - Added 5 tool handler pointers - include/MCP_Endpoint.h, lib/MCP_Endpoint.cpp - Use tool_handler base class - lib/ProxySQL_MCP_Server.cpp - Create and pass handlers to endpoints - lib/Makefile - Added new source files ## Implementation Status - Config_Tool_Handler: Functional (get_config, set_config, list_variables, get_status) - Query_Tool_Handler: Functional (wraps MySQL_Tool_Handler, all 18 tools) - Admin_Tool_Handler: Stub implementations (TODO: implement) - Cache_Tool_Handler: Stub implementations (TODO: implement) - Observe_Tool_Handler: Stub implementations (TODO: implement) See GitHub Issue #8 for detailed TODO list. Co-authored-by: Claude <claude@anthropic.com>	3 months ago
Rene Cannao	221ff23991	Add MySQL exploration MCP tools with SQLite catalog ... Implemented MCP (Model Context Protocol) server providing tools for LLM-based MySQL database exploration: - MySQL_Catalog: SQLite-based catalog for LLM external memory with upsert, get, search, list, merge, delete operations and FTS support - MySQL_Tool_Handler: 17+ database exploration tools with guardrails: * Inventory: list_schemas, list_tables * Structure: describe_table, get_constraints, describe_view * Profiling: table_profile, column_profile * Sampling: sample_rows (max 20), sample_distinct (max 50) * Query: run_sql_readonly (max 200 rows, 2s timeout, SELECT-only) * Relationship: suggest_joins, find_reference_candidates * Catalog: catalog_upsert, catalog_get, catalog_search, catalog_list, catalog_merge, catalog_delete - MCP Module Integration: * Added 6 new configuration variables for MySQL tool handler (mysql_hosts, mysql_ports, mysql_user, mysql_password, mysql_schema, catalog_path) * Added MySQL_Tool_Handler pointer to MCP_Threads_Handler * Implemented tool routing in MCP endpoint for tools/list, tools/describe, and tools/call methods - TAP Tests: Updated to expect 14 MCP variables (was 8) Files: - include/MySQL_Catalog.h, lib/MySQL_Catalog.cpp - include/MySQL_Tool_Handler.h, lib/MySQL_Tool_Handler.cpp - include/MCP_Thread.h, lib/MCP_Thread.cpp - include/MCP_Endpoint.h, lib/MCP_Endpoint.cpp - lib/Makefile, test/tap/tests/mcp_module-t.cpp	3 months ago
Rene Cannao	87fff9e046	Add MCP (Model Context Protocol) module skeleton ... Add new MCP module supporting multiple MCP server endpoints over HTTPS with JSON-RPC 2.0 protocol skeleton. Each endpoint (/mcp/config, /mcp/observe, /mcp/query, /mcp/admin, /mcp/cache) is a distinct MCP server with its own authentication configuration. Features: - HTTPS server using existing ProxySQL TLS certificates - JSON-RPC 2.0 skeleton implementation (actual protocol TBD) - 5 MCP endpoints with per-endpoint auth configuration - LOAD/SAVE MCP VARIABLES admin commands - Configuration file support (mcp_variables section) Implementation follows GenAI module pattern: - MCP_Threads_Handler: Main module handler with variable management - ProxySQL_MCP_Server: HTTPS server wrapper using libhttpserver - MCP_JSONRPC_Resource: Base endpoint class with JSON-RPC skeleton	3 months ago