14 Commits (fec97bbabec9d4bbddcec90fbd55496fde46a43f)

Author	SHA1	Message	Date
Rene Cannao	3ccfa2bcc1	Address AI code review feedback for PR #5313 ... This commit addresses concerns raised by AI code reviewers (gemini-code-assist, Copilot, coderabbitai) on the initial security fixes. Critical fixes: - Fix lock.release() → lock.unlock() in GenAI_Thread.cpp worker_loop (lock.release() detaches without unlocking, causing deadlock) - Add missing early return after schema validation failure in Query_Tool_Handler.cpp Code quality improvements: - Improve escape_string() memory management in MySQL_Tool_Handler.cpp: - Use std::string instead of new[]/delete[] for buffer management - Check return value of mysql_real_escape_string() for errors - Remove redundant validation checks in validate_sql_identifier functions (character class loop already rejects unsafe characters) - Add backslash escaping to escape_string_literal() for defense-in-depth - Improve column list validation in MySQL_Tool_Handler sample_rows(): - Replace blacklist approach with proper column identifier parsing - Allow qualified identifiers (table.column) - Allow AS aliases (column AS alias) - No longer rejects legitimate column names containing "JOIN" These changes improve robustness while maintaining the security posture of the original SQL injection fixes.	1 month ago
Rene Cannao	c914feb230	Fix security issues identified in PR #5312 code review ... This commit addresses critical and important security vulnerabilities found during comprehensive code review of the Gen AI features merge. Critical fixes: - SQL injection vulnerabilities in MySQL_Tool_Handler.cpp: - Added validate_sql_identifier() for schema/table validation - Added escape_string() for MySQL string escaping using mysql_real_escape_string - Fixed list_tables(), describe_table(), sample_rows(), sample_distinct() - SQL injection vulnerabilities in Query_Tool_Handler.cpp: - Added validate_sql_identifier_sqlite() for identifier validation - Added escape_string_literal() for SQLite string escaping - Fixed list_tables tool and catalog.get_relationships function - Use-after-free race condition in GenAI_Thread: - Changed shutdown_ from int to std::atomic<int> for proper memory ordering - Added additional shutdown check in worker_loop after popping request Important fixes: - Buffer overflow risks from sprintf usage: - Converted all sprintf() calls to snprintf() in GenAI_Thread.cpp - Converted sprintf() to snprintf() in MySQL_Session.cpp - Worker loop shutdown race condition: - Added shutdown check after popping request from queue - Properly clean up client_fd when shutdown is detected These fixes ensure: 1. All user input is properly validated before use in SQL queries 2. String values are properly escaped using database-specific escaping 3. Thread-safe shutdown with proper memory ordering guarantees 4. Bounds-safe string formatting to prevent buffer overflows	1 month ago
Rene Cannao	b4f521c634	Merge v3.1-MCP2 into v3.1-vec ... Conflict resolution summary: **Features preserved from v3.1-vec:** - mcp_use_ssl variable (HTTP/HTTPS mode support) - RAG tool handler support - FTS (Full Text Search) functionality in MySQL_Tool_Handler **Changes accepted from v3.1-MCP2:** - Schema isolation for catalog (removed mcp_catalog_path/fts_path variables) - Schema parameter added to catalog functions - Comprehensive query rewriting improvements in Query_Tool_Handler - Format string fixes in vector_db_performance-t.cpp **Key decisions:** - catalog_path/fts_path: Removed per v3.1-MCP2 commit 35b0b224 (intentional) - MySQL_Catalog.cpp: Accepted v3.1-MCP2 version (schema isolation improvements) - Query_Tool_Handler.cpp: Accepted v3.1-MCP2 version (significant improvements) - Makefile: Kept v3.1-vec's EXCLUDE_TRACKING_VARIABLES (correct spelling) **Build notes:** - MySQL_Tool_Handler now requires schema parameter for catalog operations - Catalog path is hardcoded to datadir/mcp_catalog.db - FTS functionality preserved but may need updates for schema isolation	1 month ago
Rene Cannao	02918d18b8	Fix PR #25 Review: All AI code reviewer feedback addressed ... This commit addresses all recommendations from CodeRabbit, Gemini Code Assist, and Copilot for PR #25 (FTS security and code quality improvements). Critical Security Fixes: - MCP_Thread.cpp: Rollback fts_path on reset failure to keep config consistent - MySQL_FTS.cpp: Add escape_mysql_identifier() for MySQL query identifier escaping - MySQL_FTS.cpp: Add unique hash-based fallback to sanitize_name() for empty strings - MySQL_FTS.cpp: Add where_clause validation to block dangerous SQL patterns Memory Safety Fixes: - MySQL_FTS.cpp: Fix indexes_result memory leak on early return in search() - MySQL_FTS.h: Delete copy/move operations to prevent accidental resource duplication Thread Safety Documentation: - MySQL_Tool_Handler.cpp: Add comment explaining FTS lock design rationale Test Script Improvements: - test_mcp_fts.sh: Add curl timeouts (5s connect, 30s max) - test_mcp_fts.sh: Remove unused delete_response variable - test_mcp_fts_detailed.sh: Make cleanup tolerant of non-existent indexes Build Fixes: - Makefile: Fix EXCLUDE_TRACKING_VARAIABLES typo to EXCLUDE_TRACKING_VARIABLES - vector_db_performance-t.cpp: Fix printf format specifiers to %lld with cast Schema Fixes: - Query_Tool_Handler.cpp: Change fts_index_table columns schema from string to array Code Cleanup: - MySQL_Tool_Handler.cpp: Remove all remaining debug fprintf statements (34 lines) - Documentation: Change "Full Text" to "Full-Text" (hyphenated) Total: ~50 fixes across 10 files	1 month ago
Rene Cannao	a10c09bcc9	Fix PR #21 review: Security, memory safety, thread safety, and code cleanup ... Security fixes: - Add escape_identifier() helper for proper SQLite identifier escaping - Replace sanitize_name() with allowlist validation (ASCII letters, digits, underscore only) - Fix MATCH clause FTS5 operator injection by wrapping query in double quotes - Apply escape_identifier() to all DDL statements (CREATE, DROP, triggers) Memory safety fixes: - Replace VLA with std::vector in MySQL_FTS::init(), add delete on error path - Fix memory leak: free error string before return in list_indexes() - Fix reindex_json["error"] potential exception using .value() with default Thread safety fixes: - reinit_fts(): Add mutex lock around pointer swap - reset_fts_path(): Move blocking init() outside lock, only swap pointer under lock Code cleanup: - Remove 7 debug fprintf statements from Query_Tool_Handler.cpp - Remove unused #include <memory> from MySQL_FTS.h Test script security fixes: - Use MYSQL_PWD environment variable instead of -p"..." for password - Add escape_sql() function and apply to INSERT statement - Fix CURL_OPTS quoting: ${CURL_OPTS:+"${CURL_OPTS}"} - Remove unused FTS_INDEX_NAME and SEARCH_QUERIES variables Documentation fixes: - Fix bare URL to markdown link format - Add code block language identifiers (text, bash)	1 month ago
Rahim Kanji	0d56918744	Add full-text search (FTS) tools to MCP query server ... Implement BM25-ranked full-text search capability for MySQL/MariaDB tables using SQLite-based external FTS index. Changes: - Add MySQL_FTS class for managing SQLite FTS indexes - Add FTS tools: fts_index_table, fts_search, fts_reindex, fts_delete_index, fts_list_indexes, fts_rebuild_all - Add thread-safe FTS lifecycle management with fts_lock mutex - Add reset_fts_path() for runtime FTS database path configuration - Add comprehensive FTS test scripts (test_mcp_fts.sh, test_mcp_fts_detailed.sh)	1 month ago
Rene Cannao	7de3f0c510	feat: Add schema separation to MCP catalog and discovery scope constraint ... This commit addresses two issues: 1. MCP Catalog Schema Separation: - Add 'schema' column to catalog table for proper isolation - Update all catalog methods (upsert, get, search, list, remove) to accept schema parameter - Update MCP tool handlers and JSON-RPC parameter parsing - Unique constraint changed from (kind, key) to (schema, kind, key) - FTS table updated to include schema column 2. Discovery Prompt Scope Constraint: - Add explicit SCOPE CONSTRAINT section to multi_agent_discovery_prompt.md - Agents now respect Target Schema and skip list_schemas when specified - Prevents analyzing all schemas when only one is targeted Files modified: - include/MySQL_Catalog.h: Add schema parameter to all catalog methods - include/MySQL_Tool_Handler.h: Update wrapper method signatures - lib/MySQL_Catalog.cpp: Implement schema filtering in all operations - lib/MySQL_Tool_Handler.cpp: Update wrapper implementations - lib/Query_Tool_Handler.cpp: Extract schema from JSON-RPC params, update tool descriptions - scripts/mcp/DiscoveryAgent/ClaudeCode_Headless/prompts/multi_agent_discovery_prompt.md: Add scope constraint	1 month ago
Rene Cannao	f852900365	Fix: Correct MCP catalog JSON parsing to handle special characters ... The catalog_search() and catalog_list() methods in MySQL_Catalog.cpp were manually building JSON strings by concatenating raw TEXT from SQLite without proper escaping. This caused parse errors when stored JSON contained quotes, backslashes, or newlines. Changes: - MySQL_Catalog.cpp: Use nlohmann::json to build proper nested JSON in search() and list() methods instead of manual concatenation - MySQL_Tool_Handler.cpp: Add try-catch for JSON parsing in catalog_get() - test_catalog.sh: Fix MCP URL path, add jq extraction for MCP protocol responses, add 3 special character tests (CAT013-CAT015) Test Results: All 15 catalog tests pass, including new tests that verify special characters (quotes, backslashes) are preserved.	1 month ago
Rene Cannao	ef5b99edbf	Fix MCP tool bugs: NULL value handling and query validation ... - Fixed NULL value handling in execute_query: use empty string instead of nullptr to avoid "basic_string: construction from null" errors - Fixed validate_readonly_query: corrected substring length check from substr(0,6)!="SELECT " to substr(0,6)!="SELECT" - Fixed test script: added proper variable_name parameter for get_config/set_config tools Query endpoint tools now pass all tests.	1 month ago
Rene Cannao	acb4c57db3	Fix case sensitivity issues in MySQL_Tool_Handler::execute_query ... MySQL returns column names in uppercase for information_schema tables, but the code was expecting lowercase column names. This caused crashes when accessing JSON keys that didn't exist. Changes: 1. Convert all column names to lowercase in execute_query() 2. Store lowercase column names in a vector for efficient access 3. Use lowercase column names as keys in JSON row objects This ensures consistent column name casing across all queries, preventing JSON access errors for information_schema columns. Also includes the previous use-after-free fix.	1 month ago
Rene Cannao	904283330a	Fix critical use-after-free bug in MySQL_Tool_Handler::execute_query ... The code was creating a dangling pointer by calling c_str() on a temporary std::string object, causing undefined behavior and crashes when processing query results. Before: const char* col_name = columns[i].get<std::string>().c_str(); // ^ temporary string destroyed here, col_name is dangling After: std::string col_name = columns[i].get<std::string>(); // ^ col_name is valid until end of scope This bug was causing ProxySQL to crash when running MCP tool tests.	1 month ago
Rene Cannao	06aa6d6ef7	Add comprehensive Doxygen documentation for connection pool ... Added missing documentation for MySQL connection pool implementation: Header (MySQL_Tool_Handler.h): - Added MySQLConnection struct documentation with member descriptions - Added member variable documentation using ///< Doxygen style Implementation (MySQL_Tool_Handler.cpp): - Added Doxygen blocks for close() method - Added Doxygen blocks for init_connection_pool() with detailed behavior - Added Doxygen blocks for get_connection() with thread-safety notes - Added Doxygen blocks for return_connection() with reuse behavior - Added Doxygen blocks for execute_query() with JSON format documentation All new connection pool methods now have complete @brief, @param, and @return documentation following Doxygen conventions.	1 month ago
Rene Cannao	4eab519848	Implement MySQL connection pool for MySQL_Tool_Handler ... Added built-in connection pool to MySQL_Tool_Handler for direct MySQL connections to backend servers. Changes: - Added MySQLConnection struct with MYSQL* pointer, host, port, in_use flag - Added connection_pool vector, pool_lock mutex, pool_size counter - Implemented init_connection_pool() to create MYSQL connections using mysql_init/mysql_real_connect - Implemented get_connection() and return_connection() with thread-safe locking - Implemented execute_query() helper method for executing SQL and returning JSON results - Updated tool methods to use actual MySQL connections: - list_schemas: Query information_schema.schemata - list_tables: Query information_schema.tables with metadata - describe_table: Query columns, primary keys, indexes - sample_rows: Execute SELECT with LIMIT - sample_distinct: Execute SELECT DISTINCT with GROUP BY - run_sql_readonly: Execute validated SELECT queries - explain_sql: Execute EXPLAIN queries - Fixed MYSQL forward declaration (use typedef struct st_mysql MYSQL) The connection pool creates one connection per configured host:port pair with 5-second timeouts for connect/read/write operations.	1 month ago
Rene Cannao	221ff23991	Add MySQL exploration MCP tools with SQLite catalog ... Implemented MCP (Model Context Protocol) server providing tools for LLM-based MySQL database exploration: - MySQL_Catalog: SQLite-based catalog for LLM external memory with upsert, get, search, list, merge, delete operations and FTS support - MySQL_Tool_Handler: 17+ database exploration tools with guardrails: * Inventory: list_schemas, list_tables * Structure: describe_table, get_constraints, describe_view * Profiling: table_profile, column_profile * Sampling: sample_rows (max 20), sample_distinct (max 50) * Query: run_sql_readonly (max 200 rows, 2s timeout, SELECT-only) * Relationship: suggest_joins, find_reference_candidates * Catalog: catalog_upsert, catalog_get, catalog_search, catalog_list, catalog_merge, catalog_delete - MCP Module Integration: * Added 6 new configuration variables for MySQL tool handler (mysql_hosts, mysql_ports, mysql_user, mysql_password, mysql_schema, catalog_path) * Added MySQL_Tool_Handler pointer to MCP_Threads_Handler * Implemented tool routing in MCP endpoint for tools/list, tools/describe, and tools/call methods - TAP Tests: Updated to expect 14 MCP variables (was 8) Files: - include/MySQL_Catalog.h, lib/MySQL_Catalog.cpp - include/MySQL_Tool_Handler.h, lib/MySQL_Tool_Handler.cpp - include/MCP_Thread.h, lib/MCP_Thread.cpp - include/MCP_Endpoint.h, lib/MCP_Endpoint.cpp - lib/Makefile, test/tap/tests/mcp_module-t.cpp	1 month ago