proxysql

Commit Graph

Author	SHA1	Message	Date
Rene Cannao	d7b2fea898	fix: Remove MAIN_PROXY_SQLITE3 defines from tests (v3.1-MCP2 compatibility) v3.1-MCP2 introduced lib/proxy_sqlite3_symbols.cpp which provides canonical definitions for proxy_sqlite3_* function pointers. Tests that link against libproxysql.a must NOT define MAIN_PROXY_SQLITE3, otherwise sqlite3db.h will provide conflicting definitions. Removed #define MAIN_PROXY_SQLITE3 from: - test/tap/tests/sqlite3-t.cpp - test/tap/tests/unit_test.h - test/tap/tests/setparser_test.cpp - test/tap/tests/setparser_test_common.h	3 months ago
Rene Cannao	b4f521c634	Merge v3.1-MCP2 into v3.1-vec Conflict resolution summary: Features preserved from v3.1-vec: - mcp_use_ssl variable (HTTP/HTTPS mode support) - RAG tool handler support - FTS (Full Text Search) functionality in MySQL_Tool_Handler Changes accepted from v3.1-MCP2: - Schema isolation for catalog (removed mcp_catalog_path/fts_path variables) - Schema parameter added to catalog functions - Comprehensive query rewriting improvements in Query_Tool_Handler - Format string fixes in vector_db_performance-t.cpp Key decisions: - catalog_path/fts_path: Removed per v3.1-MCP2 commit `35b0b224` (intentional) - MySQL_Catalog.cpp: Accepted v3.1-MCP2 version (schema isolation improvements) - Query_Tool_Handler.cpp: Accepted v3.1-MCP2 version (significant improvements) - Makefile: Kept v3.1-vec's EXCLUDE_TRACKING_VARIABLES (correct spelling) Build notes: - MySQL_Tool_Handler now requires schema parameter for catalog operations - Catalog path is hardcoded to datadir/mcp_catalog.db - FTS functionality preserved but may need updates for schema isolation	3 months ago
Rene Cannao	02918d18b8	Fix PR #25 Review: All AI code reviewer feedback addressed This commit addresses all recommendations from CodeRabbit, Gemini Code Assist, and Copilot for PR #25 (FTS security and code quality improvements). Critical Security Fixes: - MCP_Thread.cpp: Rollback fts_path on reset failure to keep config consistent - MySQL_FTS.cpp: Add escape_mysql_identifier() for MySQL query identifier escaping - MySQL_FTS.cpp: Add unique hash-based fallback to sanitize_name() for empty strings - MySQL_FTS.cpp: Add where_clause validation to block dangerous SQL patterns Memory Safety Fixes: - MySQL_FTS.cpp: Fix indexes_result memory leak on early return in search() - MySQL_FTS.h: Delete copy/move operations to prevent accidental resource duplication Thread Safety Documentation: - MySQL_Tool_Handler.cpp: Add comment explaining FTS lock design rationale Test Script Improvements: - test_mcp_fts.sh: Add curl timeouts (5s connect, 30s max) - test_mcp_fts.sh: Remove unused delete_response variable - test_mcp_fts_detailed.sh: Make cleanup tolerant of non-existent indexes Build Fixes: - Makefile: Fix EXCLUDE_TRACKING_VARAIABLES typo to EXCLUDE_TRACKING_VARIABLES - vector_db_performance-t.cpp: Fix printf format specifiers to %lld with cast Schema Fixes: - Query_Tool_Handler.cpp: Change fts_index_table columns schema from string to array Code Cleanup: - MySQL_Tool_Handler.cpp: Remove all remaining debug fprintf statements (34 lines) - Documentation: Change "Full Text" to "Full-Text" (hyphenated) Total: ~50 fixes across 10 files	3 months ago
Rene Cannao	fd5d433a2b	fix: Missing <string> header in ai_llm_retry_scenarios-t - Add #include <string> for C++ std::string support Resolves compilation errors for ai_llm_retry_scenarios-t test.	3 months ago
Rene Cannao	2a614f817e	fix: Missing headers and format strings in vector_db_performance-t - Add #include <string> for C++ std::string support - Add #include <cmath> for sqrt() function - Change format %lld to %ld for chrono duration types (long int, not long long) Resolves compilation errors for vector_db_performance-t test.	3 months ago
Rene Cannao	0dc3531744	fix: Linking issues for anomaly_detection-t TAP test - Add special build rule in Makefile for anomaly_detection-t that includes: - $(OBJ) for global variables (GloVars, GloGATH) - -Wl,--allow-multiple-definition to allow test's main() to override ProxySQL's - ClickHouse client libraries (libclickhouse-cpp-lib.a, libzstdstatic.a, liblz4.a) - SQLite rembed library (libsqlite_rembed.a) - -lscram -lusual for PostgreSQL SCRAM support - Add stub function SQLite3_Server_session_handler required by SQLite3_Server.cpp Resolves compilation errors for anomaly_detection-t test.	3 months ago
René Cannaò	af28598b23	Merge pull request #19 from ProxySQL/v3.1-MCP2_QR feat: Add MCP query rules and digest statistics [WIP]	3 months ago
Rene Cannao	f45506e0b6	fix: Missing <string> header in ai_llm_retry_scenarios-t - Add #include <string> for C++ std::string support Resolves compilation errors for ai_llm_retry_scenarios-t test.	3 months ago
Rene Cannao	d613816438	fix: Missing headers and format strings in vector_db_performance-t - Add #include <string> for C++ std::string support - Add #include <cmath> for sqrt() function - Change format %lld to %ld for chrono duration types (long int, not long long) Resolves compilation errors for vector_db_performance-t test.	3 months ago
Rene Cannao	b9a70f85a8	fix: Linking issues for anomaly_detection-t TAP test - Add special build rule in Makefile for anomaly_detection-t that includes: - $(OBJ) for global variables (GloVars, GloGATH) - -Wl,--allow-multiple-definition to allow test's main() to override ProxySQL's - ClickHouse client libraries (libclickhouse-cpp-lib.a, libzstdstatic.a, liblz4.a) - SQLite rembed library (libsqlite_rembed.a) - -lscram -lusual for PostgreSQL SCRAM support - Add stub function SQLite3_Server_session_handler required by SQLite3_Server.cpp Resolves compilation errors for anomaly_detection-t test.	3 months ago
René Cannaò	75a62f334d	Merge branch 'v4.0' into v3.1-vec Signed-off-by: René Cannaò <rene@proxysql.com>	3 months ago
Rene Cannao	8dc4246bdc	Introduce canonical proxy_sqlite3 symbol TU; update lib Makefile; remove MAIN_PROXY_SQLITE3 from main.cpp	3 months ago
Rene Cannao	7a7872f078	Organize RAG test files properly and update .gitignore - Move RAG test files to dedicated test/rag/ directory - Create proper Makefile in test/rag/ directory - Remove build_rag_test.sh script (replaced by Makefile) - Update .gitignore to exclude test/rag/test_rag_schema executable - Update documentation to reflect new test structure	3 months ago
Rene Cannao	acd05b60a9	Organize RAG test files properly - Move RAG test files to dedicated test/rag/ directory - Create proper Makefile in test/rag/ directory - Remove build_rag_test.sh script (replaced by Makefile) - Revert changes to main test/Makefile - Update documentation to reflect new test structure	3 months ago
Rene Cannao	3daaa5c592	feat: Implement RAG (Retrieval-Augmented Generation) subsystem Adds a complete RAG subsystem to ProxySQL with: - RAG_Tool_Handler implementing all MCP tools for retrieval operations - Database schema with FTS and vector support - FTS, vector, and hybrid search capabilities - Fetch and refetch tools for document/chunk retrieval - Admin tools for monitoring - Configuration variables for RAG parameters - Comprehensive documentation and test scripts Implements v0 deliverables from RAG blueprint: - SQLite schema initialization - Source registry management - MCP tools: search_fts, search_vector, search_hybrid, get_chunks, get_docs, fetch_from_source, admin.stats - Unit/integration tests and examples	3 months ago
Rene Cannao	3f25fb6c4e	Enhance anomaly detector unit tests with additional edge case coverage - Add edge case tests for SQL injection pattern detection: * Empty and NULL query handling * Very long query processing - Add edge case tests for query normalization: * Empty query handling * Unicode character support * Nested comment processing * Multiple line comment handling - Add edge case tests for risk scoring: * Negative match handling * Large match count capping * Boundary condition validation - Fix test assertions for more robust validation - All 39 tests now pass successfully These improvements address potential feedback from code reviewers by providing more comprehensive test coverage for edge cases and boundary conditions.	3 months ago
Rene Cannao	2888ee3f45	Fix gemini-code-assist recommendations and implement comprehensive anomaly detection tests - Fix retry logic to use is_retryable_error function for proper HTTP error handling - Add exception handling to get_json_int function with try-catch around std::stoi - Improve validate_numeric_range to use strtol instead of atoi for better error reporting - Fix Chinese characters in documentation (non-zero -> non-zero) - Replace placeholder tests with actual comprehensive tests for anomaly detection functionality - Create new standalone unit test anomaly_detector_unit-t.cpp with 29 tests covering: * SQL injection pattern detection (12 tests) * Query normalization (8 tests) * Risk scoring calculations (5 tests) * Configuration validation (4 tests) - All tests pass successfully, providing meaningful validation of core anomaly detection logic Thanks to gemini-code-assist for the thorough code review and recommendations.	3 months ago
Rene Cannao	ae4200dbc0	Enhance AI features with improved validation, memory safety, error handling, and performance monitoring - Rename validate_provider_name to validate_provider_format for clarity - Add null checks and error handling for all strdup() operations - Enhance error messages with more context and HTTP status codes - Implement performance monitoring with timing metrics for LLM calls and cache operations - Add comprehensive test coverage for edge cases, retry scenarios, and performance - Extend status variables to track performance metrics - Update MySQL session to report timing information to AI manager	3 months ago
Rene Cannao	3032dffed4	test: Add NL2SQL internal functionality unit tests Add comprehensive TAP unit tests for NL2SQL internal functions: - Error code conversion (5 tests): Validate nl2sql_error_code_to_string() covers all 13 defined error codes plus UNKNOWN_ERROR - SQL validation patterns (17 tests): Test validate_and_score_sql() * Valid SELECT queries (4 tests) * Non-SELECT queries (4 tests) * Injection pattern detection (4 tests) * Edge cases (4 tests): empty, lone keyword, semicolons, complex queries - Request ID generation (12 tests): Test UUID-like ID generation * Format validation (20 assertions for 10 IDs) * Uniqueness (100 IDs checked for duplicates) * Hexadecimal character validation - Prompt building (8 tests): Test build_prompt() * Basic prompt structure (3 tests) * Schema context inclusion (3 tests) * Section ordering (1 test) * Special character handling (2 tests) Note: Tests are self-contained with standalone implementations matching the logic in NL2SQL_Converter.cpp.	3 months ago
Rene Cannao	49092e9c8d	test: Add unit tests for AI configuration validation This commit adds comprehensive unit tests for the AI configuration validation functions used in AI_Features_Manager. Changes: - Add test/tap/tests/ai_validation-t.cpp with 61 unit tests - Test URL format validation (validate_url_format) - Test API key format validation (validate_api_key_format) - Test numeric range validation (validate_numeric_range) - Test provider name validation (validate_provider_name) - Test edge cases and boundary conditions The test file is self-contained with its own copies of the validation functions to avoid complex linking dependencies on libproxysql. Test Categories: - URL validation: 15 tests (http://, https:// protocols) - API key validation: 14 tests (OpenAI, Anthropic formats) - Numeric range: 13 tests (min/max boundaries) - Provider name: 8 tests (openai, anthropic) - Edge cases: 11 tests (NULL handling, long values) All 61 tests pass successfully. Part of: Phase 4 of NL2SQL improvement plan	3 months ago
Rene Cannao	1a8b406d9b	fix: Correct SQL query for AI variables in vector features test - Change from runtime_mysql_servers with variable_name column - To mysql_servers with ai_* prefix columns - This matches the actual schema where AI variables are stored	3 months ago
Rene Cannao	4b0cb9d95a	test: Add vector features unit test Add unit test for vector features including: - Virtual vec0 table creation verification - NL2SQL vector cache configuration tests - Anomaly embedding configuration tests - Vector database file verification - Status variables validation - Cache statistics interface tests - GenAI module availability checks 20 tests covering configuration and infrastructure validation. Tests can be extended with actual embedding generation once llama-server is running in the test environment.	3 months ago
Rene Cannao	0be9715188	test: Add comprehensive tests and documentation for Anomaly Detection Added 95 tests (50 unit + 45 integration) and 4 documentation files: Test Files: - test/tap/tests/anomaly_detection-t.cpp (50 unit tests) * Initialization and configuration tests * SQL injection pattern detection * Query normalization * Rate limiting * Statistical anomaly detection * Integration scenarios * Configuration management * False positive handling - test/tap/tests/anomaly_detection_integration-t.cpp (45 integration tests) * Real SQL injection pattern detection with actual queries * Legitimate query passthrough verification * Multi-user rate limiting scenarios * Statistical anomaly detection * Log-only mode configuration Documentation (doc/ANOMALY_DETECTION/): - README.md: User guide with quick start, configuration, examples - API.md: Complete API reference for Anomaly_Detector class - ARCHITECTURE.md: System architecture and design documentation - TESTING.md: Testing guide with test categories and examples All tests compile successfully and follow the TAP framework pattern used throughout ProxySQL.	3 months ago
Rene Cannao	eccb2bfe4d	test: Add integration tests for NL2SQL - nl2sql_integration-t.cpp: Schema-aware conversion, multi-table queries - Tests JOIN queries, aggregations, complex patterns - Tests error recovery and cross-schema queries - 30 tests across 6 categories Tests require running ProxySQL instance with admin interface to create test schema and validate SQL generation.	3 months ago
Rene Cannao	6d2b0ab303	test: Fix vector keyword conflict in NL2SQL unit tests Add 'using std::vector;' declaration to resolve conflicts with macros defined in included headers.	3 months ago
Rene Cannao	a61f709c7b	test: Add comprehensive TAP unit tests for NL2SQL - nl2sql_unit_base-t.cpp: Initialization, configuration, persistence, error handling - nl2sql_prompt_builder-t.cpp: Prompt construction, schema context, edge cases - nl2sql_model_selection-t.cpp: Model routing logic, latency handling, fallback Tests follow ProxySQL TAP framework patterns and use CommandLine helper for environment-based configuration.	3 months ago
René Cannaò	2637d28f36	Merge pull request #5299 from sysown/v3.0_pg-cancel-terminate-backend-param-support_5298 Add parameterized PID support for pg_cancel_backend/pg_terminate_backend in extended query protocol	3 months ago
René Cannaò	3bcb9e0bd5	Merge pull request #5301 from sysown/v3.0_fix-pgsql-threshold-deadlock_5300 Fix: Close Statement flood exceeds threshold before backend operation deadlock	3 months ago
Rahim Kanji	c2337d22b7	Added regression test	3 months ago
Rahim Kanji	ce42c188f5	Improvements	3 months ago
Rahim Kanji	a892d9a05b	Add pgsql-parameterized_kill_queries_test-t test to groups.json	3 months ago
Rahim Kanji	22afe6cb63	Add test	3 months ago
Rahim Kanji	b9175f8481	Fixed reg_test_5233_set_warning-t	3 months ago
Miro Stauder	b91dac044a	groups.json: add missing test Signed-off-by: Miro Stauder <miro@proxysql.com>	3 months ago
René Cannaò	313f637cf0	Merge branch 'v3.1-vec' into v3.1-MCP1 Signed-off-by: René Cannaò <rene.cannao@gmail.com>	3 months ago
René Cannaò	a8e759c970	Merge pull request #5289 from sysown/v3.0_buildTap Improve build performance by preserving test dependencies during 'make clean'	3 months ago
René Cannaò	8b2b29918a	Merge pull request #5291 from sysown/v3.0-misc260111 [skip-ci] Remove deprecated read_only_action implementations from MySQL and PgSQL HostGroups managers	3 months ago
René Cannaò	b1e37b3387	Merge branch 'v3.0' into v3.0_pgsql-use-ssl-issue-5284 Signed-off-by: René Cannaò <rene@proxysql.com>	3 months ago
René Cannaò	c6ed5b96cd	Merge pull request #5282 from sysown/v3.0_bind-format-issue-5273-fix Fix malformed Bind packets when client provides a single parameter format	3 months ago
Rene Cannao	40cff23c3b	Initialize MySQL Tool Handler and fix default MySQL port - Initialize MySQL_Tool_Handler in ProxySQL_MCP_Server constructor with MySQL configuration from MCP variables - Use GloVars.get_SSL_pem_mem() to get SSL certificates correctly - Add MySQL_Tool_Handler cleanup in destructor - Change configure_mcp.sh default MySQL port from 3307 to 3306 - Change configure_mcp.sh default password from test123 to empty - Update help text and examples to match new defaults	3 months ago
Rene Cannao	33a100c1db	Use relative path mcp_catalog.db in MCP test instead of absolute /var/lib/proxysql path	3 months ago
Rene Cannao	5a85ef04f6	Fix MCP variables persistence and add DISK command support This commit fixes several issues with MCP (Model Context Protocol) variables not being properly persisted across storage layers and adds support for DISK commands. Changes: 1. lib/Admin_FlushVariables.cpp: - Fixed flush_mcp_variables___runtime_to_database() to properly insert variables into runtime_global_variables using db->execute() with formatted strings (matching admin pattern) - Fixed SQL format string to avoid double-prefix bug (qualified_name already contains "mcp-" prefix) - Fixed lock ordering by releasing outer wrlock before calling runtime_to_database with use_lock=true, then re-acquiring - Removed explicit BEGIN/COMMIT transactions to match admin pattern 2. lib/Admin_Handler.cpp: - Added MCP DISK command handlers that rewrite commands to SQL queries: * LOAD MCP VARIABLES FROM DISK -> INSERT OR REPLACE INTO main.global_variables * SAVE MCP VARIABLES TO DISK -> INSERT OR REPLACE INTO disk.global_variables * SAVE MCP VARIABLES FROM MEMORY/MEM -> INSERT OR REPLACE INTO disk.global_variables - Separated DISK command handlers from MEMORY/RUNTIME handlers 3. lib/ProxySQL_Admin.cpp: - Added flush_mcp_variables___runtime_to_database() call to stats section to ensure MCP variables are repopulated when runtime_global_variables is cleared and refreshed 4. tests/mcp_module-t.cpp: - Added verbose diagnostic output throughout tests - Added section headers and test numbers for clarity - Added variable value logging and error logging All 52 MCP module tests now pass.	3 months ago
Rene Cannao	4aedacd83b	[skip-ci] Remove deprecated read_only_action implementations from MySQL and PgSQL HostGroups managers The old read_only_action() implementations were marked for deletion after 2025-07-14. These were replaced with new implementation that doesn't depend on the admin table. This change removes the deprecated code paths to clean up the codebase.	3 months ago
Rene Cannao	f2ac54e677	Optimize find commands in test/tap/tap/Makefile for better performance Combine multiple find commands into a single command to reduce directory traversal from 4 passes to 1 pass, improving both performance and readability. Suggestion from gemini-code-assist review.	3 months ago
Rene Cannao	221ff23991	Add MySQL exploration MCP tools with SQLite catalog Implemented MCP (Model Context Protocol) server providing tools for LLM-based MySQL database exploration: - MySQL_Catalog: SQLite-based catalog for LLM external memory with upsert, get, search, list, merge, delete operations and FTS support - MySQL_Tool_Handler: 17+ database exploration tools with guardrails: * Inventory: list_schemas, list_tables * Structure: describe_table, get_constraints, describe_view * Profiling: table_profile, column_profile * Sampling: sample_rows (max 20), sample_distinct (max 50) * Query: run_sql_readonly (max 200 rows, 2s timeout, SELECT-only) * Relationship: suggest_joins, find_reference_candidates * Catalog: catalog_upsert, catalog_get, catalog_search, catalog_list, catalog_merge, catalog_delete - MCP Module Integration: * Added 6 new configuration variables for MySQL tool handler (mysql_hosts, mysql_ports, mysql_user, mysql_password, mysql_schema, catalog_path) * Added MySQL_Tool_Handler pointer to MCP_Threads_Handler * Implemented tool routing in MCP endpoint for tools/list, tools/describe, and tools/call methods - TAP Tests: Updated to expect 14 MCP variables (was 8) Files: - include/MySQL_Catalog.h, lib/MySQL_Catalog.cpp - include/MySQL_Tool_Handler.h, lib/MySQL_Tool_Handler.cpp - include/MCP_Thread.h, lib/MCP_Thread.cpp - include/MCP_Endpoint.h, lib/MCP_Endpoint.cpp - lib/Makefile, test/tap/tests/mcp_module-t.cpp	3 months ago
Rene Cannao	245e61ee86	Make MCP_Threads_Handler a standalone independent class Remove unnecessary inheritance from MySQL_Threads_Handler. The MCP module should be independent and not depend on MySQL/PostgreSQL thread handlers. Changes: - MCP_Threads_Handler now manages its own pthread_rwlock_t for synchronization - Simplified init() signature (removed unused num/stack parameters) - Added ProxySQL_Main_init_MCP_module() call in main initialization phase - Include only standard C++ headers (pthread.h, cstring, cstdlib)	3 months ago
Rene Cannao	de3fd05a5a	Reverted change to test/tap/tests/.env	3 months ago
Rene Cannao	87fff9e046	Add MCP (Model Context Protocol) module skeleton Add new MCP module supporting multiple MCP server endpoints over HTTPS with JSON-RPC 2.0 protocol skeleton. Each endpoint (/mcp/config, /mcp/observe, /mcp/query, /mcp/admin, /mcp/cache) is a distinct MCP server with its own authentication configuration. Features: - HTTPS server using existing ProxySQL TLS certificates - JSON-RPC 2.0 skeleton implementation (actual protocol TBD) - 5 MCP endpoints with per-endpoint auth configuration - LOAD/SAVE MCP VARIABLES admin commands - Configuration file support (mcp_variables section) Implementation follows GenAI module pattern: - MCP_Threads_Handler: Main module handler with variable management - ProxySQL_MCP_Server: HTTPS server wrapper using libhttpserver - MCP_JSONRPC_Resource: Base endpoint class with JSON-RPC skeleton	3 months ago
Rene Cannao	4be31efd91	Fix cpp-dotenv preservation in clean target and add proper cleanall support - Fix: Exclude cpp-dotenv directories from find commands in 'make clean' (was removing build artifacts even when preserving source directories) - Add: proper 'cleanall' target in test/tap/tap/Makefile that removes cpp-dotenv source - Update: test/tap/Makefile cleanall to call tap's cleanall instead of clean - Now cpp-dotenv directories are properly preserved during development This fixes a bug where: 1. cpp-dotenv build artifacts were still being deleted during 'make clean' 2. There was no way to properly clean cpp-dotenv except manually 3. The preservation optimization was ineffective due to the above Behavior now: - 'make clean': preserves cpp-dotenv source and build artifacts (~213MB saved) - 'make cleanall': removes everything including cpp-dotenv source	3 months ago
Rene Cannao	ca224cfbdd	Preserve cpp-dotenv source during 'make clean' to improve build performance - Remove cpp-dotenv source cleanup from 'make clean' in test/tap/tap/Makefile - This prevents accidental removal of 213MB of extracted cpp-dotenv source - Build targets force full rebuild anyway if libraries are missing - Typical 'make clean' operation is now faster and preserves compiled artifacts Similar optimization as test/deps: - Regular development workflow preserves expensive-to-rebuild dependencies - Only 'cleanall' removes everything including source directories - Reduces repeated downloads and extraction of cpp-dotenv sources This change complements the previous test/deps optimization, further improving development workflow performance.	3 months ago

1 2 3 4 5 ...

1598 Commits (950f163bfb15c080736fd67aa67a8977cd8e9ff8)