proxysql

Commit Graph

Author	SHA1	Message	Date
Rene Cannao	a236a13787	tsdb: dedicated module for TSDB variables with tsdb- prefix and optimized background tasks	4 months ago
Rene Cannao	da877c378b	tsdb: align admin variable lifecycle, full prometheus ingestion, docs and tap coverage	4 months ago
Rene Cannao	cc53ddca23	Merge branch 'v3.0-openssl-fix' into v3.0-test0213	4 months ago
Rene Cannao	3e37337877	Use RAII for sqlite3 statements across codebase - Convert prepare_v2() calls to use unique_ptr RAII pattern - Remove manual sqlite3_finalize calls (now handled by RAII) - Update flush_*_variables functions to use new prepare_v2 return pattern - Apply changes to Admin, Cluster, Monitor, Catalog, and other modules	4 months ago
René Cannaò	3145473438	Merge pull request #5367 from sysown/v3.0_pgsql-meta-cmd-admin-5365 Add PostgreSQL psql meta-command support to admin interface	4 months ago
Rahim Kanji	9460520999	Add more tests	4 months ago
Rahim Kanji	ba5afd0bbf	Add TAP test	4 months ago
Rene Cannao	0e605f4170	build: fix anomaly_detection-t linking for PROXYSQLGENAI When PROXYSQLGENAI is enabled, anomaly_detection-t should link with libsqlite_rembed.a instead of -lscram -lusual due to different SQLite embedding requirements.	4 months ago
René Cannaò	a360dc22ae	Merge pull request #5308 from sysown/v3.0_mac Support for ProxySQL compilation on macOS (Darwin/Apple Silicon)	4 months ago
René Cannaò	41088f9128	Merge pull request #5319 from sysown/v3.0.6-fix_reg_test_5233_set_warning-t V3.0.6 fix reg test 5233 set warning t	4 months ago
René Cannaò	ce9fdc9d7a	Merge branch 'v3.0' into v3.0.6-implement_FLUSH_STATS Signed-off-by: René Cannaò <rene@proxysql.com>	4 months ago
Rene Cannao	2b0c741892	Add TAP test for FLUSH STATS commands Add tests for PROXYSQL FLUSH STATS, PROXYSQL FLUSH MYSQL STATS, and PROXYSQL FLUSH PGSQL STATS admin commands.	4 months ago
Rene Cannao	a17593b99f	Add rand5_admin_1 test group environment configuration	4 months ago
Rene Cannao	70c4049012	Add test group environment configurations	4 months ago
Rene Cannao	dd9b761ad8	Add GenAI test files to test groups configuration	4 months ago
Rene Cannao	a57b43286b	Fix TAP test builds with PROXYSQLGENAI Add sqlite-vec and sqlite-rembed libraries to TAP test linking when PROXYSQLGENAI=1 is set. The proxy_sqlite3_symbols.cpp in libproxysql.a references sqlite3_vec_init which requires these libraries. Changes to test/tap/tests/Makefile: - Add libsqlite_rembed.a and vec.o to STATIC_LIBS when PROXYSQLGENAI=1 - Add GenAI libraries to sqlite3-t target - Add GenAI libraries to setparser_test, setparser_test2, setparser_test3 targets - Fix typo: -EXCLUDE_TRACKING_VARIABLES -> -DEXCLUDE_TRACKING_VARIABLES	4 months ago
Rene Cannao	48bc7dd7bf	Merge v4.0 GenAI features into v3.0 with conditional compilation This commit merges the experimental v4.0 GenAI/MCP features into the stable v3.0 branch using conditional compilation. All v4.0 features are disabled by default and only enabled when PROXYSQLGENAI=1 is set at compile time. Changes: Build System: - Modified main Makefile to pass PROXYSQLGENAI flag to sub-makefiles - Modified deps/Makefile to conditionally build sqlite-vec and sqlite-rembed - Modified lib/Makefile to add PSQLGA flag and include GenAI object files - Modified src/Makefile to add PSQLGA flag and conditional linking Headers (wrapped with #ifdef PROXYSQLGENAI): - All 20 new GenAI header files in include/ - Modified cpp.h, proxysql_glovars.hpp, proxysql_admin.h - Modified ProxySQL_Admin_Tables_Definitions.h for GenAI/MCP tables Source Files: - All 22 new GenAI source files in lib/ wrapped with #ifdef PROXYSQLGENAI - Modified src/main.cpp for conditional global variables and init/shutdown - Modified Admin_Handler.cpp for conditional command handlers - Modified Admin_Bootstrap.cpp for conditional table registration - Modified Admin_FlushVariables.cpp for conditional variable flushing - Modified ProxySQL_Admin.cpp for conditional admin methods - Modified ProxySQL_Admin_Stats.cpp for conditional MCP stats functions - Modified proxy_sqlite3_symbols.cpp to always compile (needed by core) - Modified MySQL_Session.cpp for conditional GenAI function calls Test Files: - Renamed test_mcp_query_rules-t to test_mcp_query_rules-t.sh - Renamed test_mcp_rag_metrics-t to test_mcp_rag_metrics-t.sh - Modified anomaly_detection-t.cpp for conditional test execution Usage: # Build without GenAI (v3.0 mode - default) make clean && make build_deps -j$(nproc) && make build_lib -j$(nproc) && make build_src -j$(nproc) # Build with GenAI (v4.0 mode) make clean && PROXYSQLGENAI=1 make build_deps -j$(nproc) && PROXYSQLGENAI=1 make build_lib -j$(nproc) && PROXYSQLGENAI=1 make build_src -j$(nproc)	4 months ago
Javier Jaramago Fernández	c5db31bf9d	test: Improve testing for MCP and RAG tools	4 months ago
Javier Jaramago Fernández	4d4bb9dec5	fix: Improved MCP rule testing and helpers	4 months ago
Miro Stauder	3d3dc1732d	fix test logic and stats reset	5 months ago
Javier Jaramago Fernández	51e91378c4	test: Add/Improve TAP tests for MCP query_rules/digests	5 months ago
Rene Cannao	d7b2fea898	fix: Remove MAIN_PROXY_SQLITE3 defines from tests (v3.1-MCP2 compatibility) v3.1-MCP2 introduced lib/proxy_sqlite3_symbols.cpp which provides canonical definitions for proxy_sqlite3_* function pointers. Tests that link against libproxysql.a must NOT define MAIN_PROXY_SQLITE3, otherwise sqlite3db.h will provide conflicting definitions. Removed #define MAIN_PROXY_SQLITE3 from: - test/tap/tests/sqlite3-t.cpp - test/tap/tests/unit_test.h - test/tap/tests/setparser_test.cpp - test/tap/tests/setparser_test_common.h	5 months ago
Rene Cannao	b4f521c634	Merge v3.1-MCP2 into v3.1-vec Conflict resolution summary: Features preserved from v3.1-vec: - mcp_use_ssl variable (HTTP/HTTPS mode support) - RAG tool handler support - FTS (Full Text Search) functionality in MySQL_Tool_Handler Changes accepted from v3.1-MCP2: - Schema isolation for catalog (removed mcp_catalog_path/fts_path variables) - Schema parameter added to catalog functions - Comprehensive query rewriting improvements in Query_Tool_Handler - Format string fixes in vector_db_performance-t.cpp Key decisions: - catalog_path/fts_path: Removed per v3.1-MCP2 commit `35b0b224` (intentional) - MySQL_Catalog.cpp: Accepted v3.1-MCP2 version (schema isolation improvements) - Query_Tool_Handler.cpp: Accepted v3.1-MCP2 version (significant improvements) - Makefile: Kept v3.1-vec's EXCLUDE_TRACKING_VARIABLES (correct spelling) Build notes: - MySQL_Tool_Handler now requires schema parameter for catalog operations - Catalog path is hardcoded to datadir/mcp_catalog.db - FTS functionality preserved but may need updates for schema isolation	5 months ago
Rene Cannao	02918d18b8	Fix PR #25 Review: All AI code reviewer feedback addressed This commit addresses all recommendations from CodeRabbit, Gemini Code Assist, and Copilot for PR #25 (FTS security and code quality improvements). Critical Security Fixes: - MCP_Thread.cpp: Rollback fts_path on reset failure to keep config consistent - MySQL_FTS.cpp: Add escape_mysql_identifier() for MySQL query identifier escaping - MySQL_FTS.cpp: Add unique hash-based fallback to sanitize_name() for empty strings - MySQL_FTS.cpp: Add where_clause validation to block dangerous SQL patterns Memory Safety Fixes: - MySQL_FTS.cpp: Fix indexes_result memory leak on early return in search() - MySQL_FTS.h: Delete copy/move operations to prevent accidental resource duplication Thread Safety Documentation: - MySQL_Tool_Handler.cpp: Add comment explaining FTS lock design rationale Test Script Improvements: - test_mcp_fts.sh: Add curl timeouts (5s connect, 30s max) - test_mcp_fts.sh: Remove unused delete_response variable - test_mcp_fts_detailed.sh: Make cleanup tolerant of non-existent indexes Build Fixes: - Makefile: Fix EXCLUDE_TRACKING_VARAIABLES typo to EXCLUDE_TRACKING_VARIABLES - vector_db_performance-t.cpp: Fix printf format specifiers to %lld with cast Schema Fixes: - Query_Tool_Handler.cpp: Change fts_index_table columns schema from string to array Code Cleanup: - MySQL_Tool_Handler.cpp: Remove all remaining debug fprintf statements (34 lines) - Documentation: Change "Full Text" to "Full-Text" (hyphenated) Total: ~50 fixes across 10 files	5 months ago
Rene Cannao	fd5d433a2b	fix: Missing <string> header in ai_llm_retry_scenarios-t - Add #include <string> for C++ std::string support Resolves compilation errors for ai_llm_retry_scenarios-t test.	5 months ago
Rene Cannao	2a614f817e	fix: Missing headers and format strings in vector_db_performance-t - Add #include <string> for C++ std::string support - Add #include <cmath> for sqrt() function - Change format %lld to %ld for chrono duration types (long int, not long long) Resolves compilation errors for vector_db_performance-t test.	5 months ago
Rene Cannao	0dc3531744	fix: Linking issues for anomaly_detection-t TAP test - Add special build rule in Makefile for anomaly_detection-t that includes: - $(OBJ) for global variables (GloVars, GloGATH) - -Wl,--allow-multiple-definition to allow test's main() to override ProxySQL's - ClickHouse client libraries (libclickhouse-cpp-lib.a, libzstdstatic.a, liblz4.a) - SQLite rembed library (libsqlite_rembed.a) - -lscram -lusual for PostgreSQL SCRAM support - Add stub function SQLite3_Server_session_handler required by SQLite3_Server.cpp Resolves compilation errors for anomaly_detection-t test.	5 months ago
René Cannaò	af28598b23	Merge pull request #19 from ProxySQL/v3.1-MCP2_QR feat: Add MCP query rules and digest statistics [WIP]	5 months ago
Rene Cannao	f45506e0b6	fix: Missing <string> header in ai_llm_retry_scenarios-t - Add #include <string> for C++ std::string support Resolves compilation errors for ai_llm_retry_scenarios-t test.	5 months ago
Rene Cannao	d613816438	fix: Missing headers and format strings in vector_db_performance-t - Add #include <string> for C++ std::string support - Add #include <cmath> for sqrt() function - Change format %lld to %ld for chrono duration types (long int, not long long) Resolves compilation errors for vector_db_performance-t test.	5 months ago
Rene Cannao	b9a70f85a8	fix: Linking issues for anomaly_detection-t TAP test - Add special build rule in Makefile for anomaly_detection-t that includes: - $(OBJ) for global variables (GloVars, GloGATH) - -Wl,--allow-multiple-definition to allow test's main() to override ProxySQL's - ClickHouse client libraries (libclickhouse-cpp-lib.a, libzstdstatic.a, liblz4.a) - SQLite rembed library (libsqlite_rembed.a) - -lscram -lusual for PostgreSQL SCRAM support - Add stub function SQLite3_Server_session_handler required by SQLite3_Server.cpp Resolves compilation errors for anomaly_detection-t test.	5 months ago
René Cannaò	75a62f334d	Merge branch 'v4.0' into v3.1-vec Signed-off-by: René Cannaò <rene@proxysql.com>	5 months ago
René Cannaò	bd16436ea3	Fix MySQL 8.4 build on macOS with C++20 compatibility patch - Add mysql-8.4.0-clang17.patch to fix AppleClang 17 template strictness errors - Update test/deps/Makefile to automatically apply patch on Darwin - Enforce sequential build (-j1) for MySQL connectors on macOS to prevent race conditions - Patch fixes 9 template keyword errors in MySQL serialization headers	5 months ago
Rene Cannao	8dc4246bdc	Introduce canonical proxy_sqlite3 symbol TU; update lib Makefile; remove MAIN_PROXY_SQLITE3 from main.cpp	5 months ago
Rene Cannao	7a7872f078	Organize RAG test files properly and update .gitignore - Move RAG test files to dedicated test/rag/ directory - Create proper Makefile in test/rag/ directory - Remove build_rag_test.sh script (replaced by Makefile) - Update .gitignore to exclude test/rag/test_rag_schema executable - Update documentation to reflect new test structure	5 months ago
Rene Cannao	acd05b60a9	Organize RAG test files properly - Move RAG test files to dedicated test/rag/ directory - Create proper Makefile in test/rag/ directory - Remove build_rag_test.sh script (replaced by Makefile) - Revert changes to main test/Makefile - Update documentation to reflect new test structure	5 months ago
Rene Cannao	3daaa5c592	feat: Implement RAG (Retrieval-Augmented Generation) subsystem Adds a complete RAG subsystem to ProxySQL with: - RAG_Tool_Handler implementing all MCP tools for retrieval operations - Database schema with FTS and vector support - FTS, vector, and hybrid search capabilities - Fetch and refetch tools for document/chunk retrieval - Admin tools for monitoring - Configuration variables for RAG parameters - Comprehensive documentation and test scripts Implements v0 deliverables from RAG blueprint: - SQLite schema initialization - Source registry management - MCP tools: search_fts, search_vector, search_hybrid, get_chunks, get_docs, fetch_from_source, admin.stats - Unit/integration tests and examples	5 months ago
Rene Cannao	3f25fb6c4e	Enhance anomaly detector unit tests with additional edge case coverage - Add edge case tests for SQL injection pattern detection: * Empty and NULL query handling * Very long query processing - Add edge case tests for query normalization: * Empty query handling * Unicode character support * Nested comment processing * Multiple line comment handling - Add edge case tests for risk scoring: * Negative match handling * Large match count capping * Boundary condition validation - Fix test assertions for more robust validation - All 39 tests now pass successfully These improvements address potential feedback from code reviewers by providing more comprehensive test coverage for edge cases and boundary conditions.	5 months ago
Rene Cannao	2888ee3f45	Fix gemini-code-assist recommendations and implement comprehensive anomaly detection tests - Fix retry logic to use is_retryable_error function for proper HTTP error handling - Add exception handling to get_json_int function with try-catch around std::stoi - Improve validate_numeric_range to use strtol instead of atoi for better error reporting - Fix Chinese characters in documentation (non-zero -> non-zero) - Replace placeholder tests with actual comprehensive tests for anomaly detection functionality - Create new standalone unit test anomaly_detector_unit-t.cpp with 29 tests covering: * SQL injection pattern detection (12 tests) * Query normalization (8 tests) * Risk scoring calculations (5 tests) * Configuration validation (4 tests) - All tests pass successfully, providing meaningful validation of core anomaly detection logic Thanks to gemini-code-assist for the thorough code review and recommendations.	5 months ago
Rene Cannao	ae4200dbc0	Enhance AI features with improved validation, memory safety, error handling, and performance monitoring - Rename validate_provider_name to validate_provider_format for clarity - Add null checks and error handling for all strdup() operations - Enhance error messages with more context and HTTP status codes - Implement performance monitoring with timing metrics for LLM calls and cache operations - Add comprehensive test coverage for edge cases, retry scenarios, and performance - Extend status variables to track performance metrics - Update MySQL session to report timing information to AI manager	5 months ago
Rene Cannao	3032dffed4	test: Add NL2SQL internal functionality unit tests Add comprehensive TAP unit tests for NL2SQL internal functions: - Error code conversion (5 tests): Validate nl2sql_error_code_to_string() covers all 13 defined error codes plus UNKNOWN_ERROR - SQL validation patterns (17 tests): Test validate_and_score_sql() * Valid SELECT queries (4 tests) * Non-SELECT queries (4 tests) * Injection pattern detection (4 tests) * Edge cases (4 tests): empty, lone keyword, semicolons, complex queries - Request ID generation (12 tests): Test UUID-like ID generation * Format validation (20 assertions for 10 IDs) * Uniqueness (100 IDs checked for duplicates) * Hexadecimal character validation - Prompt building (8 tests): Test build_prompt() * Basic prompt structure (3 tests) * Schema context inclusion (3 tests) * Section ordering (1 test) * Special character handling (2 tests) Note: Tests are self-contained with standalone implementations matching the logic in NL2SQL_Converter.cpp.	5 months ago
Rene Cannao	49092e9c8d	test: Add unit tests for AI configuration validation This commit adds comprehensive unit tests for the AI configuration validation functions used in AI_Features_Manager. Changes: - Add test/tap/tests/ai_validation-t.cpp with 61 unit tests - Test URL format validation (validate_url_format) - Test API key format validation (validate_api_key_format) - Test numeric range validation (validate_numeric_range) - Test provider name validation (validate_provider_name) - Test edge cases and boundary conditions The test file is self-contained with its own copies of the validation functions to avoid complex linking dependencies on libproxysql. Test Categories: - URL validation: 15 tests (http://, https:// protocols) - API key validation: 14 tests (OpenAI, Anthropic formats) - Numeric range: 13 tests (min/max boundaries) - Provider name: 8 tests (openai, anthropic) - Edge cases: 11 tests (NULL handling, long values) All 61 tests pass successfully. Part of: Phase 4 of NL2SQL improvement plan	5 months ago
Rene Cannao	1a8b406d9b	fix: Correct SQL query for AI variables in vector features test - Change from runtime_mysql_servers with variable_name column - To mysql_servers with ai_* prefix columns - This matches the actual schema where AI variables are stored	5 months ago
Rene Cannao	4b0cb9d95a	test: Add vector features unit test Add unit test for vector features including: - Virtual vec0 table creation verification - NL2SQL vector cache configuration tests - Anomaly embedding configuration tests - Vector database file verification - Status variables validation - Cache statistics interface tests - GenAI module availability checks 20 tests covering configuration and infrastructure validation. Tests can be extended with actual embedding generation once llama-server is running in the test environment.	5 months ago
Rene Cannao	0be9715188	test: Add comprehensive tests and documentation for Anomaly Detection Added 95 tests (50 unit + 45 integration) and 4 documentation files: Test Files: - test/tap/tests/anomaly_detection-t.cpp (50 unit tests) * Initialization and configuration tests * SQL injection pattern detection * Query normalization * Rate limiting * Statistical anomaly detection * Integration scenarios * Configuration management * False positive handling - test/tap/tests/anomaly_detection_integration-t.cpp (45 integration tests) * Real SQL injection pattern detection with actual queries * Legitimate query passthrough verification * Multi-user rate limiting scenarios * Statistical anomaly detection * Log-only mode configuration Documentation (doc/ANOMALY_DETECTION/): - README.md: User guide with quick start, configuration, examples - API.md: Complete API reference for Anomaly_Detector class - ARCHITECTURE.md: System architecture and design documentation - TESTING.md: Testing guide with test categories and examples All tests compile successfully and follow the TAP framework pattern used throughout ProxySQL.	5 months ago
Rene Cannao	eccb2bfe4d	test: Add integration tests for NL2SQL - nl2sql_integration-t.cpp: Schema-aware conversion, multi-table queries - Tests JOIN queries, aggregations, complex patterns - Tests error recovery and cross-schema queries - 30 tests across 6 categories Tests require running ProxySQL instance with admin interface to create test schema and validate SQL generation.	5 months ago
Rene Cannao	6d2b0ab303	test: Fix vector keyword conflict in NL2SQL unit tests Add 'using std::vector;' declaration to resolve conflicts with macros defined in included headers.	5 months ago
Rene Cannao	a61f709c7b	test: Add comprehensive TAP unit tests for NL2SQL - nl2sql_unit_base-t.cpp: Initialization, configuration, persistence, error handling - nl2sql_prompt_builder-t.cpp: Prompt construction, schema context, edge cases - nl2sql_model_selection-t.cpp: Model routing logic, latency handling, fallback Tests follow ProxySQL TAP framework patterns and use CommandLine helper for environment-based configuration.	5 months ago
René Cannaò	2637d28f36	Merge pull request #5299 from sysown/v3.0_pg-cancel-terminate-backend-param-support_5298 Add parameterized PID support for pg_cancel_backend/pg_terminate_backend in extended query protocol	5 months ago
René Cannaò	3bcb9e0bd5	Merge pull request #5301 from sysown/v3.0_fix-pgsql-threshold-deadlock_5300 Fix: Close Statement flood exceeds threshold before backend operation deadlock	5 months ago

1 2 3 4 5 ...

1620 Commits (6404d7b3652a02cd4f24e1cf0a3c52817cf71cf3)