proxysql

Commit Graph

Author	SHA1	Message	Date
Rene Cannao	6b9918fdcc	chore: refine guard structures and fix indentation - Fixed indentation for TSDB methods in include/proxysql_admin.h. - Refined PROXYSQLTSDB guard structure in lib/Admin_FlushVariables.cpp. - Cleaned up indentation and redundant code in lib/ProxySQL_RESTAPI_Server.cpp. - Ensured consistent formatting across modified files.	1 month ago
Rene Cannao	17225e8e23	fix: resolve syntax error in ProxySQL_Statistics.cpp and verify build stability across tiers - Fixed a mismatched #endif and closing brace in ProxySQL_Statistics::disk_upgrade_mysql_connections. - Verified build stability for all three tiers (base, v3.1, v4.0) with clean builds.	1 month ago
Rene Cannao	bc524c7b27	chore: remove temporary snippet file	1 month ago
Rene Cannao	6b46b26931	chore: implement PR review feedback for release tier system - Makefile: normalized GIT_VERSION by stripping 'v' prefix before arithmetic. - Makefile: corrected .PHONY declaration for build_deps_debug_default. - Admin Handler: properly guarded TSDB commands to return error when disabled. - REST API: removed redundant nested guards inside tsdb_resource class. - Statistics: implemented rigorous input validation for set_variable. - Statistics: added return value checks and rollback logic for TSDB migrations. - Statistics: implemented bounded concurrency (batching) for backend probes. - FFTO: implemented cumulative buffer limit enforcement in MySQL and PgSQL. - FFTO: ensured initial packet capture in PgSQL_Session by early initialization. - FFTO: improved reassembly buffer compaction logic. - Headers: default-initialized ffto_bypassed to false. - Code cleanup: merged redundant PROXYSQLFFTO guards.	1 month ago
Rene Cannao	c2f678a9de	feat: implement release tiers and feature guards for v3.0, v3.1, and v4.0 This commit introduces a structured release tier system controlled by environment variables, allowing three distinct ProxySQL versions to be built from the same codebase while maintaining a clear upgrade path. Key Changes: - Tier 1 (v3.0.x Stable): Default build including bug fixes and core enhancements. - Tier 2 (v3.1.x Innovative): Enabled via PROXYSQL31=1. Adds FFTO and TSDB. - Tier 3 (v4.0.x AI/MCP): Enabled via PROXYSQLGENAI=1. Adds the AI stack. Implementation Details: - Makefile: Implemented dynamic versioning logic that increments the minor version for v3.1 and the major version for v4.0. Added tier documentation. - Feature Guards: Introduced PROXYSQLFFTO and PROXYSQLTSDB guards across ProxySQL_Admin, ProxySQL_Statistics, REST API, and HTTP server. - FFTO: Guarded Fast Forward Traffic Observer in MySQL and PgSQL sessions, including thread-local variables and configuration parameters. - TSDB: Guarded the entire Time Series Database subsystem, including SQLite table definitions, background loops, and REST API endpoints. - Build Fixes: * Fixed 'proxy_debug_func' linkage by adding extern "C" and moving the definition outside of DEBUG guards to ensure non-debug builds link correctly. * Corrected misplaced/nested DEBUG and PROXYSQLFFTO guards. * Verified build stability for all tiers (unset, PROXYSQL31=1, PROXYSQLGENAI=1).	1 month ago
René Cannaò	c60f390b44	Merge pull request #5422 from sysown/v3.0-pgsql_monitor_repl_lag feat: add 'replication_lag' support for PostgreSQL monitoring	1 month ago
Rene Cannao	ff1398ead5	fix: plumb real hostgroup_id for pgsql replication lag monitoring - Add hostgroup_id to mon_srv_t and ext_srvs.\n- Update fetch_updated_conf queries to return hostgroup_id.\n- Use real hostgroup_id in replication_lag_action call.	1 month ago
Rene Cannao	1911b02157	feat: temporarily disable pgsql monitor replication lag group_by_host	1 month ago
Rene Cannao	f45173b9c6	fix: make cmake3 discovery in test deps portable - Fallback to 'cmake' if 'cmake3' is not found.\n- Fixes build failures on systems where CMake 3.x is provided as 'cmake' (Ubuntu/Debian/Fedora).	1 month ago
René Cannaò	16f8837e0d	Merge branch 'v3.0' into v3.0-pgsql_monitor_repl_lag Signed-off-by: René Cannaò <rene@proxysql.com>	1 month ago
Rene Cannao	c4fc60bdb0	fix: address security and logic issues in PostgreSQL replication lag monitoring - Fix SQL injection in pt-heartbeat query concatenation.\n- Fix copy-paste errors between mysql_thread_ and pgsql_thread_ variables.\n- Restore monitor_replication_lag_group_by_host logic.	1 month ago
René Cannaò	d3af6c11bf	Merge pull request #5428 from sysown/v3.0-future_build_fixes fix: minor build fixes and improvements	1 month ago
Rene Cannao	131e69f94a	fix: portability and macOS compatibility for test dependencies - Use CMAKE3 variable for flexibility on Linux systems.\n- Add -Wno-incompatible-pointer-types for macOS to support modern Clang versions.	1 month ago
René Cannaò	5b721a6594	Merge pull request #5426 from sysown/tsdb-metric-update-fix [skip-ci] fix: TSDB metric collection & config load from disk	1 month ago
Javier Jaramago Fernández	2bfa23665b	fix: compilation of test dep 'mysql-connector-c 5.7' in newer CMake (> 4) Compilation for 'mysql-connector-c' for 5.7 is broken in newer CMake versions. There is no simple around this, since CMake 4 introduces multiple breaking changes and 'CMAKE_POLICY_VERSION_MINIMUM' isn't sufficient to fix this issues. Due to this, the simplest fix is to install in the testing environment CMake3. This fixed version of CMake should be used specifically for building this dependency. Arch Linux has a package that could be used as reference https://aur.archlinux.org/packages/cmake3.	1 month ago
Javier Jaramago Fernández	f88dddaefb	fix: set maxdepth for OpenSSL libraries search Previously OpenSSL library search didn't specify the depth of the search. This results in long waits during building searching in unnecessary system directories. The previous is unnecessary. When pkg-config is requested with 'libdir', the expectation is that the directory in which the binaries are located is returned. Due to this, the search can be contained only in that target directory.	1 month ago
Javier Jaramago Fernández	1c311fd5f1	fix: coredumper build in newer CMake versions	1 month ago
Javier Jaramago Fernández	26c6eab5ae	fix: improve error reporting for sqlite3 TAP utils	1 month ago
Javier Jaramago Fernández	5f347c15a6	chore: remove test cert files incorrectly pushed to TAP dir	1 month ago
Javier Jaramago Fernández	ff729a657e	chore: disable 'pt-heartbeat' in 'PgSQL_Threads_Handler::get_variable' This isn't for functional purposes, just for consistency in reporting the motivation for disabling the feature in the codebase.	1 month ago
Javier Jaramago Fernández	0132075d47	fix: simplified expression in 'test_pgsql_replication_lag' TAP scripts	1 month ago
Javier Jaramago Fernández	35254fad8c	fix: typo in 'pgsql-repl' infra script message	1 month ago
Javier Jaramago Fernández	36348fa797	fix: minor escaping issues with 'pgsql-repl' infra scripts	1 month ago
Wazir Ahmed	5421a382ab	fix: Load TSDB variables from disk during startup - TSDB variables stored on disk were not being loaded into runtime during startup. Add the missing `flush_tsdb_variables___database_to_runtime()` and invoke it during init. - Inline the existing load/save wrappers in the header as thin delegations to the underlying flush functions Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	1 month ago
Wazir Ahmed	1684d30725	fix: Refresh Prometheus metrics before TSDB sampling - `tsdb_sampler_loop()` calls `registry->Collect()` without refreshing module metrics, causing stale metric values to be stored in the TSDB. - Added `update_modules_metrics()` call before `Collect()` in the sampler. Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	1 month ago
René Cannaò	d683dd7b8b	Merge pull request #5410 from sysown/v3.0-unified-large-pr v3.0: unify ff_inspect + mcp-stats2 + 5384 + noise-testing	1 month ago
René Cannaò	148dfb2af3	Merge pull request #5423 from sysown/v3.0-genai_keys v3.0-genai_keys: GenAI live validations, LLM bridge tests, and portability fix	1 month ago
Rene Cannao	cffd980bb9	fix: Conditionally include sys/eventfd.h only on Linux Add #ifdef __linux__ guard around sys/eventfd.h include to fix compilation on non-Linux platforms.	1 month ago
Rene Cannao	bcefd093fc	test(genai): add live TAP validations for GenAI, LLM bridge, and MCP semantic lifecycle This commit introduces three new C++ TAP tests that validate ProxySQL's live GenAI and MCP behaviors using real provider credentials supplied via environment variables. The goal is to move beyond mock-style checks and verify actual runtime integration across request transport, tool execution, and semantic outputs. High-level scope - Add a live GenAI embed/rerank validation TAP test. - Add a live LLM bridge accuracy/error-path TAP test. - Add a live MCP semantic lifecycle TAP test that combines discovery, LLM-generated artifacts, upsert, and semantic search. - Register all three new tests in ai-g1 group mapping. Files added - test/tap/tests/genai_live_validation-t.cpp - test/tap/tests/llm_bridge_accuracy-t.cpp - test/tap/tests/mcp_semantic_lifecycle-t.cpp File updated - test/tap/groups/groups.json Detailed behavior by test 1) genai_live_validation-t.cpp - Reads required live env inputs: TAP_EMBED_URL, TAP_EMBED_TYPE, TAP_EMBED_MODEL, TAP_EMBED_DIMENSION, TAP_RERANK_URL, TAP_RERANK_MODEL - Skips (does not fail) when required environment is missing. - Configures runtime for test stability: - sets genai-vector_db_path to ./ai_features.db - enables genai-enabled - sets embed/rerank endpoints and embedding model - loads GENAI variables to runtime - Embedding integrity validation: - sends GENAI embed request with multiple documents - verifies row count matches input document count - verifies each returned embedding dimension matches TAP_EMBED_DIMENSION - Rerank semantic validation: - sends query with one intentionally relevant document and irrelevant distractors - checks highest score maps to the relevant document index - Stress validation: - opens 5 client connections - executes 20 total requests (4 per connection) concurrently - validates all requests succeed and no failures are reported - Includes high-verbosity diagnostics for SQL requests and parsed rows. 2) llm_bridge_accuracy-t.cpp - Reads required live env inputs: TAP_LLM_PROVIDER, TAP_LLM_URL, TAP_LLM_MODEL, TAP_LLM_KEY - Skips (does not fail) when required environment is missing. - Configures LLM bridge runtime: - sets genai-vector_db_path - enables genai-enabled and genai-llm_enabled - sets provider/url/model/key - loads GENAI variables to runtime - Special-character prompt handling: - issues LLM: prompt containing quotes, backslashes, JSON-like text, and emoji bytes - verifies request succeeds and response structure is valid - verifies returned provider column aligns with TAP_LLM_PROVIDER - Timeout/error-path validation: - reconfigures provider URL to an unroutable timeout-probe endpoint - sets genai-llm_timeout_ms=1000 (minimum valid bound in current code) - verifies client receives an error path with SQLSTATE HY000 and non-empty message - Captures and restores modified global variables at end of test. 3) mcp_semantic_lifecycle-t.cpp - Reads required live env inputs: TAP_LLM_PROVIDER, TAP_LLM_URL, TAP_LLM_MODEL, TAP_LLM_KEY - Skips (does not fail) when required environment is missing. - Configures LLM and MCP runtime for end-to-end lifecycle checks: - enables genai + llm bridge - configures MCP port/auth endpoint settings - creates MCP auth and target profiles - loads MCP variables/profiles to runtime - End-to-end lifecycle: - calls discovery.run_static and validates run_id - lists discovered table objects and selects object_ids - starts agent run via agent.run_start - generates summary text through LLM: bridge for two semantic markers - persists summaries via llm.summary_upsert for two objects - validates llm.search("customer") finds customer marker - validates llm.search("index") finds index marker - finishes run via agent.run_finish - Cleans up test MCP profiles and restores runtime variables. Groups registration - Added to ai-g1 in test/tap/groups/groups.json: - llm_bridge_accuracy-t - genai_live_validation-t - mcp_semantic_lifecycle-t Implementation notes and constraints reflected in tests - Tests are intentionally environment-gated and skip when live credentials are unavailable. - All tests include verbose diagnostics for outbound requests and parsed provider/tool responses. - Runtime variable mutations are restored best-effort to reduce suite side effects. - llm timeout validation uses 1000ms because current runtime validation enforces [1000..600000] for genai-llm_timeout_ms. Build verification performed - Compiled successfully (as jenkins user): - genai_live_validation-t - llm_bridge_accuracy-t - mcp_semantic_lifecycle-t This commit intentionally focuses on live integration correctness and transport behavior under real endpoints, while remaining TAP-friendly for CI environments that may not provide credentials (skip semantics instead of hard failures).	1 month ago
Javier Jaramago Fernández	a4ea3b2174	Merge branch 'v3.0' of github.com:sysown/proxysql into v3.0-pgsql_monitor_repl_lag	1 month ago
Javier Jaramago Fernández	4103d69979	feat: disable 'pt-heartbeat' support for PostgreSQL The feature has been disabled until 'pt-heartbeat' support for PostgreSQL is fixed. See https://perconadev.atlassian.net/browse/PT-2030.	1 month ago
Javier Jaramago Fernández	86590fc799	test: add 'test_pgsql_replication_lag-t.cpp' to new 'pgsql-repl' TAP group	1 month ago
Javier Jaramago Fernández	b7e98f170d	feat: add new TAP group infra for PostgreSQL replication testing	1 month ago
Javier Jaramago Fernández	b14f88226c	fix: compilation of 'cpp-dotenv' for newer CMake versions	1 month ago
Javier Jaramago Fernández	cbce30527f	feat-wip: Add initial version of TAP test for replication_lag This TAP test requires infra not currently present in the branch. The infra is yet to be adapted to work as TAP group.	1 month ago
Javier Jaramago Fernández	d18fd15a76	feat: Improve error report for TAP utility function 'CHECK_EXT_VAL'	1 month ago
Wazir Ahmed	d5ef84e535	TAP: Add `mcp_query_rules-t` in groups.json Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	1 month ago
René Cannaò	88b3c9dbd9	Merge pull request #5420 from sysown/mcp-config-fix fix: MCP config flush logic and runtime query rule representation	1 month ago
Wazir Ahmed	daa0b2037e	fix: Use consistent value representation in MCP query rules `mcp_query_rules` and `runtime_mcp_query_rules` uses different representations for the same fields, causing the web UI to always report config differences even after loading to runtime. - Use `-1` sentinel for nullable int fields (`flagOUT`, `timeout_ms`, `log`) so `NULL` values from main are preserved when written to runtime, following the `MySQL QP_rule_t` pattern. - Serialize `re_modifiers` bitmask back to `VARCHAR` (`1→"CASELESS"`) to match the format used in main table - Update `evaluate_mcp_query_rules()` to handle tri-state log field Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	1 month ago
Wazir Ahmed	c05bc66655	fix: MCP variables overwritten due to incorrect flush logic `flush_mcp_variables___runtime_to_database()` ignores the `replace` parameter. The prepared statement is hardcoded to `REPLACE INTO`, so even when callers pass `replace=false` (e.g. `GenericRefreshStatistics`), the function overwrites values in memory config with stale runtime values. Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	1 month ago
Rene Cannao	f29a326ca0	Fix QueryDigestTopK heap comparator semantics	1 month ago
Rene Cannao	2f82b3e5e6	test: Final cleanup of deprecated MCP shell scripts and GenAI diags This commit completes the transition of MCP and GenAI testing to a modernized architecture. Changes: - Removed ~4,300 lines of deprecated shell scripts in mcp_rules_testing/ and associated orchestrators (test_mcp_query_rules-t.sh). These tests are now fully covered by the C++ test mcp_query_rules-t.cpp. - Added final diagnostic hints to genai_async-t.cpp to explicitly guide users when backend AI services (llama-server) are missing or unreachable. - Cleaned up the working tree to ensure all functional logic is consolidated in robust, observable C++ tests.	1 month ago
Rene Cannao	c902d21219	build: Cleanup redundant test targets in test/tap/tests/Makefile Removed explicit listings of several tests from the 'tests' target dependency list, as they are already automatically discovered and compiled by the 'tests-cpp' target via the *-t.cpp wildcard rule. Removed: - test_tsdb_variables-t - test_tsdb_api-t - test_ffto_mysql-t - test_ffto_pgsql-t - test_ffto_bypass-t - mcp_query_rules-t These tests do not require custom linking or special build rules beyond the generic %-t pattern, making their explicit inclusion in the main tests list redundant.	1 month ago
Rene Cannao	a72d2bdd20	test: Replace deprecated shell MCP query rules tests with C++ TAP test This commit modernizes the MCP query rules validation by replacing a complex collection of 15+ shell scripts with a single, high-performance C++ TAP test. Changes: - Implemented mcp_query_rules-t.cpp: * Full CRUD validation for mcp_query_rules table. * Verification of LOAD MCP QUERY RULES TO RUNTIME command. * Runtime evaluation tests for Block, Rewrite, and OK_msg actions. * End-to-end verification of hits tracking in stats_mcp_query_rules. - Updated test/tap/tests/Makefile to build mcp_query_rules-t by default. - Removed deprecated test artifacts: * Deleted test_mcp_query_rules-t.sh and its environment files. * Deleted the entire collection of test_phase.sh scripts in mcp_rules_testing/ directory. Kept mcp_test_helpers.sh as it is still required by other MCP-related shell tests. - Improved diagnostic output and error reporting for better observability in CI environments.	1 month ago
Rene Cannao	6f72f30539	test: Fix and enhance MCP Phase-B LLM discovery test This commit ensures the MCP Phase-B test is robust and provides clear diagnostics for CI environments. Changes: - Implemented automatic MCP initialization via Admin interface: * Enabled MCP (mcp-enabled=true). * Disabled SSL (mcp-use_ssl=false) to simplify CI connectivity. * Registered a valid MCP target profile (mysql-127.0.0.1-13306). * Registered an authentication profile (default_mysql). * Properly loaded MCP variables and profiles into runtime. - Improved diagnostic logging: * Redirected all technical diags to stderr to avoid polluting TAP output and breaking jq parsing. * Added explicit 'Executing MCP Tool Call' messages for every step. - Enhanced robustness: * Switched to 'sysbench'/full harvest when default 'testdb' is empty. * Fixed JSON parsing logic to handle nested string content in MCP responses correctly using jq. * Updated plan to 14 to account for added setup and verification steps. * Fixed search verification by using a broad query to ensure newly created LLM artifacts are found in the FTS index.	1 month ago
Rene Cannao	f9a9f25349	test: Fix and enhance MCP Claude headless flow smoke test This commit resolves path resolution issues and correctly initializes the MCP environment for testing the headless discovery pipeline. Changes: - Corrected REPO_ROOT calculation to accurately locate script artifacts. - Implemented automatic MCP initialization via Admin interface: * Enabled MCP (mcp-enabled=true). * Disabled SSL (mcp-use_ssl=false) to avoid handshake issues in CI. * Registered a valid MCP target profile (mysql-127.0.0.1-13306). * Registered an authentication profile (default_mysql). * Properly loaded MCP variables and profiles into runtime. - Updated static_harvest.sh invocation to use the unencrypted HTTP endpoint. - Added extensive diagnostic logging to show exact command execution and intermediate results (Run IDs). - Verified the end-to-end dry-run orchestration of the discovery pipeline.	1 month ago
Rene Cannao	3b84b04616	test: Enhance genai_async-t with detailed diagnostics and robustness This commit improves the GenAI async architecture test by adding extensive logging, safety checks, and connection resilience. Changes: - Added a multi-line test description explaining the verified features. - Implemented a connection retry loop for the ProxySQL client interface to handle cases where the server is still initializing. - Added a safety step to set 'genai-vector_db_path' to a writable local path ('./ai_features.db') before enabling GenAI, preventing crashes due to permission errors on default system paths. - Explicitly enabled GenAI via 'genai-enabled=true' and verified initialization. - Significantly increased verbosity in execute_genai_query() and execute_genai_query_expect_error(): * Logs every GENAI: JSON command being executed. * Logs the result row count for successful queries. * Logs detailed error messages from ProxySQL when queries fail. - Added descriptive diag() messages to all test parts (1-11) to clarify the specific scenario being validated.	1 month ago
Rene Cannao	2bd89db0f3	test: Enhance nl2sql_unit_base-t with detailed diagnostics This commit improves the clarity and observability of the NL2SQL basic functionality test by adding extensive diagnostic information. Changes: - Added a detailed multi-line test description at startup. - Updated helper functions to log every SQL command (SELECT, UPDATE, LOAD) sent to the ProxySQL Admin interface. - Added diags to print the actual variable values read from the database, ensuring visibility into whether changes were applied. - Removed the 'SAVE GENAI VARIABLES TO DISK' command from the test. - Corrected the test plan count to 18 to match actual execution. - Improved all ok() messages to explicitly reference the ProxySQL global variables (genai-llm_*) being verified. - Ensured consistent usage of 'global_variables' and 'runtime_global_variables' tables for configuration checks.	1 month ago
Rene Cannao	766043ecc7	test: Enhance verbosity and diagnostics in nl2sql_model_selection-t This commit adds detailed diagnostic information to the NL2SQL model selection test to improve observability and failure diagnosis. Changes: - Added diag() messages in simulate_model_selection() to show input parameters (latency, preference, API keys) for each test case. - Updated get_nl2sql_variable() and set_nl2sql_variable() to log the actual SQL queries sent to the Admin interface and the values read back from the database. - Improved ok() test messages to explicitly state the expected ProxySQL global variable names (genai-llm_*) and the specific values being verified. - Fixed a minor formatting issue in comments.	1 month ago
Rene Cannao	c81b155ec2	fix: Update nl2sql_prompt_builder-t.cpp plan and add diagnostics The test was failing because it planned 30 tests but only executed 25. The plan has been corrected to 25. Changes: - Corrected test plan count from 30 to 25. - Added a detailed description of the test using diag() at the beginning. - Increased verbosity by adding diag() messages for each test case, printing the natural language query being processed and other relevant information. - Improved comments throughout the test file.	1 month ago

1 2 3 4 5 ...

8741 Commits (6b9918fdcc4088746f7291213f066396c14c187d) All Branches Search

8741 Commits (6b9918fdcc4088746f7291213f066396c14c187d)

All Branches