proxysql

Commit Graph

Author	SHA1	Message	Date
Rene Cannao	482ce386cd	chore(genai-carveout): strip dead PROXYSQLGENAI blocks + plugin hardening Two threads of cleanup that had to land together because they touch both core (where the dead code lives) and the plugin (where the new module-owned state lives): # Strip dead #ifdef PROXYSQLGENAI from lib/, include/, src/ (Spec Q9) Step 7 of the carve-out left PSQLGA empty in lib/Makefile, so every #ifdef PROXYSQLGENAI block in the core build became permanently false. The blocks remained as dead text (visible to readers, irrelevant to the binary). Strip ~600 lines spanning: src/main.cpp 2 sites lib/Admin_Bootstrap.cpp 3 sites (mcpdb open + attach + flush__variables hooks) lib/Admin_FlushVariables.cpp 2 sites (~250 lines of stub + #if 0 _ORIGINAL bodies) lib/Admin_Handler.cpp 5 sites (LOAD/SAVE GENAI VARIABLES dispatch + LOAD/SAVE MCP VARIABLES dispatch + MCP profiles handler) lib/ProxySQL_Admin.cpp ~10 sites (init__variables + load_mcp_server + load/save_mcp_query_rules + GenericRefreshStatistics MCP probes) lib/ProxySQL_Admin_Stats.cpp 3 stats___mcp_* function bodies include/proxysql_admin.h 7 method declarations include/ProxySQL_Admin_Tables_Definitions.h 4 unused DDL macros The plugin now owns LOAD/SAVE MCP VARIABLES, LOAD/SAVE GENAI VARIABLES, LOAD/SAVE MCP PROFILES, LOAD/SAVE MCP QUERY RULES via its admin command registry; the chassis dispatcher catches them before any of the dead core branches would have fired. Caveat — known break: the `stats_mcp_` tables are still registered (empty schema) but the populator path that fed them is gone. SELECTs return empty result sets until a follow-up wires the populator into the plugin (likely via register_runtime_view or a parallel stats-side ABI). Tracked separately. # Plugin hardening (review-feedback items 4-7) plugins/genai/Makefile: add -fvisibility=hidden, -fvisibility-inlines-hidden, -fstack-protector-strong (mirrors plugins/mysqlx). Tag proxysql_plugin_descriptor_v1 with __attribute__((visibility("default"))) so the loader's dlsym still resolves; without the explicit annotation the hidden-default flag would also hide the entry point. * plugins/genai/src/Anomaly_Detector.cpp: convert genai_anomaly_embed_fn to std::atomic<…> with relaxed loads/stores. The detector hot-path read races with init/stop on the lifecycle thread; atomics give us a portable no-torn-pointer guarantee without paying for acquire/release fences we don't need (no other state synchronises with the pointer). Replaces the hand-wavy "no mutex needed" comment with the actual reasoning. * plugins/genai/src/{plugin_main,plugin_commands,plugin_tables}.cpp: route the 17 fprintf(stderr, "genai plugin: ...") log lines through a new genai_log() helper that calls services->log_message when the chassis is wired up and falls back to stderr for early-init / unit-test paths. Lines now land in proxysql.log alongside the rest of the runtime. * plugins/genai/include/MCP_Thread.h: add get_variable_string(name, std::string&) returning a std::string, and switch mcp_save_variables_to_admindb (plugin_main.cpp) to it. The legacy get_variable(name, char) had an unbounded sprintf into a stack buffer and the SAVE path was passing char val[256] — a real overflow risk for _endpoint_auth bearer tokens. * plugins/genai/src/plugin_main.cpp: rewrite the genai_init / genai_stop comment block with the actual teardown ordering rationale (consumers before producers; atomic-clear-then-delete for the embed hook; why Prometheus counters stay registered).	2 months ago
Rene Cannao	04bccec51e	chore(plugin-chassis): tighten gating, drop dead paths, gate forgery setters Five clean-up items from the independent review of PR #5651. None of these change behaviour on a normal run; they each fix a concrete way the current code is misleading or unnecessarily exposed. 1) lib/ProxySQL_Admin.cpp: gate the three plugin-DB-handle getters (proxysql_plugin_get_admindb / _configdb / _statsdb) under #ifdef PROXYSQL40. Previously these were defined unconditionally and emitted symbols into v3.0/v3.1 binaries. The chassis is a v4.0 feature; the user explicitly required that v3.x builds carry no plugin-aware code. Wrap in PROXYSQL40 so they are entirely absent from v3.x linkage. ProxySQL_PluginManager.cpp's extern declarations (lines 23-25) are already inside the file-wide PROXYSQL40 gate and so resolve only when the gate is active. 2) lib/ProxySQL_PluginManager.cpp + include/ProxySQL_PluginManager.h: delete the dead `#else /* !PROXYSQL40 /` branches. Both files are wrapped in a top-level `#ifdef PROXYSQL40` covering the entire body. Inner `#ifdef PROXYSQL40 ... #else ... #endif` blocks therefore had unreachable `#else` arms — 30+ lines of "pre-chassis two-phase loader" in the .cpp, plus a redundant declaration of proxysql_load_configured_plugins in the .h. The dead arms read as load-bearing alternative implementations on review and that is exactly the wrong signal. Drop them; the single PROXYSQL40 path is the only one. 3) lib/Admin_Bootstrap.cpp + include/proxysql_admin.h + src/main.cpp: remove ProxySQL_Admin::materialize_plugin_tables(). `Admin::init()` already merges plugin-declared schemas into tables_defs_{admin,config,stats} (~line 944) and runs the DDL via check_and_build_standard_tables() (~line 994), all on the same first-boot/reload code path the core tables use. The follow-up call to GloAdmin->materialize_plugin_tables() in main.cpp ran a second name-dedup pass that found everything already present and produced an empty new-rows set — i.e. the post-init helper was a no-op disguised as load-bearing infrastructure. Delete the helper, the header declaration, and the main.cpp call site, and update the comments in main.cpp + ProxySQL_PluginManager.cpp to point at Admin::init() as the single canonical materialization site. Leave a NOTE in Admin_Bootstrap.cpp at the old call site so anyone re-adding a similar helper sees why the prior one was removed. 4) plugins/mysqlx/include/mysqlx_session.h + plugins/mysqlx/src/mysqlx_session.cpp: gate the two genuine forgery vectors behind MYSQLX_TEST_BUILD. inject_identity_for_test() bypasses the full auth flow — no credential check, no capability negotiation, just sets identity_ to a caller-supplied MysqlxResolvedIdentity. resolve_backend_target_ for_test() drives a private routing helper without an authenticated identity. Both are necessary for unit tests but should not be reachable at all in shipped binaries; an in-process exploit reaching the session can call inject_identity_for_test() to forge an authenticated identity. Wrap them in #ifdef MYSQLX_TEST_BUILD; define -DMYSQLX_TEST_BUILD in the test Makefile only. The remaining target__for_test() getters are read-only state observers and are left unconditional — they cannot mutate the session and a debugger could observe the same state regardless. 5) test/tap/tests/unit/Makefile: define -DMYSQLX_TEST_BUILD on every unit test compile line via OPT. This is the test-only knob that re-enables the gated forgery methods so unit tests still compile. plugins/mysqlx/Makefile does NOT define this macro, so the production .so does not compile in the entry points. Verified locally: - plugins/mysqlx/ProxySQL_MySQLX_Plugin.so builds clean with PROXYSQL40=1 PROXYSQL31=1 PROXYSQLFFTO=1 PROXYSQLTSDB=1 PROXYSQLGENAI=1 (no MYSQLX_TEST_BUILD). - test/tap/tests/unit/mysqlx_robustness_unit-t builds and runs: 74/74 assertions pass, including the ones that exercise inject_identity_for_test (visible only because the test Makefile defines MYSQLX_TEST_BUILD).	2 months ago
Rene Cannao	3ba92815f8	fix(plugin-chassis): make chassis fully invisible in v3.x builds Review feedback identified that the chassis was leaking into the v3.0 stable build via unfenced references in core code: * Admin_Bootstrap.cpp had `if (auto* m = proxysql_get_plugin_manager())` and a full materialize_plugin_tables() implementation outside any PROXYSQL40 fence * proxysql_admin.h declared materialize_plugin_tables() * proxysql_glovars.hpp held GloVars.plugin_modules and proxysql_load_plugin_modules_from_config() * ProxySQL_GloVars.cpp had three unfenced .clear()/use sites * ProxySQL_Admin.cpp::dispatch_plugin_admin_command (template + two instantiations) was unfenced * src/main.cpp's GloPluginManager, LoadConfiguredPlugins, InitConfiguredPlugins, StartConfiguredPlugins, StopConfiguredPlugins, UnloadPlugins teardown, the four-phase orchestration block, and the proxysql_load_plugin_modules_from_config call were all unfenced Separately, Admin_Handler.cpp held 16 hardcoded MYSQLX alias vectors (LOAD_MYSQLX_USERS_FROM_MEMORY, etc.) and a 16-deep if-ladder that called resolve_admin_alias_to_canonical(). This embedded plugin-specific knowledge in core code -- core should know nothing about any specific plugin's commands. The agreed architecture: v3.0/v3.1 builds have NO plugin loader at all; the entire plugin chassis (including its public manager surface, its loader, and its dispatch helpers) only exists under PROXYSQL40. v3.0 core code therefore holds zero plugin-related code paths. Changes: * include/ProxySQL_Plugin.h, include/ProxySQL_PluginManager.h, lib/ProxySQL_PluginManager.cpp: wrap entire file body in `#ifdef PROXYSQL40 ... #endif`. Including the header from a v3.x translation unit yields nothing; the .cpp compiles to an empty object file. Removed redundant inner PROXYSQL40 fences inside the now-empty v3.x body of Plugin.h (kept the few that are inside the file-wide outer for clarity). PROXYSQL_PLUGIN_ABI_VERSION is now unconditionally 2 (the macro is no longer reachable from a v3.x compile, so the PROXYSQL40-vs-1 split is moot). * include/proxysql_glovars.hpp, lib/ProxySQL_GloVars.cpp: gate GloVars.plugin_modules field, all .clear() sites, the proxysql_load_plugin_modules_from_config declaration + definition. * include/proxysql_admin.h: gate materialize_plugin_tables and dispatch_plugin_admin_command<S> declarations. * lib/Admin_Bootstrap.cpp: gate `#include "ProxySQL_PluginManager.h"`, the `if (auto* m = proxysql_get_plugin_manager())` merge block, and the materialize_plugin_tables definition. * lib/Admin_Handler.cpp: delete the 16 LOAD_/SAVE_MYSQLX_* const std::vector<std::string> declarations entirely. Delete the helper resolve_admin_alias_to_canonical(). Delete the 16-deep if-ladder + the !PROXYSQL40 fallback dispatch branch. The remaining plugin dispatch is one generic call to proxysql_resolve_configured_plugin_admin_alias() inside an #ifdef PROXYSQL40 block; v3.x compiles it out entirely. Core now holds no plugin-specific knowledge of any kind. * lib/ProxySQL_Admin.cpp: gate dispatch_plugin_admin_command template body + both explicit instantiations. * src/main.cpp: gate `#include "ProxySQL_PluginManager.h"`, GloPluginManager declaration, all four wrapper functions (LoadConfiguredPlugins / InitConfiguredPlugins / StartConfiguredPlugins / StopConfiguredPlugins), the StopConfiguredPlugins call inside UnloadPlugins, the four-phase orchestration in ProxySQL_Main_init_phase2___not_started, the GloVars.plugin_modules.clear() and the proxysql_load_plugin_modules_from_config call in ProxySQL_Main_process_global_variables. v3.x builds reduce that whole block to a single ProxySQL_Main_init_Admin_module call. * plugins/mysqlx/src/mysqlx_plugin.cpp, plugins/mysqlx/src/mysqlx_admin_schema.cpp: drop the !PROXYSQL40 fallback paths. The plugin only loads under PROXYSQL40 now (no loader exists in v3.x), so the legacy combined-init code path and the 16 fallback register_command() calls were dead code. The descriptor unconditionally wires register_schemas; mysqlx_init holds only context setup. * Makefile (top-level): the four build_src_* recipes that recurse into plugins/mysqlx are wrapped in `$(if $(filter 1,$(PROXYSQL40)),...)`. v3.x builds emit "[skip] mysqlx plugin (PROXYSQL40 not set)" instead of building the .so. Clean targets stay unconditional. * test/tap/tests/unit/mysqlx_admin_alias_resolution_unit-t.cpp: delete the test (it covered resolve_admin_alias_to_canonical, which is itself deleted). Remove from unit Makefile UNIT_TESTS list. Verification: * v3.0 stable build (`make`, no flags): - Produces 147 MB src/proxysql binary - `nm src/proxysql \| grep -cE 'invoke_register_schemas_phase\|GloPluginManager\|materialize_plugin_tables'` -> 0 - mysqlx plugin .so is correctly skipped: "[skip] mysqlx plugin (PROXYSQL40 not set)" * PROXYSQLGENAI=1 build (cascades PROXYSQL40/31/FFTO/TSDB): - Produces 200 MB src/proxysql binary with chassis symbols - mysqlx plugin .so built (9.2 MB), contains mysqlx_register_schemas * All 5 plugin unit tests pass under PROXYSQL40: plugin_config_unit-t 48/48 plugin_dispatch_unit-t 52/52 plugin_lifecycle_unit-t 26/26 plugin_query_hook_unit-t 46/46 plugin_prometheus_unit-t 10/10	2 months ago
Rene Cannao	5c1549dcc6	merge: resolve conflicts with v3.0 branch Merge origin/v3.0 into ProtocolX. One conflict in test/tap/tests/unit/Makefile resolved by keeping both mysqlx unit tests (ProtocolX) and pgsql_servers_ssl_params unit test + GENAI vec.o link (v3.0).	3 months ago
Rene Cannao	82fd3afb34	feat(mysqlx): wire admin interface for MYSQLX LOAD/SAVE commands The MYSQLX plugin admin commands (LOAD/SAVE MYSQLX USERS/ROUTES/ BACKEND_ENDPOINTS/VARIABLES TO RUNTIME/FROM MEMORY) were returning 'syntax error' because the MYSQLX dispatch block in Admin_Handler.cpp was placed AFTER the generic LOAD/SAVE catch-all at line 3883. This meant all LOAD/SAVE commands were intercepted by admin_handler_command_load_or_save() which knows nothing about MYSQLX. Changes: - Move MYSQLX dispatch block before the generic LOAD/SAVE handler so MYSQLX commands are checked first (Admin_Handler.cpp lines ~3883-3900) - Add LOAD_MYSQLX_VARIABLES_FROM_MEMORY and SAVE_MYSQLX_VARIABLES_TO_MEMORY alias vectors for MYSQLX VARIABLES commands - Add ProxySQL_Admin::materialize_plugin_tables() in Admin_Bootstrap.cpp that merges plugin table definitions into admin/config/stats table lists and creates the tables in SQLite - Call materialize_plugin_tables() in main.cpp after LoadConfiguredPlugins() but before StartConfiguredPlugins() so plugin tables exist before the plugin starts - Declare materialize_plugin_tables() in proxysql_admin.h This ensures that when a user issues e.g. 'LOAD MYSQLX USERS TO RUNTIME', the command reaches dispatch_plugin_admin_command() which forwards it to the MYSQLX plugin's registered admin command handlers.	3 months ago
René Cannaò	669ab91494	Merge pull request #5594 from sysown/v3.0-lint lint + static analysis: clang-tidy and cppcheck fixes	3 months ago
Rene Cannao	7e1a78fd6b	lint: fix reserved identifiers and include guards - Rename __HGM_TABLES_SIZE -> HGM_TABLES_SIZE_ (Base/MySQL/PgSQL HGM) - Rename __TYPE_SIZE -> TYPE_SIZE_ (Base/MySQL/PgSQL HGM) - Fix include guard defines: __PROXYSQL_* -> PROXYSQL_* - Fix include guard comments: __CLASS_* -> PROXYSQL_*_H (29 files) - Add uint8_t underlying type to small enums	3 months ago
Rene Cannao	9bdfb8b1cd	lint: optimize enum types and fix reserved identifiers in Query_Cache, admin, thread, debug performance-enum-size fixes: - p_qc_counter, p_qc_gauge: add uint8_t (7 and 2 values) - p_admin_counter, p_admin_gauge, p_admin_dyn_counter, p_admin_dyn_gauge: add uint8_t - p_debug_dyn_counter: add uint8_t (2 values) bugprone-reserved-identifier fixes: - Rename __size -> SIZE_ in all metric enums - Update std::array size references across multiple files - Update prometheus_helpers.h template parameter names and comments Files changed: - Query_Cache.h - proxysql_admin.h - MySQL_Thread.h - proxysql_debug.h - prometheus_helpers.h (template changes) - debug.cpp (usage update)	3 months ago
Rene Cannao	4810f1f05b	lint: fix reserved identifier in include guards Replace __CLASS_* and ____CLASS_* patterns with PROXYSQL_*_H to comply with the C++ standard where identifiers starting with __ are reserved in the global namespace. Also fixes: - ____CLASS_STANDARD_MYSQL_THREAD_H -> PROXYSQL_STANDARD_MYSQL_THREAD_H - ____CLASS_STANDARD_PGSQL_THREAD_H -> PROXYSQL_STANDARD_PGSQL_THREAD_H - __CLASS_PROXYSQL_POLL -> PROXYSQL_POLL_H (added _H for consistency)	3 months ago
Rene Cannao	17225cf2a8	lint: fix bugprone-macro-parentheses in 6 header files - MySQL_Thread.h: wrap negative hostgroup constants in parens - PgSQL_Thread.h: wrap negative hostgroup constants in parens - proxysql_admin.h: wrap expression multipliers in parens - proxysql_debug.h: wrap macro args in ASSERT_SQLITE* macros - proxysql_macros.h: wrap macro args in strdup_null, free_null, itostr, Copy4B - proxysql_structs.h: wrap macro args in IS_PGTRACKED_VAR_OPTION_* macros	3 months ago
Rene Cannao	80ff2242e6	merge: resolve v3.0 conflict and add pgsql_servers_ssl_params cluster sync v3.0 added pgsql_servers_ssl_params table. Integrate into pgsql cluster sync path: - Add CLUSTER_QUERY_PGSQL_SERVERS_SSL_PARAMS to Admin_Handler interception block alongside other pgsql cluster queries - Add ssl_params fetch query to pull_pgsql_servers_v2_from_peer - Add update_pgsql_servers_ssl_params() for applying synced data - Update convert_pgsql_servers_resultsets to handle 5 result sets - Include ssl_params in v2 checksum computation (4 tables)	3 months ago
René Cannaò	8a61232517	fix: decouple MySQL and PostgreSQL checksum/sync controls Two bugs in the PostgreSQL cluster sync admin variables: 1. Cross-protocol coupling (critical): Disabling a MySQL checksum flag (e.g. SET admin-checksum_mysql_query_rules='false') incorrectly zeroed out the corresponding PostgreSQL cluster sync threshold (cluster_pgsql_query_rules_diffs_before_sync). An operator disabling MySQL checksums would unexpectedly halt PostgreSQL cluster sync. Fix: MySQL checksum setters now only affect MySQL sync thresholds. The warning messages no longer mention pgsql side effects. 2. Single-gate for all pgsql modules (operational limitation): All four PostgreSQL cluster sync thresholds (cluster_pgsql_{query_rules,servers,users,variables}_diffs_before_sync) were gated by a single flag (checksum_pgsql_variables). This meant you could not independently control checksum computation per-module. By contrast, MySQL has per-module flags (checksum_mysql_query_rules, checksum_mysql_servers, checksum_mysql_users, checksum_mysql_variables). Fix: Add three new per-module admin variables: - admin-checksum_pgsql_query_rules - admin-checksum_pgsql_servers - admin-checksum_pgsql_users Each gates only its corresponding cluster_*_diffs_before_sync setter, mirroring the MySQL pattern. The existing admin-checksum_pgsql_variables now only gates cluster_pgsql_variables_diffs_before_sync. Disabling admin-checksum_pgsql_variables no longer cascades to zero out query_rules/servers/users sync thresholds. Files changed: - include/proxysql_admin.h: Add checksum_pgsql_query_rules, checksum_pgsql_servers, checksum_pgsql_users to checksum_variables struct. - lib/ProxySQL_Admin.cpp: Add variable names, constructor defaults, get/set handlers for new flags. Update diffs_before_sync guards to use per-module flags. Remove pgsql zeroing from MySQL checksum setters. Fix checksum_pgsql_variables setter to only affect pgsql variables sync threshold.	3 months ago
Rene Cannao	47758a468d	merge: resolve groups.json conflict with v3.0 Take v3.0's groups.json as base (legacy-gN naming, mysql84-gN groups, new test entries) and add test_cluster_sync_pgsql-t from this branch. Drop duplicate pgsql_admin_metacmds-t (already exists as pgsql-admin_metacmds-t in v3.0).	3 months ago
Rene Cannao	cd15afdd1e	feat: support plugin-owned admin tables and commands	3 months ago
Rahim Kanji	516fa468ba	feat: add admin load/save/dump for pgsql_servers_ssl_params	3 months ago
Rene Cannao	c8408d76b3	test: add 6 unit test files with 588 assertions (Round 5) New test files (test/tap/tests/unit/): Core: - pgsql_tokenizer_unit-t (65 tests): PgSQL SQL digest — dollar- quoted strings, ::type casts, ARRAY literals, boolean literal replacement, E/B/X/U& prefix strings, comment handling, grouping/IN-list collapsing, edge cases - admin_disk_upgrade_unit-t (60 tests): all 6 disk_upgrade_*() schema migration functions with in-memory SQLite — scheduler v1.2.0→current, mysql_users v1.3.0→current, rest_api_routes v2.0.15→current, mysql_servers v1.1.0→current (with weight cap and compression normalization bugs), pgsql_replication_hostgroups v3.0.1→current, mysql_query_rules v1.2.2→current - gtid_utils_unit-t (37 tests): addGtid interval merging (adjacent, non-consecutive, duplicate, bridge, reverse-order, within- existing), addGtidInterval (update, duplicate, non-overlapping), gtid_executed_to_string formatting - glovars_unit-t (78 tests): GloVars constructor defaults, ProxySQL_Checksum_Value (set/replace_zeros/version/epoch), generate_global_checksum determinism, command-line parsing (--initial/--reload/--no-version-check/-D/--bootstrap/-n/-f/-M) - query_processor_unit-t (+217 tests, 259 total): EXTENDED with PgSQL-specific tests — all-fields rule creation, client_addr wildcard, flagOUT attributes JSON, command counters, stats, rule text 35-column serialization, query_parser_command_type for 100+ PgSQL commands (DML/DDL/views/sequences/functions/ triggers/policies/LISTEN/NOTIFY/VACUUM/TEXT SEARCH/etc.), fast routing hashmap, firewall whitelist, query digests GenAI/MCP (PROXYSQLGENAI=1): - genai_mysql_catalog_unit-t (89 tests): MySQL_Catalog with in-memory SQLite — upsert/get/remove, schema isolation, list with pagination, FTS search, merge, multiple kinds, special character handling Production changes for testability: - include/proxysql_admin.h: add friend class TestDiskUpgrade - test/tap/tests/unit/Makefile: add DEBUG auto-detection from libproxysql.a symbols (fixes struct layout ODR violations when library is built with -DDEBUG) Running totals: 32 test files, 2553 assertions, all passing.	3 months ago
René Cannaò	efedd93c92	Rename stats_global to stats_proxysql_global to avoid strstr substring collision stats_global is a substring of stats_mysql_global and stats_pgsql_global, causing the strstr-based dispatch to match incorrectly. Exclusion guards don't work either because a query referencing both tables would suppress the stats_proxysql_global refresh. Renaming to stats_proxysql_global follows the existing naming convention (stats_mysql_global, stats_pgsql_global) and eliminates the substring collision entirely.	3 months ago
copilot-swe-agent[bot]	e274862857	feat: introduce stats_global table and move TLS metrics out of stats_mysql_global; add TAP tests - Add STATS_SQLITE_TABLE_GLOBAL definition (stats_global table for non-MySQL/PgSQL metrics) - Register stats_global in Admin_Bootstrap.cpp - Add stats___global() implementation and declaration; document with Doxygen comment - Remove TLS_* variables from stats___mysql_global() - they were misplaced there - Move all TLS tracking metrics to stats___global() under ssl_mutex - Wire up stats_global query detection and refresh in GenericRefreshStatistics() - Add TAP test test_tls_stats-t.cpp: - Verifies stats_global contains all 6 TLS tracking variables - Checks value ranges and validity of each TLS variable - Verifies stats_tls_certificates has 2 rows (server + ca) with correct fields - Verifies TLS_Load_Count increments and TLS_Last_Load_Timestamp increases after PROXYSQL RELOAD TLS - Confirms TLS variables are absent from stats_mysql_global Co-authored-by: renecannao <3645227+renecannao@users.noreply.github.com>	3 months ago
copilot-swe-agent[bot]	4f88740b20	feat: add stats_tls_certificates table and TLS tracking metrics - Add STATS_SQLITE_TABLE_TLS_CERTIFICATES definition with cert_type, file_path, subject_cn, issuer_cn, serial_number, not_before, not_after, days_until_expiry, sha256_fingerprint, and loaded_at columns - Add TLS tracking fields to GloVars.global: tls_load_count, tls_last_load_timestamp, tls_last_load_ok, tls_cert_file, tls_ca_file, tls_key_file - Update ProxySQL_create_or_load_TLS() to populate tracking fields on success - Register stats_tls_certificates table in Admin_Bootstrap - Add stats___tls_certificates() function that reads certs from disk at query time, extracts metadata (CN, serial, validity dates, SHA-256 fingerprint, days_until_expiry) and populates the table - Add TLS variables to stats___mysql_global(): TLS_Load_Count, TLS_Last_Load_Timestamp, TLS_Last_Load_Result, TLS_Server_Cert_File, TLS_CA_Cert_File, TLS_Key_File - Wire up query detection and refresh in GenericRefreshStatistics() Co-authored-by: renecannao <3645227+renecannao@users.noreply.github.com>	3 months ago
Rene Cannao	6b9918fdcc	chore: refine guard structures and fix indentation - Fixed indentation for TSDB methods in include/proxysql_admin.h. - Refined PROXYSQLTSDB guard structure in lib/Admin_FlushVariables.cpp. - Cleaned up indentation and redundant code in lib/ProxySQL_RESTAPI_Server.cpp. - Ensured consistent formatting across modified files.	4 months ago
Rene Cannao	c2f678a9de	feat: implement release tiers and feature guards for v3.0, v3.1, and v4.0 This commit introduces a structured release tier system controlled by environment variables, allowing three distinct ProxySQL versions to be built from the same codebase while maintaining a clear upgrade path. Key Changes: - Tier 1 (v3.0.x Stable): Default build including bug fixes and core enhancements. - Tier 2 (v3.1.x Innovative): Enabled via PROXYSQL31=1. Adds FFTO and TSDB. - Tier 3 (v4.0.x AI/MCP): Enabled via PROXYSQLGENAI=1. Adds the AI stack. Implementation Details: - Makefile: Implemented dynamic versioning logic that increments the minor version for v3.1 and the major version for v4.0. Added tier documentation. - Feature Guards: Introduced PROXYSQLFFTO and PROXYSQLTSDB guards across ProxySQL_Admin, ProxySQL_Statistics, REST API, and HTTP server. - FFTO: Guarded Fast Forward Traffic Observer in MySQL and PgSQL sessions, including thread-local variables and configuration parameters. - TSDB: Guarded the entire Time Series Database subsystem, including SQLite table definitions, background loops, and REST API endpoints. - Build Fixes: * Fixed 'proxy_debug_func' linkage by adding extern "C" and moving the definition outside of DEBUG guards to ensure non-debug builds link correctly. * Corrected misplaced/nested DEBUG and PROXYSQLFFTO guards. * Verified build stability for all tiers (unset, PROXYSQL31=1, PROXYSQLGENAI=1).	4 months ago
Wazir Ahmed	5421a382ab	fix: Load TSDB variables from disk during startup - TSDB variables stored on disk were not being loaded into runtime during startup. Add the missing `flush_tsdb_variables___database_to_runtime()` and invoke it during init. - Inline the existing load/save wrappers in the header as thin delegations to the underlying flush functions Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	4 months ago
Wazir Ahmed	1684d30725	fix: Refresh Prometheus metrics before TSDB sampling - `tsdb_sampler_loop()` calls `registry->Collect()` without refreshing module metrics, causing stale metric values to be stored in the TSDB. - Added `update_modules_metrics()` call before `Collect()` in the sampler. Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	4 months ago
Rene Cannao	6dbd0e9cb1	Merge branch 'v3.0' into fix/postgresql-cluster-sync_2 Resolve merge conflicts in: - lib/ProxySQL_Cluster.cpp - test/tap/tap/Makefile Conflict resolution details: - Kept PostgreSQL/MySQL cluster-sync updates from this branch (SQLQueries delete constants and pgsql sync flow changes) while adopting upstream SQLite prepare_v2 RAII usage where introduced in v3.0 conflict regions. - Combined TAP archive fixes from this branch (archive recreation via rm -f to prevent stale objects) with upstream v3.0 mcp_client.o additions to libtap archives. Validation performed after conflict resolution: - make -C test/tap/tap debug -j4 (success) - make -C lib -j4 (fails in current workspace due missing sqlite-vec.h/vec.o dependency in broader v3.0 content, not conflict markers) - make -C test/tap/tests admin-listen_on_unix-t -j4 (fails in current workspace due missing deps/sqlite3/sqlite3/vec.o, same dependency issue) No conflict markers remain.	4 months ago
Rene Cannao	e53d2c76f9	Address PR #5410 review findings across FFTO, sessions, TAP, and docs Apply actionable bot review feedback from PR #5410 without behavior drift:\n- fix MySQL/PgSQL FFTO digest normalization and max-length hashing\n- add early FFTO packet/message size guards for bypass safety\n- restore MySQL session GTID reset and harden GenAI async error ownership\n- add null-check for stripped KILL query normalization path\n- call FFTO on_close() before observer reset/bypass in session flows\n- fix processlist query-options default pagination limit semantics\n- fix TAP noise_failures locking race and noise-aware plan count\n- extend wait_get_enpoint_ready with optional basic-auth passthrough\n- update MCP architecture docs from observe->stats naming\n- clean corrupted character in pgsql query-cache test comment\n\nValidation run:\n- git diff --check\n- make -C test/tap/tap tap.o utils_mariadb.o utils_mysql57.o utils_mysql8.o -j4\n- make -C lib obj/MySQLFFTO.oo obj/PgSQLFFTO.oo obj/MySQL_Session.oo obj/PgSQL_Session.oo obj/Query_Processor.oo -j4	4 months ago
Rene Cannao	e2ebc0c449	Merge branch 'v4.0-mcp-stats2' into v3.0-unified-large-pr	4 months ago
Rene Cannao	e762f91cda	MCP stats: add in-memory processlist filtering and PgSQL concurrency stress TAP Replace direct stats-schema reads in MCP show_processlist with in-memory processlist access via SQL3_Processlist query options. Add processlist query option types in proxysql_admin interfaces and implement filtering, sorting, and pagination post-processing in both MySQL and PgSQL thread implementations. Introduce configurable MCP cap mcp_stats_show_processlist_max_rows (default 200, max 1000), wire it through MCP variable loading, and expose it in default config. Expand Stats_Tool_Handler processlist handling to parse filter/sort arguments, enforce cap metadata, and return consistent MCP payload fields from in-memory snapshots. Add extensive doxygen documentation in the touched headers and source blocks to describe API contracts, filtering behavior, and runtime constraints. Add new TAP test mcp_pgsql_concurrency_stress-t that generates sustained concurrent PgSQL traffic (simple reads, read/write table workload, randomized pg_sleep) while polling MCP show_processlist and show_queries in parallel with filter and ordering assertions. Validation performed locally: mcp_pgsql_concurrency_stress-t (31 assertions) and mcp_show_queries_topk-t (12 assertions).	4 months ago
René Cannaò	9c6945fae8	Merge branch 'v3.0' into v3.0-tsdb-feature Signed-off-by: René Cannaò <rene@proxysql.com>	4 months ago
Rene Cannao	70b61a86e3	MCP stats: implement in-memory show_queries Top-K path Replace stats table reads in show_queries with Query Processor in-memory aggregation to avoid stale/empty admin-db snapshots.\n\nIntroduce Top-K digest filtering primitives and wire them into Stats_Tool_Handler so filtering/sorting/limit are executed against live digest memory. Add runtime MCP variable mcp-stats_show_queries_max_rows with validation, defaulting, and configuration plumbing, plus cap metadata in the tool response.\n\nEnforce configurable hard cap semantics while preserving deterministic ordering and pagination behavior for MCP clients. Add internal DEBUG validator path (PROXYSQLTEST 56) in admin tests to stress generated digest data and validate the Top-K/filter pipeline against in-memory state.\n\nKeep the validator callable from startup/testing workflows without depending on MCP linkage. Add TAP coverage in test/tap/tests/mcp_show_queries_topk-t.cpp for:\n- MCP endpoint setup and reachability\n- digest generation + optional DEBUG internal validator\n- payload validation and cap metadata assertions\n- descending count order validation\n- match_digest_text filter verification\n\nMake payload parsing backward-compatible in TAP (direct tool result object and legacy wrapped success/result payloads).	4 months ago
Rene Cannao	2a46c239be	Add PostgreSQL advanced eventslog schema and variable scaffolding	4 months ago
Rene Cannao	5f5b7d604f	tsdb: improve administrative interface and command support - Move 'stats_tsdb' to 'main' schema for better visibility in 'SHOW TSDB STATUS'. - Implement 'PROXYSQL TSDB DOWNSAMPLE' command to trigger manual metrics aggregation. - Ensure 'SHOW TSDB STATUS' triggers a statistics refresh before returning data. - Integrate 'tsdb-' variables into 'runtime_global_variables' refresh logic. - Update TAP tests to verify new commands and improve troubleshooting output.	4 months ago
Rene Cannao	2d51aeff64	tsdb: correctly implement SHOW TSDB commands and variable initialization	4 months ago
Rene Cannao	a236a13787	tsdb: dedicated module for TSDB variables with tsdb- prefix and optimized background tasks	4 months ago
Rene Cannao	da877c378b	tsdb: align admin variable lifecycle, full prometheus ingestion, docs and tap coverage	4 months ago
Rene Cannao	f17e999830	Implement SQLite-based TSDB subsystem Add time-series database extension to ProxySQL_Statistics with: - Three SQLite tables: tsdb_metrics, tsdb_metrics_hour, tsdb_backend_health - Automatic downsampling from raw to hourly aggregates - Retention management (7 days raw, 1 year hourly) - Prometheus metrics sampling - Backend health monitoring via TCP probes - SQL query interface with JSON label filtering - Admin variable integration Documentation: - embedded_tsdb_overview.md - Architecture and benefits - embedded_tsdb_architecture.md - System design and components - embedded_tsdb_reference.md - Configuration and API reference - embedded_tsdb_specs.md - Technical specifications - embedded_tsdb_quickstart.md - Getting started guide All 4 build combinations pass: - make debug / make release - PROXYSQLGENAI=1 make debug / make release	4 months ago
René Cannaò	ce9fdc9d7a	Merge branch 'v3.0' into v3.0.6-implement_FLUSH_STATS Signed-off-by: René Cannaò <rene@proxysql.com>	5 months ago
Rene Cannao	48bc7dd7bf	Merge v4.0 GenAI features into v3.0 with conditional compilation This commit merges the experimental v4.0 GenAI/MCP features into the stable v3.0 branch using conditional compilation. All v4.0 features are disabled by default and only enabled when PROXYSQLGENAI=1 is set at compile time. Changes: Build System: - Modified main Makefile to pass PROXYSQLGENAI flag to sub-makefiles - Modified deps/Makefile to conditionally build sqlite-vec and sqlite-rembed - Modified lib/Makefile to add PSQLGA flag and include GenAI object files - Modified src/Makefile to add PSQLGA flag and conditional linking Headers (wrapped with #ifdef PROXYSQLGENAI): - All 20 new GenAI header files in include/ - Modified cpp.h, proxysql_glovars.hpp, proxysql_admin.h - Modified ProxySQL_Admin_Tables_Definitions.h for GenAI/MCP tables Source Files: - All 22 new GenAI source files in lib/ wrapped with #ifdef PROXYSQLGENAI - Modified src/main.cpp for conditional global variables and init/shutdown - Modified Admin_Handler.cpp for conditional command handlers - Modified Admin_Bootstrap.cpp for conditional table registration - Modified Admin_FlushVariables.cpp for conditional variable flushing - Modified ProxySQL_Admin.cpp for conditional admin methods - Modified ProxySQL_Admin_Stats.cpp for conditional MCP stats functions - Modified proxy_sqlite3_symbols.cpp to always compile (needed by core) - Modified MySQL_Session.cpp for conditional GenAI function calls Test Files: - Renamed test_mcp_query_rules-t to test_mcp_query_rules-t.sh - Renamed test_mcp_rag_metrics-t to test_mcp_rag_metrics-t.sh - Modified anomaly_detection-t.cpp for conditional test execution Usage: # Build without GenAI (v3.0 mode - default) make clean && make build_deps -j$(nproc) && make build_lib -j$(nproc) && make build_src -j$(nproc) # Build with GenAI (v4.0 mode) make clean && PROXYSQLGENAI=1 make build_deps -j$(nproc) && PROXYSQLGENAI=1 make build_lib -j$(nproc) && PROXYSQLGENAI=1 make build_src -j$(nproc)	5 months ago
Miro Stauder	cc1d93d08b	implement admin command 'PROXYSQL FLUSH STATS' for DEBUG builds	5 months ago
Wazir Ahmed	50de536534	MCP: Fix crash during server restarts Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	5 months ago
Rene Cannao	b4f521c634	Merge v3.1-MCP2 into v3.1-vec Conflict resolution summary: Features preserved from v3.1-vec: - mcp_use_ssl variable (HTTP/HTTPS mode support) - RAG tool handler support - FTS (Full Text Search) functionality in MySQL_Tool_Handler Changes accepted from v3.1-MCP2: - Schema isolation for catalog (removed mcp_catalog_path/fts_path variables) - Schema parameter added to catalog functions - Comprehensive query rewriting improvements in Query_Tool_Handler - Format string fixes in vector_db_performance-t.cpp Key decisions: - catalog_path/fts_path: Removed per v3.1-MCP2 commit `35b0b224` (intentional) - MySQL_Catalog.cpp: Accepted v3.1-MCP2 version (schema isolation improvements) - Query_Tool_Handler.cpp: Accepted v3.1-MCP2 version (significant improvements) - Makefile: Kept v3.1-vec's EXCLUDE_TRACKING_VARIABLES (correct spelling) Build notes: - MySQL_Tool_Handler now requires schema parameter for catalog operations - Catalog path is hardcoded to datadir/mcp_catalog.db - FTS functionality preserved but may need updates for schema isolation	5 months ago
René Cannaò	75a62f334d	Merge branch 'v4.0' into v3.1-vec Signed-off-by: René Cannaò <rene@proxysql.com>	5 months ago
Rene Cannao	cc3cc25532	fix: Remove unused reset parameter from stats___mcp_query_rules() Change function signature from stats___mcp_query_rules(bool reset) to stats___mcp_query_rules() to match MySQL query rules pattern. The reset parameter was never used in the function body and MySQL's stats___mysql_query_rules() has no reset parameter.	5 months ago
Rene Cannao	f01fc79584	feat: Add runtime_mcp_query_rules table and fix stats_mcp_query_rules schema - Add ADMIN_SQLITE_TABLE_RUNTIME_MCP_QUERY_RULES schema (17 columns, same as mcp_query_rules) - Fix STATS_SQLITE_TABLE_MCP_QUERY_RULES to only have rule_id and hits columns - Add runtime_mcp_query_rules detection and refresh in ProxySQL_Admin - Implement save_mcp_query_rules_from_runtime(bool _runtime) for both config and runtime tables - Update get_mcp_query_rules() to return 17 columns (no hits) - get_stats_mcp_query_rules() returns 2 columns (rule_id, hits) Mirrors the MySQL query rules pattern: - mcp_query_rules: config table (17 cols) - runtime_mcp_query_rules: runtime state (17 cols) - stats_mcp_query_rules: hit counters (2 cols)	5 months ago
Rene Cannao	fb66af7c1b	feat: Expose MCP catalog database in ProxySQL Admin interface The MCP catalog database is now accessible as the 'mcp_catalog' schema from the ProxySQL Admin interface, enabling direct SQL queries against discovered schemas and LLM memories.	5 months ago
Rene Cannao	a816a756d4	feat: Add MCP query tool usage counters to stats schema Add stats_mcp_query_tools_counters and stats_mcp_query_tools_counters_reset tables to track MCP query tool usage statistics. - Added get_tool_usage_stats_resultset() method to Query_Tool_Handler - Defined table schemas in ProxySQL_Admin_Tables_Definitions.h - Registered tables in Admin_Bootstrap.cpp - Added pattern matching in ProxySQL_Admin.cpp - Added stats___mcp_query_tools_counters() in ProxySQL_Admin_Stats.cpp - Fixed friend declaration for track_tool_invocation() - Fixed Discovery_Schema.cpp log_llm_search() to use prepare_v2/finalize	5 months ago
Rene Cannao	4018a0ad3b	fix: Follow MCP pattern for GenAI variables runtime table population Update flush_genai_variables___database_to_runtime() to match the MCP pattern exactly: - Add 'lock' parameter (default true) for flexibility - Use ProxySQL_Admin's wrlock()/wrunlock() instead of GloGATH's - Use consistent variable naming (var_name = name + 6 for 'genai-' prefix) - Follow exact same locking pattern as MCP variables This fixes the issue where runtime_global_variables table was not being populated on startup because the locking pattern was incorrect.	5 months ago
Rene Cannao	b177750242	fix: correct PostgreSQL cluster sync architecture by removing separate module infrastructure - Remove incorrect separate module declarations for pgsql_replication_hostgroups and pgsql_hostgroup_attributes from ProxySQL_Cluster.hpp - Remove separate module counter enums and atomic variables - Remove separate module metrics definitions and configurations - Remove incorrect admin variable implementations for non-distinct modules - Verify that PostgreSQL tables correctly sync as part of unified pgsql_servers module - Confirm build succeeds and no remaining references to removed infrastructure These tables are part of the pgsql_servers module and should not be treated as separate modules. The PgSQL_HostGroups_Manager::commit_update_checksums_from_tables() function already correctly includes both tables in the unified checksum computation. Resolves the architectural error where separate module infrastructure was added for tables that are not distinct modules.	6 months ago
Rene Cannao	f9e3e85e68	fix: address all AI review concerns for PostgreSQL cluster sync - Fix critical Admin_FlushVariables bug where PostgreSQL SELECT appended to MySQL checksum query - Add security check for integer overflow in pgsql_variable_validate_integer - Fix test suite tuple indexing and NULL safety issues - Add missing admin variables for PostgreSQL replication hostgroups and attributes - Implement checksum_pgsql_variables get/set variable behavior - Fix misleading log messages for PostgreSQL disable operations - Fix cluster_pgsql_variables_diffs_before_sync gating logic - Add missing sync_delayed metrics for pgsql modules - Fix CLUSTER_QUERY_RUNTIME_PGSQL_SERVERS macro numeric vs VARCHAR comparison Resolves issues identified by coderabbitai and gemini-code-assist reviews in PR 5297.	6 months ago
René Cannaò	5154f9e4c4	Merge branch 'v3.0' into fix/postgresql-cluster-sync_2 Signed-off-by: René Cannaò <rene@proxysql.com>	6 months ago
René Cannaò	313f637cf0	Merge branch 'v3.1-vec' into v3.1-MCP1 Signed-off-by: René Cannaò <rene.cannao@gmail.com>	6 months ago

1 2 3 4 5 ...

281 Commits (feature/pgsql-native-backend-protocol)