1043 Commits (docs/passthrough-auth-spec)

Author	SHA1	Message	Date
Rene Cannao	8dfe921ad4	feat(passthrough-auth): counters + stats_mysql_passthrough_auth_metrics ... Adds operational-visibility metrics for the pass-through auth feature. Previously the only diagnostics available to an SRE were the audit log (optional, often off) and proxy_warning (added in the preceding commit). Both are good but neither lets an operator answer "is anything happening?" / "how often does pass-through succeed?" at a glance. Nine atomic counters added to MySQL_Passthrough_Auth_Cache, incremented at the single well-defined call site each: probes_attempted -- a real probe was about to run (past all eligibility gates, past inflight cap, past rate limits) probes_ok -- probe succeeded, cache populated probes_failed_credentials -- probe got 1045/1698/1130 from backend probes_failed_transport -- probe got 2xxx / unknown errno lockouts_user -- per-user lockout fired lockouts_ip -- per-IP lockout fired inflight_cap_rejects -- max_inflight_probes saturated cache_hits -- PPHR_verify_password used the cache cache_invalidations -- backend ER 1045 evicted a cached entry Plus two current-state gauges (read on demand from existing methods): inflight_probes -- current Glo->inflight() cache_entries -- current Glo->size() Exposed via a new admin virtual table: stats_mysql_passthrough_auth_metrics( metric_name VARCHAR NOT NULL PRIMARY KEY, metric_value BIGINT NOT NULL ) stats___mysql_passthrough_auth_metrics is registered alongside the existing stats___mysql_passthrough_auth_cache; both refresh from GloMyPTAuthCache on SELECT. All counter ops use std::memory_order_relaxed -- stats are advisory and don't need to synchronize with the increments. Counters are monotonic since process start; reset only by process restart. Phase 2 may add a Prometheus exposer alongside this; for now the admin table is the canonical surface for monitoring integrations. Discovered by the production-readiness subagent during the second deep review of PR #5810. Listed there as a blocker for production GA because operational visibility into pass-through misbehavior was effectively zero.	1 month ago
Rene Cannao	29f84c0730	feat(passthrough-auth): stats_mysql_passthrough_auth_cache virtual table ... Adds the read-only observability table from spec §8.6: SELECT username, learned_at, age_s, hostgroup_probed FROM stats_mysql_passthrough_auth_cache Password is never exposed; the snapshot is taken from GloMyPTAuthCache->snapshot() which returns only username + monotonic learned_at timestamp + the hostgroup the probe targeted. age_s is derived from learned_at at query time. Schema lives in ProxySQL_Admin_Tables_Definitions.h and registers through the standard insert_into_tables_defs / strstr-detect / stats___* dispatch in Admin_Bootstrap and ProxySQL_Admin. Cache is empty until pass-through auth wiring lands; the table will return zero rows in current builds.	1 month ago
Wazir Ahmed	6c5d0fd5db	Merge branch 'v3.0' into cluster-simulator	2 months ago
Rene Cannao	e9637be4f7	feat: add SCRAM cache invalidation, TAP test, and bench_connect tool ... Generation counter for SCRAM verifier cache: - Global atomic counter incremented on pgsql_users reload - Thread-local caches check generation on lookup; stale entries are cleared and rebuilt from scratch - scram_cache_invalidate() called from __refresh_pgsql_users() - extern "C" guards added to scram_hmac_cached.h for C++ callers TAP test (pgsql-scram_cache_invalidation-t): - Creates user with pass1, populates SCRAM cache via connect loop - Changes password to pass2, reloads users (invalidates cache) - Verifies pass2 works and pass1 is rejected - Assigned to legacy-g4 test groups Benchmark tool (tools/bench_connect): - Multi-threaded SCRAM+TLS connect/disconnect benchmark - Uses vendored PostgreSQL from deps/ - Added to tools/Makefile with clean target	2 months ago
Wazir Ahmed	eaa57d0f58	Merge branch 'v3.0' into cluster-simulator	2 months ago
Rene Cannao	e51ad030df	feat(admin): pass all 3 DB handles to plugin runtime view refresh (issue #5729)	2 months ago
Rene Cannao	482ce386cd	chore(genai-carveout): strip dead PROXYSQLGENAI blocks + plugin hardening ... Two threads of cleanup that had to land together because they touch both core (where the dead code lives) and the plugin (where the new module-owned state lives): # Strip dead #ifdef PROXYSQLGENAI from lib/, include/, src/ (Spec Q9) Step 7 of the carve-out left PSQLGA empty in lib/Makefile, so every #ifdef PROXYSQLGENAI block in the core build became permanently false. The blocks remained as dead text (visible to readers, irrelevant to the binary). Strip ~600 lines spanning: src/main.cpp 2 sites lib/Admin_Bootstrap.cpp 3 sites (mcpdb open + attach + flush_*_variables hooks) lib/Admin_FlushVariables.cpp 2 sites (~250 lines of stub + #if 0 _ORIGINAL bodies) lib/Admin_Handler.cpp 5 sites (LOAD/SAVE GENAI VARIABLES dispatch + LOAD/SAVE MCP VARIABLES dispatch + MCP profiles handler) lib/ProxySQL_Admin.cpp ~10 sites (init_*_variables + load_mcp_server + load/save_mcp_query_rules + GenericRefreshStatistics MCP probes) lib/ProxySQL_Admin_Stats.cpp 3 stats___mcp_* function bodies include/proxysql_admin.h 7 method declarations include/ProxySQL_Admin_Tables_Definitions.h 4 unused DDL macros The plugin now owns LOAD/SAVE MCP VARIABLES, LOAD/SAVE GENAI VARIABLES, LOAD/SAVE MCP PROFILES, LOAD/SAVE MCP QUERY RULES via its admin command registry; the chassis dispatcher catches them before any of the dead core branches would have fired. Caveat — known break: the `stats_mcp_*` tables are still registered (empty schema) but the populator path that fed them is gone. SELECTs return empty result sets until a follow-up wires the populator into the plugin (likely via register_runtime_view or a parallel stats-side ABI). Tracked separately. # Plugin hardening (review-feedback items 4-7) * plugins/genai/Makefile: add -fvisibility=hidden, -fvisibility-inlines-hidden, -fstack-protector-strong (mirrors plugins/mysqlx). Tag proxysql_plugin_descriptor_v1 with __attribute__((visibility("default"))) so the loader's dlsym still resolves; without the explicit annotation the hidden-default flag would also hide the entry point. * plugins/genai/src/Anomaly_Detector.cpp: convert genai_anomaly_embed_fn to std::atomic<…> with relaxed loads/stores. The detector hot-path read races with init/stop on the lifecycle thread; atomics give us a portable no-torn-pointer guarantee without paying for acquire/release fences we don't need (no other state synchronises with the pointer). Replaces the hand-wavy "no mutex needed" comment with the actual reasoning. * plugins/genai/src/{plugin_main,plugin_commands,plugin_tables}.cpp: route the 17 fprintf(stderr, "genai plugin: ...") log lines through a new genai_log() helper that calls services->log_message when the chassis is wired up and falls back to stderr for early-init / unit-test paths. Lines now land in proxysql.log alongside the rest of the runtime. * plugins/genai/include/MCP_Thread.h: add get_variable_string(name, std::string&) returning a std::string, and switch mcp_save_variables_to_admindb (plugin_main.cpp) to it. The legacy get_variable(name, char*) had an unbounded sprintf into a stack buffer and the SAVE path was passing char val[256] — a real overflow risk for *_endpoint_auth bearer tokens. * plugins/genai/src/plugin_main.cpp: rewrite the genai_init / genai_stop comment block with the actual teardown ordering rationale (consumers before producers; atomic-clear-then-delete for the embed hook; why Prometheus counters stay registered).	2 months ago
Rene Cannao	4552236700	feat(plugins/genai): Step 6 — move RAG/Vector/discovery surface into the plugin ... Final file-move step in the carve-out: relocates the leaf modules that didn't depend on (and weren't depended on by) anything still in core. After this commit, the only PROXYSQLGENAI-guarded code remaining in libproxysql.a is `MCP_Tool_Handler` — the base class the plugin's AI_Tool_Handler / RAG_Tool_Handler inherit from — and that lives in core only because removing it would force pulling those derived plugin classes into core just for the inheritance edge. Step 7 deletes the PROXYSQLGENAI macro entirely. Files moved (git mv preserves history) Headers (include/ -> plugins/genai/include/) AI_Vector_Storage.h MySQL_Catalog.h Discovery_Schema.h Static_Harvester.h PgSQL_Static_Harvester.h Sources (lib/ -> plugins/genai/src/) AI_Vector_Storage.cpp MySQL_Catalog.cpp Discovery_Schema.cpp Static_Harvester.cpp PgSQL_Static_Harvester.cpp Core changes were minimal because these are leaf modules: include/proxysql.h: drop the AI_Vector_Storage.h include. include/cpp.h: drop AI_Vector_Storage / Discovery_Schema / MySQL_Catalog / Static_Harvester / PgSQL_Static_Harvester includes (replaced with a single carve-out history comment that Step 7 will delete entirely). lib/ProxySQL_Admin.cpp: drop the Discovery_Schema.h include — its only live use was via the now-#if-0'd save/load_mcp_query_rules blocks 4.C left behind for follow-up reference. lib/Makefile: drop AI_Vector_Storage.oo, MySQL_Catalog.oo, Discovery_Schema.oo, Static_Harvester.oo, PgSQL_Static_Harvester.oo from the GenAI _OBJ_CXX list. Test fixture probe update test/tap/tests/unit/Makefile: PROXYSQLGENAI auto-detect probe switched from `AI_Vector_Storage` (now plugin-only) to `MCP_Tool_Handler` (the last GenAI-tier symbol still in core). Step 7 deletes the auto-detect entirely. All three plugin unit tests pass: genai_plugin_load_unit-t : 25/25 genai_plugin_anomaly_unit-t : 6/6 genai_plugin_backend_client : 27/27 Binary size deltas: src/proxysql 159 MB -> 152 MB (-7 MB) plugins/genai/...Plugin.so 47 MB -> 55 MB (+8 MB) Net process RSS roughly neutral, as the design predicted. Carve-out scoreboard: Step 0-2 : infrastructure Step 3 : ~1.1 K LOC (Anomaly_Detector) Step 4.B : ~0.5 K LOC (backend_client) Step 4.C : ~13 K LOC (MCP subsystem + tool handlers + MySQL_FTS) Step 4.F : runtime-reconfig admin SQL routed through plugin commands Step 4.G : MCP admin / config / stats tables registered via plugin ABI Step 5 : ~7 K LOC (GenAI/LLM/AI/RAG) Step 6 : ~5 K LOC (AI_Vector_Storage + Discovery_Schema + MySQL_Catalog + Static_Harvester pair) Pending: Step 7 — delete the PROXYSQLGENAI macro from build flags and any remaining `#ifdef PROXYSQLGENAI` blocks.	2 months ago
Rene Cannao	28cf4ceab8	feat(plugins/genai): Step 5 — move GenAI/LLM/AI surface into the plugin ... Largest carve-out commit since 4.C: moves the GenAI / LLM / AI / RAG surface (about 7K LOC) out of core into plugins/genai/. After this commit, libproxysql.a no longer publishes any of GenAI_Threads_Handler, LLM_Bridge, LLM_Clients, AI_Features_Manager, AI_Tool_Handler, RAG_Tool_Handler, or the legacy GloGATH/GloAI globals. Files moved (git mv preserves history) Headers (include/ -> plugins/genai/include/) GenAI_Thread.h LLM_Bridge.h AI_Features_Manager.h AI_Tool_Handler.h RAG_Tool_Handler.h (promoted from Step 6 — bidirectional dep with AI_Features_Manager forced the early move, same way MySQL_FTS.h got promoted in 4.C) Sources (lib/ -> plugins/genai/src/ or …/tool_handlers/) GenAI_Thread.cpp LLM_Bridge.cpp LLM_Clients.cpp (.cpp without a header; class decls live in LLM_Bridge.h) AI_Features_Manager.cpp AI_Tool_Handler.cpp -> tool_handlers/ RAG_Tool_Handler.cpp -> tool_handlers/ Plugin lifecycle takeover plugins/genai/src/plugin_main.cpp: - Defines plugin-local GloGATH and GloAI alongside the existing GloMCPH (all three are .so-internal aliases for the contexts genai_init constructs). - genai_init now also constructs GenAI_Threads_Handler (calls ->init()) and AI_Features_Manager (calls ->init()), mirroring the pre-Step-5 ProxySQL_Main_init_GenAI_module sequence. - genai_stop tears them down in reverse order. plugins/genai/src/ProxySQL_MCP_Server.cpp: - AI / RAG endpoint registration re-enabled (was #if 0'd in 4.C because the constructors weren't reachable from the plugin then). /mcp/ai and /mcp/rag are once again live when the operator has GloAI configured. plugins/genai/src/Anomaly_Detector.cpp + plugin_main.cpp: - get_query_embedding() now calls through a function-pointer hook (genai_anomaly_embed_fn) that plugin_main installs in genai_init and clears in genai_stop. This indirection lets the test fixture (which compiles Anomaly_Detector.cpp directly into a test binary, without GenAI_Thread.cpp) link cleanly — the pointer stays null, the detector short-circuits to empty embedding, and the existing nullptr-guard in check_embedding_similarity does the rest. Core neutralization src/main.cpp: - Drops the GloGATH / GloAI globals, ProxySQL_Main_init_GenAI_module definition + call, the shutdown branch, and the `if (GloGATH) init_genai_variables()` call. lib/Admin_Bootstrap.cpp / Admin_FlushVariables.cpp / Admin_Handler.cpp / ProxySQL_Admin.cpp / MySQL_Session.cpp / ProxySQL_Admin_Stats.cpp: - Drop the corresponding `#include` and `extern` declarations. - flush_genai_variables___* stubbed (mirrors the 4.C MCP pattern; original bodies preserved in `#if 0` for the upcoming LOAD/SAVE GENAI verbs port). lib/Admin_Handler.cpp: - Loose-prefix accept for `genai-*` in is_valid_global_variable (mirrors the 4.F mcp-* fix), so SET genai-X=Y still reaches main.global_variables; runtime push happens via the plugin's LOAD GENAI VARIABLES TO RUNTIME (registered in a follow-up). include/cpp.h, include/proxysql.h: - Removed the AI_Features_Manager.h / LLM_Bridge.h / GenAI_Thread.h / AI_Tool_Handler.h / RAG_Tool_Handler.h includes that no longer resolve from core's -I path. test/tap/test_helpers/test_globals.cpp: - Drops the matching `#include`s and stub globals. Test fixture test/tap/tests/unit/Makefile: - PROXYSQLGENAI auto-detect probe switched from `GenAI_Thread` (now plugin-only) to `AI_Vector_Storage` (still in core for Step 6). After Step 6 the probe will switch again, and Step 7 deletes the auto-detect entirely. All three plugin unit tests pass: - genai_plugin_load_unit-t : 25/25 - genai_plugin_anomaly_unit-t : 6/6 - genai_plugin_backend_client : 27/27 Carve-out scoreboard: Step 0-2 : infrastructure Step 3 : ~1.1 K LOC (Anomaly_Detector) Step 4.B : ~0.5 K LOC (backend_client) Step 4.C : ~13 K LOC (MCP subsystem) Step 4.F : runtime-reconfig admin SQL routed through plugin commands Step 4.G : MCP admin / config / stats tables registered via plugin ABI Step 5 : ~7 K LOC (GenAI/LLM/AI/RAG) Pending: Step 6 (AI_Vector_Storage, MySQL_Catalog, Discovery_Schema, Static_Harvester, PgSQL_Static_Harvester) and Step 7 (delete the PROXYSQLGENAI macro entirely).	2 months ago
Rene Cannao	5fe286e972	feat(plugins/genai): Step 4.C — move MCP subsystem into the plugin ... Per docs/superpowers/plans/2026-04-19-step4-mcp-subsystem-move.md "4.C / 4.D / 4.E merge" revision. This is the largest single carve-out commit so far: ~13 K LOC physically relocated, plus surgical edits across the core admin SQL surface to neutralize references to the moved code. What moved (git mv preserves history) Tool handlers (lib/<name>.cpp -> plugins/genai/src/tool_handlers/<name>.cpp; include/<name>.h -> plugins/genai/include/<name>.h) Admin_Tool_Handler, Cache_Tool_Handler, Config_Tool_Handler, MySQL_Tool_Handler, Observe_Tool_Handler, Query_Tool_Handler, Stats_Tool_Handler. MCP listener stack (lib/<name>.cpp -> plugins/genai/src/<name>.cpp; include/<name>.h(pp) -> plugins/genai/include/...) MCP_Endpoint, MCP_Thread, ProxySQL_MCP_Server. Promoted from Step 6 (bidirectional dep with MySQL_Tool_Handler): MySQL_FTS. MCP_Tool_Handler base stays in core because AI_Tool_Handler and RAG_Tool_Handler (Steps 5/6 scope) inherit from it; moving the base would force pulling those in too. Plugin lifecycle takeover plugins/genai/include/genai_plugin.h: GenAIPluginContext now holds the MCP_Threads_Handler*. plugins/genai/src/plugin_main.cpp: genai_init constructs the MCP_Threads_Handler and assigns to a plugin-local GloMCPH symbol (the legacy global name, kept for the moved tool handlers that still reference it; the symbol no longer exists in core). genai_stop tears down on shutdown. genai_start is a no-op for now; the MCP listener auto-start is gated on the admin-SQL surface that 4.F restores. plugins/genai/src/ProxySQL_MCP_Server.cpp: AI_Tool_Handler and RAG_Tool_Handler endpoint registration is disabled via stub assignments (the headers stay included so member-pointer types resolve). Steps 5/6 re-enable when those classes move into the plugin. Core neutralization (FIXME stubs awaiting 4.F/4.G) src/main.cpp: GloMCPH global removed; ProxySQL_Main_init_MCP_module removed; shutdown block deleted. lib/ProxySQL_Admin.cpp: init_mcp_variables, load_mcp_server, load_mcp_query_rules_to_runtime, save_mcp_query_rules_from_runtime stubbed to no-ops; original bodies preserved inside `#if 0` for 4.F reference. lib/Admin_FlushVariables.cpp: flush_mcp_variables___database_to_runtime and flush_mcp_variables___runtime_to_database stubbed similarly. lib/ProxySQL_Admin_Stats.cpp: stats___mcp_query_tools_counters, stats___mcp_query_digest, stats___mcp_query_rules stubbed. lib/Admin_Handler.cpp: has_variable("mcp-…") and the load_target_auth_map call site in LOAD MCP PROFILES … stubbed. Result: the MCP listener does NOT start automatically between this commit and 4.F (mcp-* admin variables are not pushed into the running plugin). Documented in the plan's "4.C/4.D/4.E merge" section. src link change: --whole-archive on libproxysql.a src/Makefile: the proxysql binary now links libproxysql.a with -Wl,--whole-archive so all symbols are exported via -Wl,--export-dynamic, allowing the genai plugin .so to dlopen cleanly even when core code paths don't reference moved-but-still- in-core classes (LLM_Bridge, Discovery_Schema, MySQL_Catalog, Static_Harvester, …) directly. test/tap/tests/unit/Makefile gets the same change so unit tests can dlopen the plugin. Without this, ld dead-code-eliminates the unreferenced symbols and dlopen fails with "undefined symbol". Tests passing - genai_plugin_anomaly_unit-t : 6/6 - genai_plugin_load_unit-t : 8/8 (lifecycle init/start/stop + admin table registration) - genai_plugin_backend_client_unit-t : 27/27 Carve-out scoreboard Step 0-2 : 0 (infrastructure only) Step 3 : ~1.1 K (Anomaly_Detector) Step 4.B : ~0.5 K (backend_client helper, no callers yet) Step 4.C : ~13 K (this commit; MCP subsystem) Pending : Steps 4.F, 4.G, 5, 6, 7	2 months ago
Rene Cannao	90e888e1d0	fix(chassis): runtime-view dispatch fires unconditionally on admin ... Doc-accuracy review found that proxysql_refresh_configured_plugin_ runtime_views was unreachable for plugin-registered tables. The call was placed inside if (refresh==true) { // ProxySQL_Admin.cpp:1637 ... if (admin) { // existing core runtime_* refreshes proxysql_refresh_configured_plugin_runtime_views(...) } } `refresh` only gets set to true by hardcoded substring matches against core's own table names (lines 1358-1634: runtime_mysql_users, runtime_mysql_servers, runtime_mysql_query_rules, etc.). None of those substrings match runtime_mysqlx_* (or any other plugin- registered view), so on a bare SELECT * FROM runtime_mysqlx_users the gate was false, my dispatch was skipped, the table was empty (or stale from a previous query that DID match a core substring), and the SELECT returned wrong data. The whole "on-demand projection" mechanism the previous commits documented was broken for the entry case. Issue #5687 / PR #5688. The fix is one-line structurally: hoist the dispatch out of the `if (refresh==true)` block and place it right after the substring- detection section, gated only on `if (admin)`. The chassis dispatcher itself (refresh_runtime_views_for_query in ProxySQL_PluginManager.cpp) already does its own per-view substring match against query_no_space, so a query that touches no registered view is a cheap no-op (one shared lock + N substring scans, N == registered-view count). Calling unconditionally on every admin query is therefore both correct and cheap. Test: new plugin_runtime_views_unit-t (20 ok asserts) drives ProxySQL_PluginManager::register_runtime_view + refresh_runtime_views_for_query directly. Covers: - register_runtime_view rejects null callback / empty name / case-insensitive duplicate. - Per-query dispatch fan-out: only matching callbacks fire, join queries fire all referenced views, unrelated queries fire nothing, case-insensitive match works, backtick-quoted identifiers match. - Whole-identifier boundary: longer-suffix overlap (runtime_ mysqlx_users_extra), left-prefix overlap (stats_runtime_ mysqlx_users), embedded-in-string-literal — none falsely match. Boundary cases (start of string, end of string) do match. This is the test the PR-#5688 review pass identified as the chassis- hook coverage gap. Builds standalone (no fake-plugin loader needed) since it drives the manager directly.	2 months ago
Rene Cannao	f42c3ee1ab	feat(chassis): add register_runtime_view ABI for module-owned state ... Plugins that declare admin-side runtime views of their own in-memory state need a way to register a projection callback the chassis can invoke when admin SELECTs against the registered table. Without this hook, the only way for a plugin to surface its runtime state to admin operators was to *persist* a duplicate copy of the data into admin_db, which violates the separation of duties between Admin (owns configuration tables and views) and the module (owns runtime state). Refs #5687. ABI surface (in include/ProxySQL_Plugin.h): - struct ProxySQL_PluginRuntimeView { table_name, refresh, opaque } - new services.register_runtime_view callback - bumps PROXYSQL_PLUGIN_ABI_VERSION to 3 The new field is appended at the end of ProxySQL_PluginServices, so ABI-2 plugins keep working — they neither set nor read past the previous layout. ABI 3 plugins set abi_version=3 and use the new field. Loader accepts ABI 1, 2, and 3. Plumbing (in lib/ProxySQL_PluginManager.cpp): - register_runtime_view_service() free fn wires through to the manager's runtime_views_ vector - ProxySQL_PluginManager::register_runtime_view() rejects empty table names, null callbacks, and duplicate registrations - sql_references_table_ci() does whole-identifier substring match, so a SELECT on `runtime_mysqlx_users` doesn't fire the refresh for `runtime_mysqlx_users_extra` if both ever coexist - refresh_runtime_views_for_query() iterates registered views and invokes the matching refresh callbacks - proxysql_refresh_configured_plugin_runtime_views() is the Admin-callable wrapper that takes the manager shared lock Admin-handler integration (in lib/ProxySQL_Admin.cpp): - the existing pre-SELECT refresh block (where runtime_mysql_users triggers save_mysql_users_runtime_to_database(true), etc.) now also calls proxysql_refresh_configured_plugin_runtime_views(). This is the SAME refresh point the canonical core tables use, so plugin views are guaranteed to refresh before the admin query actually executes against admindb. Doc block at the bottom of ProxySQL_Plugin.h rewritten to spell out the separation-of-duties contract explicitly, replacing the previous "copy_table guidance" that incorrectly endorsed plugins persisting their runtime state to admin_db tables. This commit is the foundation only — no plugin uses the new API yet. The follow-up commits convert the mysqlx plugin's four entity pairs (users / routes / endpoints / variables) to the canonical pattern.	2 months ago
Wazir Ahmed	8c6e85f343	aurora: Fix monitor race condition and implement autopurge ... - Closes #5546 and #5547 Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	2 months ago
Rene Cannao	04bccec51e	chore(plugin-chassis): tighten gating, drop dead paths, gate forgery setters ... Five clean-up items from the independent review of PR #5651. None of these change behaviour on a normal run; they each fix a concrete way the current code is misleading or unnecessarily exposed. 1) lib/ProxySQL_Admin.cpp: gate the three plugin-DB-handle getters (proxysql_plugin_get_admindb / _configdb / _statsdb) under #ifdef PROXYSQL40. Previously these were defined unconditionally and emitted symbols into v3.0/v3.1 binaries. The chassis is a v4.0 feature; the user explicitly required that v3.x builds carry no plugin-aware code. Wrap in PROXYSQL40 so they are entirely absent from v3.x linkage. ProxySQL_PluginManager.cpp's extern declarations (lines 23-25) are already inside the file-wide PROXYSQL40 gate and so resolve only when the gate is active. 2) lib/ProxySQL_PluginManager.cpp + include/ProxySQL_PluginManager.h: delete the dead `#else /* !PROXYSQL40 */` branches. Both files are wrapped in a top-level `#ifdef PROXYSQL40` covering the entire body. Inner `#ifdef PROXYSQL40 ... #else ... #endif` blocks therefore had unreachable `#else` arms — 30+ lines of "pre-chassis two-phase loader" in the .cpp, plus a redundant declaration of proxysql_load_configured_plugins in the .h. The dead arms read as load-bearing alternative implementations on review and that is exactly the wrong signal. Drop them; the single PROXYSQL40 path is the only one. 3) lib/Admin_Bootstrap.cpp + include/proxysql_admin.h + src/main.cpp: remove ProxySQL_Admin::materialize_plugin_tables(). `Admin::init()` already merges plugin-declared schemas into tables_defs_{admin,config,stats} (~line 944) and runs the DDL via check_and_build_standard_tables() (~line 994), all on the same first-boot/reload code path the core tables use. The follow-up call to GloAdmin->materialize_plugin_tables() in main.cpp ran a second name-dedup pass that found everything already present and produced an empty new-rows set — i.e. the post-init helper was a no-op disguised as load-bearing infrastructure. Delete the helper, the header declaration, and the main.cpp call site, and update the comments in main.cpp + ProxySQL_PluginManager.cpp to point at Admin::init() as the single canonical materialization site. Leave a NOTE in Admin_Bootstrap.cpp at the old call site so anyone re-adding a similar helper sees why the prior one was removed. 4) plugins/mysqlx/include/mysqlx_session.h + plugins/mysqlx/src/mysqlx_session.cpp: gate the two genuine forgery vectors behind MYSQLX_TEST_BUILD. inject_identity_for_test() bypasses the full auth flow — no credential check, no capability negotiation, just sets identity_ to a caller-supplied MysqlxResolvedIdentity. resolve_backend_target_ for_test() drives a private routing helper without an authenticated identity. Both are necessary for unit tests but should not be reachable at all in shipped binaries; an in-process exploit reaching the session can call inject_identity_for_test() to forge an authenticated identity. Wrap them in #ifdef MYSQLX_TEST_BUILD; define -DMYSQLX_TEST_BUILD in the test Makefile only. The remaining target_*_for_test() getters are read-only state observers and are left unconditional — they cannot mutate the session and a debugger could observe the same state regardless. 5) test/tap/tests/unit/Makefile: define -DMYSQLX_TEST_BUILD on every unit test compile line via OPT. This is the test-only knob that re-enables the gated forgery methods so unit tests still compile. plugins/mysqlx/Makefile does NOT define this macro, so the production .so does not compile in the entry points. Verified locally: - plugins/mysqlx/ProxySQL_MySQLX_Plugin.so builds clean with PROXYSQL40=1 PROXYSQL31=1 PROXYSQLFFTO=1 PROXYSQLTSDB=1 PROXYSQLGENAI=1 (no MYSQLX_TEST_BUILD). - test/tap/tests/unit/mysqlx_robustness_unit-t builds and runs: 74/74 assertions pass, including the ones that exercise inject_identity_for_test (visible only because the test Makefile defines MYSQLX_TEST_BUILD).	2 months ago
Rene Cannao	3ba92815f8	fix(plugin-chassis): make chassis fully invisible in v3.x builds ... Review feedback identified that the chassis was leaking into the v3.0 stable build via unfenced references in core code: * Admin_Bootstrap.cpp had `if (auto* m = proxysql_get_plugin_manager())` and a full materialize_plugin_tables() implementation outside any PROXYSQL40 fence * proxysql_admin.h declared materialize_plugin_tables() * proxysql_glovars.hpp held GloVars.plugin_modules and proxysql_load_plugin_modules_from_config() * ProxySQL_GloVars.cpp had three unfenced .clear()/use sites * ProxySQL_Admin.cpp::dispatch_plugin_admin_command (template + two instantiations) was unfenced * src/main.cpp's GloPluginManager, LoadConfiguredPlugins, InitConfiguredPlugins, StartConfiguredPlugins, StopConfiguredPlugins, UnloadPlugins teardown, the four-phase orchestration block, and the proxysql_load_plugin_modules_from_config call were all unfenced Separately, Admin_Handler.cpp held 16 hardcoded MYSQLX alias vectors (LOAD_MYSQLX_USERS_FROM_MEMORY, etc.) and a 16-deep if-ladder that called resolve_admin_alias_to_canonical(). This embedded plugin-specific knowledge in core code -- core should know nothing about any specific plugin's commands. The agreed architecture: v3.0/v3.1 builds have NO plugin loader at all; the entire plugin chassis (including its public manager surface, its loader, and its dispatch helpers) only exists under PROXYSQL40. v3.0 core code therefore holds zero plugin-related code paths. Changes: * include/ProxySQL_Plugin.h, include/ProxySQL_PluginManager.h, lib/ProxySQL_PluginManager.cpp: wrap entire file body in `#ifdef PROXYSQL40 ... #endif`. Including the header from a v3.x translation unit yields nothing; the .cpp compiles to an empty object file. Removed redundant inner PROXYSQL40 fences inside the now-empty v3.x body of Plugin.h (kept the few that are inside the file-wide outer for clarity). PROXYSQL_PLUGIN_ABI_VERSION is now unconditionally 2 (the macro is no longer reachable from a v3.x compile, so the PROXYSQL40-vs-1 split is moot). * include/proxysql_glovars.hpp, lib/ProxySQL_GloVars.cpp: gate GloVars.plugin_modules field, all .clear() sites, the proxysql_load_plugin_modules_from_config declaration + definition. * include/proxysql_admin.h: gate materialize_plugin_tables and dispatch_plugin_admin_command<S> declarations. * lib/Admin_Bootstrap.cpp: gate `#include "ProxySQL_PluginManager.h"`, the `if (auto* m = proxysql_get_plugin_manager())` merge block, and the materialize_plugin_tables definition. * lib/Admin_Handler.cpp: delete the 16 LOAD_/SAVE_MYSQLX_* const std::vector<std::string> declarations entirely. Delete the helper resolve_admin_alias_to_canonical(). Delete the 16-deep if-ladder + the !PROXYSQL40 fallback dispatch branch. The remaining plugin dispatch is one generic call to proxysql_resolve_configured_plugin_admin_alias() inside an #ifdef PROXYSQL40 block; v3.x compiles it out entirely. Core now holds no plugin-specific knowledge of any kind. * lib/ProxySQL_Admin.cpp: gate dispatch_plugin_admin_command template body + both explicit instantiations. * src/main.cpp: gate `#include "ProxySQL_PluginManager.h"`, GloPluginManager declaration, all four wrapper functions (LoadConfiguredPlugins / InitConfiguredPlugins / StartConfiguredPlugins / StopConfiguredPlugins), the StopConfiguredPlugins call inside UnloadPlugins, the four-phase orchestration in ProxySQL_Main_init_phase2___not_started, the GloVars.plugin_modules.clear() and the proxysql_load_plugin_modules_from_config call in ProxySQL_Main_process_global_variables. v3.x builds reduce that whole block to a single ProxySQL_Main_init_Admin_module call. * plugins/mysqlx/src/mysqlx_plugin.cpp, plugins/mysqlx/src/mysqlx_admin_schema.cpp: drop the !PROXYSQL40 fallback paths. The plugin only loads under PROXYSQL40 now (no loader exists in v3.x), so the legacy combined-init code path and the 16 fallback register_command() calls were dead code. The descriptor unconditionally wires register_schemas; mysqlx_init holds only context setup. * Makefile (top-level): the four build_src_* recipes that recurse into plugins/mysqlx are wrapped in `$(if $(filter 1,$(PROXYSQL40)),...)`. v3.x builds emit "[skip] mysqlx plugin (PROXYSQL40 not set)" instead of building the .so. Clean targets stay unconditional. * test/tap/tests/unit/mysqlx_admin_alias_resolution_unit-t.cpp: delete the test (it covered resolve_admin_alias_to_canonical, which is itself deleted). Remove from unit Makefile UNIT_TESTS list. Verification: * v3.0 stable build (`make`, no flags): - Produces 147 MB src/proxysql binary - `nm src/proxysql | grep -cE 'invoke_register_schemas_phase|GloPluginManager|materialize_plugin_tables'` -> 0 - mysqlx plugin .so is correctly skipped: "[skip] mysqlx plugin (PROXYSQL40 not set)" * PROXYSQLGENAI=1 build (cascades PROXYSQL40/31/FFTO/TSDB): - Produces 200 MB src/proxysql binary with chassis symbols - mysqlx plugin .so built (9.2 MB), contains mysqlx_register_schemas * All 5 plugin unit tests pass under PROXYSQL40: plugin_config_unit-t 48/48 plugin_dispatch_unit-t 52/52 plugin_lifecycle_unit-t 26/26 plugin_query_hook_unit-t 46/46 plugin_prometheus_unit-t 10/10	2 months ago
Rene Cannao	5c1549dcc6	merge: resolve conflicts with v3.0 branch ... Merge origin/v3.0 into ProtocolX. One conflict in test/tap/tests/unit/Makefile resolved by keeping both mysqlx unit tests (ProtocolX) and pgsql_servers_ssl_params unit test + GENAI vec.o link (v3.0).	3 months ago
René Cannaò	669ab91494	Merge pull request #5594 from sysown/v3.0-lint ... lint + static analysis: clang-tidy and cppcheck fixes	3 months ago
Rene Cannao	f1826cf206	lint: fix reserved identifier in member variables ... Rename __active -> active_, __frontend -> frontend_, __backend -> backend_ in authentication and hostgroups classes to comply with C++ reserved identifier rules. Includes updates to: - MySQL_Authentication.hpp - ClickHouse_Authentication.hpp - PgSQL_Authentication.h - MySQL_HostGroups_Manager.h - Base_HostGroups_Manager.h - Corresponding .cpp files	3 months ago
Rene Cannao	ac0f9304b0	fix: address code review findings for pgsql cluster sync ... 1. Remove dead declarations and implementations: PullOperationConfig, pull_from_peer_unified, fetch_query_with_metrics, compute_*_checksum, ScopedChar* wrappers, and all safe_* memory helpers (-690 lines) 2. Document mutex sharing: pgsql pull functions intentionally reuse MySQL mutexes since pulls are sequential in the monitoring thread 3. Fix variable shadowing: rename local 'error' to 'exec_error' in Admin_Handler.cpp pgsql_servers_v2 fallback to avoid shadowing the outer error variable 4. Fix misleading debug message in process_component_checksum: the else branch fires when peer checksum is unchanged, not different 5. Fix indentation in ProxySQL_Admin.cpp (3 lines with double-tab) and Admin_FlushVariables.cpp (NULL-guard block with extra indent)	3 months ago
Rene Cannao	80ff2242e6	merge: resolve v3.0 conflict and add pgsql_servers_ssl_params cluster sync ... v3.0 added pgsql_servers_ssl_params table. Integrate into pgsql cluster sync path: - Add CLUSTER_QUERY_PGSQL_SERVERS_SSL_PARAMS to Admin_Handler interception block alongside other pgsql cluster queries - Add ssl_params fetch query to pull_pgsql_servers_v2_from_peer - Add update_pgsql_servers_ssl_params() for applying synced data - Update convert_pgsql_servers_resultsets to handle 5 result sets - Include ssl_params in v2 checksum computation (4 tables)	3 months ago
René Cannaò	93eabfc7bc	fix: address code review issues in PostgreSQL cluster sync ... - Fix fetch_and_store incrementing success counter on failure - Fix sqlite3_stmt leak in pull_pgsql_variables_from_peer (use RAII) - Convert get_peer_to_sync_pgsql_query_rules and _servers_v2 to unified framework delegates (fixes atomic data race on .load()) - Merge dual scan loops in set_checksums to respect enabled_check - Remove spurious "deprecated" label from new checksum_pgsql_variables - Replace sprintf with snprintf in pgsql diffs_before_sync getters - Fix indentation on cluster_pgsql_variables blocks - Fix TAP plan mismatch: replace MYSQL_QUERY with inline mysql_query - Fix trivially-true table accessibility assertions - Add mysql_close on failed connect in test - Move test_cluster_sync_pgsql-t to legacy-g5 group	3 months ago
Rahim Kanji	e68f8bb74a	refactor: simplify pgsql_servers_ssl_params schema and pre-parse TLS range ... - Drop ssl_capath and ssl_cipher columns (no libpq equivalents) - Pre-parse ssl_protocol_version_range into ssl_min/max_protocol_version at construction time instead of on every backend connection - Fix Servers_SslParams 3-arg delegating constructor - Update tests, docs, and all consumers (Connection, Monitor, Kill, Admin)	3 months ago
René Cannaò	8a61232517	fix: decouple MySQL and PostgreSQL checksum/sync controls ... Two bugs in the PostgreSQL cluster sync admin variables: 1. Cross-protocol coupling (critical): Disabling a MySQL checksum flag (e.g. SET admin-checksum_mysql_query_rules='false') incorrectly zeroed out the corresponding PostgreSQL cluster sync threshold (cluster_pgsql_query_rules_diffs_before_sync). An operator disabling MySQL checksums would unexpectedly halt PostgreSQL cluster sync. Fix: MySQL checksum setters now only affect MySQL sync thresholds. The warning messages no longer mention pgsql side effects. 2. Single-gate for all pgsql modules (operational limitation): All four PostgreSQL cluster sync thresholds (cluster_pgsql_{query_rules,servers,users,variables}_diffs_before_sync) were gated by a single flag (checksum_pgsql_variables). This meant you could not independently control checksum computation per-module. By contrast, MySQL has per-module flags (checksum_mysql_query_rules, checksum_mysql_servers, checksum_mysql_users, checksum_mysql_variables). Fix: Add three new per-module admin variables: - admin-checksum_pgsql_query_rules - admin-checksum_pgsql_servers - admin-checksum_pgsql_users Each gates only its corresponding cluster_*_diffs_before_sync setter, mirroring the MySQL pattern. The existing admin-checksum_pgsql_variables now only gates cluster_pgsql_variables_diffs_before_sync. Disabling admin-checksum_pgsql_variables no longer cascades to zero out query_rules/servers/users sync thresholds. Files changed: - include/proxysql_admin.h: Add checksum_pgsql_query_rules, checksum_pgsql_servers, checksum_pgsql_users to checksum_variables struct. - lib/ProxySQL_Admin.cpp: Add variable names, constructor defaults, get/set handlers for new flags. Update diffs_before_sync guards to use per-module flags. Remove pgsql zeroing from MySQL checksum setters. Fix checksum_pgsql_variables setter to only affect pgsql variables sync threshold.	3 months ago
Rene Cannao	47758a468d	merge: resolve groups.json conflict with v3.0 ... Take v3.0's groups.json as base (legacy-gN naming, mysql84-gN groups, new test entries) and add test_cluster_sync_pgsql-t from this branch. Drop duplicate pgsql_admin_metacmds-t (already exists as pgsql-admin_metacmds-t in v3.0).	3 months ago
Rene Cannao	9b87260b70	fix: harden plugin init failures and command dispatch	3 months ago
Rene Cannao	fd5f029474	fix: expose plugin db handles during startup	3 months ago
Rene Cannao	cd15afdd1e	feat: support plugin-owned admin tables and commands	3 months ago
Rahim Kanji	93d0ddf477	fix: add pgsql_servers_ssl_params to tablenames vector and fix monitor fallback ... - Add pgsql_servers_ssl_params to pgsql_servers_tablenames so SAVE/LOAD PGSQL SERVERS TO/FROM DISK correctly persists and restores SSL params. - Fix monitor SSL fallback to use per-server params directly (empty fields omitted) instead of per-field fallback to globals, matching the backend connection behavior.	3 months ago
Rahim Kanji	516fa468ba	feat: add admin load/save/dump for pgsql_servers_ssl_params	3 months ago
René Cannaò	efedd93c92	Rename stats_global to stats_proxysql_global to avoid strstr substring collision ... stats_global is a substring of stats_mysql_global and stats_pgsql_global, causing the strstr-based dispatch to match incorrectly. Exclusion guards don't work either because a query referencing both tables would suppress the stats_proxysql_global refresh. Renaming to stats_proxysql_global follows the existing naming convention (stats_mysql_global, stats_pgsql_global) and eliminates the substring collision entirely.	3 months ago
René Cannaò	9850d9daeb	Fix stats_global substring collision and reload failure tracking ... 1. stats_global strstr match: add exclusion guards for stats_mysql_global and stats_pgsql_global to prevent spurious stats___global() calls and unnecessary ssl_mutex contention on every MySQL/PgSQL global stats query. 2. TLS reload failure tracking: set tls_last_load_ok=false and update tls_load_count/tls_last_load_timestamp on non-bootstrap reload failures. Previously, a failed PROXYSQL RELOAD TLS would leave stale SUCCESS status in stats_global.	3 months ago
copilot-swe-agent[bot]	e274862857	feat: introduce stats_global table and move TLS metrics out of stats_mysql_global; add TAP tests ... - Add STATS_SQLITE_TABLE_GLOBAL definition (stats_global table for non-MySQL/PgSQL metrics) - Register stats_global in Admin_Bootstrap.cpp - Add stats___global() implementation and declaration; document with Doxygen comment - Remove TLS_* variables from stats___mysql_global() - they were misplaced there - Move all TLS tracking metrics to stats___global() under ssl_mutex - Wire up stats_global query detection and refresh in GenericRefreshStatistics() - Add TAP test test_tls_stats-t.cpp: - Verifies stats_global contains all 6 TLS tracking variables - Checks value ranges and validity of each TLS variable - Verifies stats_tls_certificates has 2 rows (server + ca) with correct fields - Verifies TLS_Load_Count increments and TLS_Last_Load_Timestamp increases after PROXYSQL RELOAD TLS - Confirms TLS variables are absent from stats_mysql_global Co-authored-by: renecannao <3645227+renecannao@users.noreply.github.com>	3 months ago
copilot-swe-agent[bot]	4f88740b20	feat: add stats_tls_certificates table and TLS tracking metrics ... - Add STATS_SQLITE_TABLE_TLS_CERTIFICATES definition with cert_type, file_path, subject_cn, issuer_cn, serial_number, not_before, not_after, days_until_expiry, sha256_fingerprint, and loaded_at columns - Add TLS tracking fields to GloVars.global: tls_load_count, tls_last_load_timestamp, tls_last_load_ok, tls_cert_file, tls_ca_file, tls_key_file - Update ProxySQL_create_or_load_TLS() to populate tracking fields on success - Register stats_tls_certificates table in Admin_Bootstrap - Add stats___tls_certificates() function that reads certs from disk at query time, extracts metadata (CN, serial, validity dates, SHA-256 fingerprint, days_until_expiry) and populates the table - Add TLS variables to stats___mysql_global(): TLS_Load_Count, TLS_Last_Load_Timestamp, TLS_Last_Load_Result, TLS_Server_Cert_File, TLS_CA_Cert_File, TLS_Key_File - Wire up query detection and refresh in GenericRefreshStatistics() Co-authored-by: renecannao <3645227+renecannao@users.noreply.github.com>	3 months ago
Rene Cannao	859c89b30e	cluster: harden PostgreSQL cluster sync parity paths after real two-node validation ... This commit turns the current working branch follow-up into one coherent changeset after reviewing the PostgreSQL cluster sync implementation against real multi-node behavior instead of relying only on checksum presence and basic TAP accessibility checks. The main implementation change is that PostgreSQL cluster sync now updates the replica admin-memory tables in the places where the previous branch state was still weaker than the established MySQL behavior. In the PostgreSQL servers path, the replica now rewrites pgsql_servers, pgsql_replication_hostgroups, and pgsql_hostgroup_attributes before promoting the fetched state into runtime. That closes the most important parity gap that showed up during replica-side validation, because the cluster sync operation no longer leaves runtime state correct while replica admin-memory tables lag behind. In the PostgreSQL query rules path, the fetched query-rules and fast-routing resultsets are now hashed using the same converted representation that runtime loading consumes, then passed directly into load_pgsql_query_rules_to_runtime(), and finally written back from runtime into the replica admin-memory tables. In the PostgreSQL users path, the branch now relies on PostgreSQL-specific runtime checksum logic and persists accepted rows into the replica pgsql_users admin-memory table. This commit also fixes a real correctness bug that was discovered only after running a true two-node manual validation flow. The refactored module table in ProxySQL_Cluster::set_checksums() passed the full module names "pgsql_variables" and "ldap_variables" into pull_global_variables_from_peer(), but that shared helper only accepts the short dispatch keys "pgsql" and "ldap". In practice, changing a PostgreSQL variable on the primary caused the replica to abort on an assertion as soon as the cluster monitor attempted the variable pull. This commit restores the correct short dispatch keys so the PostgreSQL and LDAP variable sync paths actually execute instead of crashing. On the TAP side, the PostgreSQL cluster sync test is strengthened so it checks what the implementation is supposed to guarantee rather than what the earlier test happened to probe. The server sync test now mutates the real pgsql_servers admin table, then validates replica runtime_pgsql_servers, replica pgsql_servers, and disk.pgsql_servers when save_to_disk is enabled. The users sync test now validates runtime, main, and optional disk state on the replica. The query-rules sync test now validates both query rules and fast-routing rows across runtime, main, and optional disk persistence. The polling helpers were also hardened to support fresh replica admin sessions, because during local validation a long-lived admin session did not reliably observe synced rows even though fresh sessions and the manual harness did. The test now asserts the externally observable replica state seen by new admin clients. The documentation is updated to reflect the branch as it actually exists after this review pass. The PostgreSQL cluster sync status document now records the remaining non-CI TODO, the module-by-module behavior, the real two-node test plan, the outcomes for save_to_disk=false and save_to_disk=true, the variable sync dispatcher crash that was found and fixed locally, and the current MySQL parity assessment. This makes the branch easier to continue without having to reconstruct the reasoning from terminal logs or partial memory. Validation performed for this commit: - git diff --check - make build_src -j2 - PostgreSQL cluster-sync TAP two-node validation with save_to_disk=false - PostgreSQL cluster-sync TAP two-node validation with save_to_disk=true - manual two-node validation of pgsql_servers, pgsql_replication_hostgroups, pgsql_hostgroup_attributes, and a real PostgreSQL variable with save_to_disk=false - manual two-node validation of the same paths with save_to_disk=true Residual caveat intentionally documented but not treated as a blocker in this commit: - a long-lived replica admin session did not reliably observe synced PostgreSQL rows during polling, while fresh admin sessions and the manual validation harness did observe the expected state - the underlying cluster-sync apply path now validates correctly in real two-node runs, but the admin-session visibility quirk should still be understood better before calling the branch completely finished	3 months ago
Rene Cannao	c2f678a9de	feat: implement release tiers and feature guards for v3.0, v3.1, and v4.0 ... This commit introduces a structured release tier system controlled by environment variables, allowing three distinct ProxySQL versions to be built from the same codebase while maintaining a clear upgrade path. Key Changes: - Tier 1 (v3.0.x Stable): Default build including bug fixes and core enhancements. - Tier 2 (v3.1.x Innovative): Enabled via PROXYSQL31=1. Adds FFTO and TSDB. - Tier 3 (v4.0.x AI/MCP): Enabled via PROXYSQLGENAI=1. Adds the AI stack. Implementation Details: - Makefile: Implemented dynamic versioning logic that increments the minor version for v3.1 and the major version for v4.0. Added tier documentation. - Feature Guards: Introduced PROXYSQLFFTO and PROXYSQLTSDB guards across ProxySQL_Admin, ProxySQL_Statistics, REST API, and HTTP server. - FFTO: Guarded Fast Forward Traffic Observer in MySQL and PgSQL sessions, including thread-local variables and configuration parameters. - TSDB: Guarded the entire Time Series Database subsystem, including SQLite table definitions, background loops, and REST API endpoints. - Build Fixes: * Fixed 'proxy_debug_func' linkage by adding extern "C" and moving the definition outside of DEBUG guards to ensure non-debug builds link correctly. * Corrected misplaced/nested DEBUG and PROXYSQLFFTO guards. * Verified build stability for all tiers (unset, PROXYSQL31=1, PROXYSQLGENAI=1).	4 months ago
Wazir Ahmed	5421a382ab	fix: Load TSDB variables from disk during startup ... - TSDB variables stored on disk were not being loaded into runtime during startup. Add the missing `flush_tsdb_variables___database_to_runtime()` and invoke it during init. - Inline the existing load/save wrappers in the header as thin delegations to the underlying flush functions Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	4 months ago
Rene Cannao	5c7e616f9b	PR5297: resolve remaining actionable review findings for PGSQL checksum sync and TAP assertions ... This commit addresses the review findings that are still reproducible on top of fix/postgresql-cluster-sync_2 after the v3.0 merge, while leaving already-resolved or no-longer-applicable reports untouched. Changes included: - Admin_FlushVariables: restore active PGSQL checksum generation path in flush_pgsql_variables___database_to_runtime(). - Remove the stale commented-out checksum block. - After releasing GloPTH write lock, compute runtime PGSQL vars snapshot and call flush_GENERIC_variables__checksum__database_to_runtime("pgsql", checksum, epoch) under checksum mutex, matching the mysql/admin pattern. - ProxySQL_Admin::set_variable(checksum_pgsql_variables): when disabling, reset all PGSQL *_diffs_before_sync thresholds (variables/query_rules/servers/users) in both admin variables and GloProxyCluster state, and update warning text accordingly. - TAP test hardening in test_cluster_sync_pgsql-t: - Ensure each checksum-loop failure path emits an explicit failing ok(false, ...) before continue, preventing TAP plan/accounting mismatches. - Simplify pgsql_servers_v2 test tuple shape by removing the unused int field and using the comment field consistently. Validation performed: - make -C lib -j4 (pass) - make -C test/tap/tests test_cluster_sync_pgsql-t -j4 (pass) Note: - The previously reported double wrunlock() in flush_pgsql_variables__from_memory_to_disk() is not present in current HEAD; function has a single wrlock/wrunlock pair.	4 months ago
Rene Cannao	6dbd0e9cb1	Merge branch 'v3.0' into fix/postgresql-cluster-sync_2 ... Resolve merge conflicts in: - lib/ProxySQL_Cluster.cpp - test/tap/tap/Makefile Conflict resolution details: - Kept PostgreSQL/MySQL cluster-sync updates from this branch (SQLQueries delete constants and pgsql sync flow changes) while adopting upstream SQLite prepare_v2 RAII usage where introduced in v3.0 conflict regions. - Combined TAP archive fixes from this branch (archive recreation via rm -f to prevent stale objects) with upstream v3.0 mcp_client.o additions to libtap archives. Validation performed after conflict resolution: - make -C test/tap/tap debug -j4 (success) - make -C lib -j4 (fails in current workspace due missing sqlite-vec.h/vec.o dependency in broader v3.0 content, not conflict markers) - make -C test/tap/tests admin-listen_on_unix-t -j4 (fails in current workspace due missing deps/sqlite3/sqlite3/vec.o, same dependency issue) No conflict markers remain.	4 months ago
Rene Cannao	4cdf61cbae	cluster: complete PostgreSQL servers sync flow and fix TAP libtap stale-archive linker failures ... This commit finalizes the remaining PostgreSQL cluster-sync implementation work and includes a targeted TAP build fix that addresses the unresolved perform_simple_get() linker error seen while building debug TAP tests. PostgreSQL cluster sync changes: - Add conversion support to transform fetched pgsql resultsets into SQLite3_result objects suitable for admin/runtime loaders. - Rework runtime pgsql server pull path to: - use peer-selected checksums, - fetch runtime rows from the selected peer, - commit runtime state through PgHGM in only-runtime mode, - persist runtime state and keep locking semantics around pgsql server updates. - Rework pgsql_servers_v2 pull path to fetch and validate all related datasets together: - pgsql_servers_v2, - pgsql_replication_hostgroups, - pgsql_hostgroup_attributes, - runtime_pgsql_servers (when applicable). - Ensure checksum validation includes all fetched tables and rejects partial/inconsistent peer payloads. - Load the actual incoming converted datasets into runtime instead of placeholder/empty local resultsets. Test updates: - Extend test_cluster_sync_pgsql-t with additional verification for synchronized pgsql data and optional replica/runtime sync checks. - Adjust TAP planning and resultset handling so all executed queries are validated and cleaned up consistently. TAP build/linker fix: - Update test/tap/tap/Makefile archive targets (libtap_mariadb.a/libtap_mysql57.a/libtap_mysql8.a) to remove existing archives before recreating them. - Add Makefile as a prerequisite for those archive targets so this behavior is applied on rebuild after pulling. - This prevents stale archive members (for example old noise_utils*.o objects) from being retained and reintroduced into libtap.so, which was causing undefined references to an obsolete 4-argument perform_simple_get() symbol. Validation performed in this branch after these changes: - make -C lib -j4 - make -C src -j4 - make -C test/tap/tests test_cluster_sync_pgsql-t -j4 - make -C test/tap/tests admin-listen_on_unix-t -j4	4 months ago
Rene Cannao	e2ebc0c449	Merge branch 'v4.0-mcp-stats2' into v3.0-unified-large-pr	4 months ago
René Cannaò	9c6945fae8	Merge branch 'v3.0' into v3.0-tsdb-feature ... Signed-off-by: René Cannaò <rene@proxysql.com>	4 months ago
Rene Cannao	70b61a86e3	MCP stats: implement in-memory show_queries Top-K path ... Replace stats table reads in show_queries with Query Processor in-memory aggregation to avoid stale/empty admin-db snapshots.\n\nIntroduce Top-K digest filtering primitives and wire them into Stats_Tool_Handler so filtering/sorting/limit are executed against live digest memory. Add runtime MCP variable mcp-stats_show_queries_max_rows with validation, defaulting, and configuration plumbing, plus cap metadata in the tool response.\n\nEnforce configurable hard cap semantics while preserving deterministic ordering and pagination behavior for MCP clients. Add internal DEBUG validator path (PROXYSQLTEST 56) in admin tests to stress generated digest data and validate the Top-K/filter pipeline against in-memory state.\n\nKeep the validator callable from startup/testing workflows without depending on MCP linkage. Add TAP coverage in test/tap/tests/mcp_show_queries_topk-t.cpp for:\n- MCP endpoint setup and reachability\n- digest generation + optional DEBUG internal validator\n- payload validation and cap metadata assertions\n- descending count order validation\n- match_digest_text filter verification\n\nMake payload parsing backward-compatible in TAP (direct tool result object and legacy wrapped success/result payloads).	4 months ago
René Cannaò	4a6809b60f	Merge branch 'v3.0' into v3.0-tsdb-feature ... Signed-off-by: René Cannaò <rene@proxysql.com>	4 months ago
Rene Cannao	ef397f9f1d	Fix TSDB variable patterns and backend monitor runtime source	4 months ago
Rene Cannao	b4e85f1799	Expose PostgreSQL eventslog metrics in stats and Prometheus	4 months ago
Rene Cannao	b100235fbd	Add PGSQL eventslog dump commands and periodic disk sync scheduling	4 months ago
Rene Cannao	2a46c239be	Add PostgreSQL advanced eventslog schema and variable scaffolding	4 months ago
Rene Cannao	700459f67b	tsdb: address AI code review feedback, fix memory safety and logic bugs ... Summary of changes: - Fixed allocator mismatches (sqlite3_free vs free) across multiple files. - Converted manual SQL construction to prepared statements for TSDB variables. - Resolved memory leaks in statistics and monitoring loops. - Fixed critical 'LOAD TSDB VARIABLES FROM CONFIG' command mismatch. - Improved command parsing robustness using string searching. - Standardized SQL LIKE patterns with escaped wildcards (\%). - Switched to data-driven variable management in ProxySQL_Statistics. - Improved web dashboard security with URL encoding and const correctness. - Clarified documentation regarding configuration lifecycle and schema names. - Updated TAP tests for better reliability and correct persistence verification.	4 months ago
Rene Cannao	b8c6b4f573	tsdb: address code review comments and improve stability ... - Refactored variable flushing to use prepared statements for safety. - Replaced brittle command parsing with robust string searching. - Extracted TSDB dashboard HTML/JS to separate source file for maintainability. - Added NULL guards for GloProxyStats in Admin and REST API. - Implemented batched metric insertions using transactions for performance. - Refactored backend probing to use non-blocking connect() with timeout. - Fixed potential integer overflows in timer calculations. - Improved error handling and fixed memory leaks in statistics reporting. - Updated TAP tests to verify UI and REST API functionality.	4 months ago
Rene Cannao	6a788e48c4	mcp: make /mcp/query self-healing when targets/backends appear after startup ... Problem addressed: - MCP query endpoint could stay unusable with 'Tool Handler not initialized' after restart/reload flows. - This was triggered when Query_Tool_Handler could not build an executable pool at init time (for example profiles loaded before ONLINE servers), leaving runtime commands like 'LOAD MCP QUERY RULES FROM MEMORY' blocked behind a NULL query tool handler. - Users had to manually toggle MCP enablement to recover, which is the opposite of expected self-healing behavior. What this commit changes: 1) Query_Tool_Handler pool init is now resilient and idempotent - init_connection_pool() now starts with close() so reinitialization fully resets stale mysql/pgsql pool state before rebuilding from runtime profile + server tables. - If no executable targets are available, init_connection_pool() now returns success with a warning instead of hard failure. This allows the query tool handler (and /mcp/query endpoint) to stay initialized even before backends are ready. 2) Lazy auto-rebuild on first query usage - get_connection() and get_pgsql_connection() now: - refresh target registry before resolution, - attempt to use an existing pooled connection, - if unavailable, trigger a full pool rebuild (init_connection_pool()) and retry once. - This provides automatic recovery when hostgroups/servers/profiles are loaded or changed after MCP startup, without requiring manual MCP disable/enable. 3) Admin runtime rule load path attempts MCP recovery - load_mcp_query_rules_to_runtime() now detects NULL query tool handler and calls load_mcp_server() once before failing. - This turns a hard, immediate admin error into a self-recovery attempt consistent with MCP runtime semantics. Behavioral impact: - /mcp/query endpoint remains online even when there are temporarily zero executable targets. - As soon as compatible runtime targets/backends exist, run_sql_readonly/explain_sql can recover automatically on demand. - LOAD MCP QUERY RULES TO/FROM RUNTIME no longer fails immediately on first NULL handler condition; it retries after MCP server recovery. Validation performed: - Recompiled modified objects successfully: - lib/obj/Query_Tool_Handler.oo - lib/obj/ProxySQL_Admin.oo - Full TAP runtime test execution is not possible in this sandbox due blocked local TCP socket creation; validation should be run in the normal test environment where ProxySQL/MySQL/PGSQL are reachable.	4 months ago