7 Commits (feature/pgsql-native-backend-protocol)

Author	SHA1	Message	Date
Rene Cannao	b8cfa32c63	fix: address all critical and important code review findings (issue #5729) ... Critical fixes: - Move db_kind to end of ProxySQL_PluginRuntimeView (tail-append = free ABI-3 compat) - Add mutex to protect Anomaly_Detector::user_statistics from concurrent session threads - Replace system(mkdir -p) with pure mkdir() loop (command injection fix) - Replace 77 std::stoi/stoll with non-throwing parse_int_or_zero/parse_ll_or_zero - Fix proxysql.cnf filename: Mysqlx -> MySQLX - Add --whole-archive to non-ClickHouse link path (plugin symbol resolution) Important fixes: - Add default: case with warning to db_kind switch in ProxySQL_PluginManager.cpp - Add config_db dispatch and null-handle tests to plugin_runtime_views_unit-t (6 new tests) - Update CLAUDE.md with sqlite-vec PROXYSQLGENAI exception - Remove stale anomaly_detection-t Makefile target - Replace fixed 1024-byte buffer with dynamic sqlite3_mprintf in Anomaly_Detector Minor: update ABI.md, PLUGIN_API.md, stale comments, unit test plan counts	2 months ago
Rene Cannao	29ee30daf9	docs(plugin-chassis): address PR-#5690 review feedback ... CodeRabbit + Gemini review pass surfaced four small doc-precision items, all legitimate. None were code bugs. # CodeRabbit (markdownlint nit) - doc/PLUGIN_API.md:455: fenced code block at "DISK ↔ MEMORY ↔ RUNTIME" lacked a language specifier. Added `text` to satisfy MD040 / fenced-code-language. # Gemini (3x same root: clarify the matching algorithm) The chassis dispatch matcher is case-insensitive whole-identifier substring match, but three docs described it loosely. Plugin authors deciding what table name to register need to know that runtime_X does not match runtime_X_extra or stats_runtime_X. - doc/PLUGIN_API.md:289 ("any admin SELECT against it") -> spell out the whole-identifier rule with the longer-prefix / longer-suffix rejection examples. - doc/plugin-chassis/ABI.md:156 ("appears in the SQL") -> "is referenced as a whole identifier in the SQL query (case-insensitive; identifier-aware, so runtime_X_extra or stats_runtime_X do not match runtime_X)". - doc/plugin-chassis/FILE_CHANGES.md:102 ("substring match") -> "case-insensitive whole-identifier substring match", matching the precise wording already used in section B for the sql_references_table_ci helper. Cross-section consistency. No source code touched. No new content sections, only existing prose tightened.	2 months ago
Rene Cannao	64e7334297	docs(plugin-chassis): align FILE_CHANGES/ABI/REVIEW_GUIDE/PLUGIN_API with PR #5688 ... The four chassis docs still described the pre-fix architecture: - ABI version 2 (now 3 — register_runtime_view appended at services tail) - MysqlxConfigStore::load_from_runtime (removed; replaced by per-entity install/save/project triplets) - copy_table / copy_to_runtime as the LOAD/SAVE shovels (deleted) - Listener reconciler reading runtime_mysqlx_routes (now reads MysqlxConfigStore::snapshot_active_routes) - "Three-tier" disk/memory/runtime model with the empty-source-sync invariant applied uniformly (the runtime tier was wrong; runtime_<X> is now an admin-side projection of module state, not a tier; the invariant only applies to disk-tier copies) - Plugin-context layout listing a non-existent MysqlxStatsStore Updates per file (no source code touched): - FILE_CHANGES.md §A: PROXYSQL_PLUGIN_ABI_VERSION 2 → 3, services list gains register_runtime_view, ProxySQL_PluginRuntimeView struct documented, separation-of-duties contract spelled out. §B: PluginManager methods list gains register_runtime_view, refresh_runtime_views_for_query; service trampolines list gains register_runtime_view_service; sql_references_table_ci whole-identifier match described; services_phase_b_ wiring of register_runtime_view explained. §C: new pre-SELECT runtime-view dispatch site in GenericRefreshStatistics described — gated on if (admin), OUTSIDE the if (refresh==true) block, so any admin-port query gets the chassis dispatcher fired. §H: mysqlx_start() no longer does runtime sync; describes the four install_<X>_from_admin calls in their place. §I: copy_table removed from the description; new LOAD/SAVE callbacks calling MysqlxConfigStore install/save APIs; four refresh_<X>_runtime_view callbacks registered via services.register_runtime_view(). §J: MysqlxConfigStore reframed as the canonical source of truth (runtime_mysqlx_<X> are projections of it). API list replaced load_from_runtime with the four install/save/ project triplets, install_all_from_admin (test convenience), snapshot_active_routes, and MysqlxBackendEndpointOverride. §K: listener reconciler reads MysqlxConfigStore directly via snapshot_active_routes(); inline rationale notes why we must NOT read runtime_mysqlx_routes (empty between LOAD calls under the new architecture). - ABI.md: services availability matrix gains register_runtime_view (live in Phase B and D); current-ABI block 3/3; §5 retitled "Separation of duties" with the new contract; disk-tier exception keeps the empty-source-sync invariant. - REVIEW_GUIDE.md: ABI-surface bullets updated; mysqlx phase descriptions updated to point at install_*_from_admin / runtime-view projections / snapshot_active_routes-driven reconciler. - PLUGIN_API.md: descriptor table updated for ABI 3; services struct snippet shows the ABI-2 and ABI-3 tail extensions; new register_runtime_view callback section with the contract; "Three-Tier" section retitled "Separation of duties", clarifies that runtime_<X> is an admin-side projection (not a persistent tier) and the empty-source-sync invariant applies only to disk- tier copies; alias-handling note corrected. The only surviving "ABI version 2" mention is in PLUGIN_API.md line 46 ("plugins that declare ABI version 2 or higher") — that is a correct version-or-higher predicate, not a stale reference.	2 months ago
Rene Cannao	c723ede0cf	Merge remote-tracking branch 'origin/plugin-chassis' into ProtocolX-rebased ... # Conflicts: # lib/ProxySQL_PluginManager.cpp # plugins/mysqlx/Makefile # plugins/mysqlx/include/mysqlx_plugin.h # plugins/mysqlx/include/mysqlx_session.h # test/tap/groups/groups.json # test/tap/tests/test_mysqlx_listener_smoke-t.cpp # test/tap/tests/unit/Makefile	2 months ago
Rene Cannao	e462bb0cb4	fix: address PR review feedback ... Resolves in-scope review items on PR #5651. Items deemed out of scope for this PR (O(N) admin dispatch lookup, per-plugin mutex, worker backpressure, runtime-reload DDL semantics, E2E tests going through ProxySQL rather than directly to MySQL) are tracked elsewhere; items that were already fixed in earlier commits on this branch (Admin_Handler MYSQLX alias vectors, Admin_Bootstrap PROXYSQL40 gating) are acknowledged as stale. Changes: * .github/workflows/CI-mysqlx.yml: add `include/` and `src/proxysql_global.cpp` to both `sparse-checkout` blocks. The plugin Makefile discovers the repo root via `src/proxysql_global.cpp` and includes headers from `include/`; without these, `make` inside `plugins/mysqlx` climbs up to `/` or fails on missing includes. (coderabbit: "Checkout the files required by the plugin Makefile.") * .github/workflows/CI-mysqlx.yml: `fail-on-cache-miss` now depends on `github.event_name == 'workflow_run'`. Strict behavior is retained for the cache-driven pipeline that CI-trigger populates; manual `workflow_dispatch` runs no longer fail when the SHA-specific cache doesn't exist yet. (coderabbit: "Don't make manual runs depend on a pre-existing build cache.") * plugins/mysqlx/Makefile: replace the unbounded `while [ ! -f ./src/proxysql_global.cpp ]; do cd ..; done` with a bounded upward walk (up to 12 levels) that fails fast with a clear error message when the marker file is missing — e.g. on a sparse checkout that omits `src/proxysql_global.cpp`. Build output no longer appears to hang indefinitely. (coderabbit: "Prevent root discovery from hanging on sparse or invalid checkouts.") * doc/PLUGIN_API.md: update startup sequence from the old three-phase (load → init → start) description to the four-phase chassis lifecycle (load → register_schemas → admin materialize → init → start). Document the `register_schemas` descriptor field and the `abi_version` / `PROXYSQL_PLUGIN_ABI_VERSION` contract. Clarify that `stop()` pairs with `init()` (not `start()`) for teardown symmetry. (coderabbit: "Update plugin lifecycle documentation to reflect four-phase model and ABI versioning.") * plugins/mysqlx/include/mysqlx_session.h: default-initialize every field in `MysqlxCredentials`. `bool x_enabled` was indeterminate for a default-constructed instance, which would let a lookup miss produce a value that silently authenticates. In-class initializers give every field a deterministic default. (coderabbit: "Default- initialize credential fields used by auth decisions.") * plugins/mysqlx/src/mysqlx_backend_session.cpp: close `backend_fd_` when `authenticate_backend()` returns false. Previously `connect()` stored the newly-opened fd in `backend_fd_` and returned the auth result; a retry on the same `MysqlxBackendSession` would then overwrite `backend_fd_` and leak the previous socket. (coderabbit: "Close/reset `backend_fd_` when authentication fails.") * lib/Admin_Bootstrap.cpp: `materialize_plugin_tables()` now runs DDL only for the subset of plugin-declared tables that were freshly inserted into `tables_defs_*` (the rest are already materialized). DDL failures are startup-fatal: on `execute()` returning false, log the plugin kind + table name + SQL and `exit(EXIT_FAILURE)`. A partially materialized schema produces opaque runtime errors from the plugin later on, so aborting startup is the right default. (gemini + coderabbit: "Execute DDL only for newly materialized plugin tables" / "add error checking".) Not changed in this commit (deferred / out of scope / stale): * CI-mysqlx E2E tests going through ProxySQL (requires plumbing listener + admin config in the CI workflow). * O(N) admin dispatch scan → hash map. * Descriptor build-id hash for libstdc++ compat detection. * Per-plugin mutex on mutable plugin context. * Worker client-fd enqueue backpressure. * Stale plan-doc feedback on docs/superpowers/plans/*.md. Verification (clean tree): * `unset PROXYSQL*; make cleanall && make debug -j$(nproc) && make build_tap_test_debug -j$(nproc)` -- 0 errors. * `PROXYSQLGENAI=1 make cleanall && PROXYSQLGENAI=1 make debug -j$(nproc) && PROXYSQLGENAI=1 make build_tap_test_debug -j$(nproc)` -- 0 errors. * Plugin unit tests (PROXYSQL40): 48 + 52 + 26 + 46 + 10 = 182 pass.	2 months ago
Rene Cannao	f104116926	fix(plugin-mgr): serialize lifecycle and always reset manager on stop ... Add g_plugin_lifecycle_mutex to serialize load/start/stop transitions so two reload paths cannot race on g_registry_target and the registration-failure globals. g_active_plugin_manager_mutex remains the pointer-read guard for the dispatch path; the new mutex is held only for the duration of a lifecycle transition. In proxysql_stop_configured_plugins, always call manager.reset() so the .so is unmapped and no stale function pointers remain reachable, even when stop_all() reports a failure. stop_all() is idempotent across failure (each plugin is marked stopped after one attempt) so the destructor's stop_all() will be a no-op. Also fix the log-level table in PLUGIN_API.md: numeric levels are 3 (Error), 4 (Warning), and any-other (Info), matching the internal proxy_* severity scheme — not 1/2/3 as previously documented.	2 months ago
Rene Cannao	cd11df3a7f	doc: add Plugin API documentation for external users ... ProxySQL's plugin system was previously undocumented except for the header file (include/ProxySQL_Plugin.h) and the mysqlx reference implementation. A developer wanting to write a plugin had to reverse-engineer the contract from those sources. Add doc/PLUGIN_API.md covering: - How plugins are loaded (plugins= directive in proxysql.cnf) - Startup/shutdown lifecycle sequence - The ProxySQL_PluginDescriptor contract (name, abi_version, init/start/stop/status_json) - The proxysql_plugin_descriptor_v1 entry point - ProxySQL_PluginServices callbacks (register_table, register_command, log_message, get_admindb/configdb/statsdb, snapshots) - ProxySQL_PluginCommandContext and ProxySQL_PluginCommandResult types - Complete minimal plugin example - Build requirements and compiler compatibility notes - Admin integration patterns (three-tier config model, table registration) - Current limitations (no hot-loading, no dependency resolution, compiler coupling) - Pointer to the mysqlx reference implementation	3 months ago