From 64e733429755b2beadb9500ce5a4bfe3d4afb674 Mon Sep 17 00:00:00 2001 From: Rene Cannao Date: Fri, 1 May 2026 03:54:40 +0000 Subject: [PATCH 1/3] docs(plugin-chassis): align FILE_CHANGES/ABI/REVIEW_GUIDE/PLUGIN_API with PR #5688 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The four chassis docs still described the pre-fix architecture: - ABI version 2 (now 3 — register_runtime_view appended at services tail) - MysqlxConfigStore::load_from_runtime (removed; replaced by per-entity install/save/project triplets) - copy_table / copy_to_runtime as the LOAD/SAVE shovels (deleted) - Listener reconciler reading runtime_mysqlx_routes (now reads MysqlxConfigStore::snapshot_active_routes) - "Three-tier" disk/memory/runtime model with the empty-source-sync invariant applied uniformly (the runtime tier was wrong; runtime_ is now an admin-side projection of module state, not a tier; the invariant only applies to disk-tier copies) - Plugin-context layout listing a non-existent MysqlxStatsStore Updates per file (no source code touched): - FILE_CHANGES.md §A: PROXYSQL_PLUGIN_ABI_VERSION 2 → 3, services list gains register_runtime_view, ProxySQL_PluginRuntimeView struct documented, separation-of-duties contract spelled out. §B: PluginManager methods list gains register_runtime_view, refresh_runtime_views_for_query; service trampolines list gains register_runtime_view_service; sql_references_table_ci whole-identifier match described; services_phase_b_ wiring of register_runtime_view explained. §C: new pre-SELECT runtime-view dispatch site in GenericRefreshStatistics described — gated on if (admin), OUTSIDE the if (refresh==true) block, so any admin-port query gets the chassis dispatcher fired. §H: mysqlx_start() no longer does runtime sync; describes the four install__from_admin calls in their place. §I: copy_table removed from the description; new LOAD/SAVE callbacks calling MysqlxConfigStore install/save APIs; four refresh__runtime_view callbacks registered via services.register_runtime_view(). §J: MysqlxConfigStore reframed as the canonical source of truth (runtime_mysqlx_ are projections of it). API list replaced load_from_runtime with the four install/save/ project triplets, install_all_from_admin (test convenience), snapshot_active_routes, and MysqlxBackendEndpointOverride. §K: listener reconciler reads MysqlxConfigStore directly via snapshot_active_routes(); inline rationale notes why we must NOT read runtime_mysqlx_routes (empty between LOAD calls under the new architecture). - ABI.md: services availability matrix gains register_runtime_view (live in Phase B and D); current-ABI block 3/3; §5 retitled "Separation of duties" with the new contract; disk-tier exception keeps the empty-source-sync invariant. - REVIEW_GUIDE.md: ABI-surface bullets updated; mysqlx phase descriptions updated to point at install_*_from_admin / runtime-view projections / snapshot_active_routes-driven reconciler. - PLUGIN_API.md: descriptor table updated for ABI 3; services struct snippet shows the ABI-2 and ABI-3 tail extensions; new register_runtime_view callback section with the contract; "Three-Tier" section retitled "Separation of duties", clarifies that runtime_ is an admin-side projection (not a persistent tier) and the empty-source-sync invariant applies only to disk- tier copies; alias-handling note corrected. The only surviving "ABI version 2" mention is in PLUGIN_API.md line 46 ("plugins that declare ABI version 2 or higher") — that is a correct version-or-higher predicate, not a stale reference. --- doc/PLUGIN_API.md | 169 ++++++++++++++++++++++------- doc/plugin-chassis/ABI.md | 35 ++++-- doc/plugin-chassis/FILE_CHANGES.md | 55 ++++++---- doc/plugin-chassis/REVIEW_GUIDE.md | 15 +-- 4 files changed, 192 insertions(+), 82 deletions(-) diff --git a/doc/PLUGIN_API.md b/doc/PLUGIN_API.md index 259a3d706..27c6fe06a 100644 --- a/doc/PLUGIN_API.md +++ b/doc/PLUGIN_API.md @@ -43,22 +43,24 @@ startup phase before the database takes precedence). ProxySQL uses a **four-phase** plugin lifecycle. Every phase but Phase B is mandatory; Phase B is optional via the `register_schemas` descriptor -field and only enabled when the plugin declares ABI version 2. +field and only enabled when the plugin declares ABI version 2 or higher. 1. **Phase A — load.** ProxySQL parses `proxysql.cnf` and populates the `plugins` list. For each plugin path, ProxySQL calls `dlopen()`, resolves the `proxysql_plugin_descriptor_v1` symbol, and validates the descriptor (`abi_version`, `name`, callback pointers). -2. **Phase B — register_schemas (optional, ABI 2 only).** If the +2. **Phase B — register_schemas (optional, ABI 2+).** If the descriptor wires `register_schemas`, the loader invokes it with a `ProxySQL_PluginServices` whose `register_table` / - `register_command` / `register_command_alias` entries are LIVE but - whose DB-handle getters (`get_admindb`, `get_configdb`, - `get_statsdb`) are non-null stubs that return `nullptr`. The plugin - declares the tables it owns and (optionally) its admin commands; it - MUST NOT touch DB handles here. Plugins that leave - `register_schemas` null (or that declare ABI 1) skip this phase - entirely and do all their setup in Phase D. + `register_command` / `register_command_alias` / + `register_runtime_view` (ABI 3+) entries are LIVE but whose + DB-handle getters (`get_admindb`, `get_configdb`, `get_statsdb`) + are non-null stubs that return `nullptr`. The plugin declares the + tables it owns, its admin commands, and any admin-side runtime + views it wants the chassis to project from module state; it MUST + NOT touch DB handles here. Plugins that leave `register_schemas` + null (or that declare ABI 1) skip this phase entirely and do all + their setup in Phase D. 3. **Phase C — admin materialization.** The admin module initializes and materializes the SQLite schemas collected during Phase B (`merge_plugin_tables` + `CREATE TABLE`). On DDL failure ProxySQL @@ -101,24 +103,24 @@ All types are defined in `include/ProxySQL_Plugin.h`: ```cpp struct ProxySQL_PluginDescriptor { const char *name; // Human-readable plugin name - uint32_t abi_version; // PROXYSQL_PLUGIN_ABI_VERSION (1 or 2) + uint32_t abi_version; // PROXYSQL_PLUGIN_ABI_VERSION (1, 2, or 3) proxysql_plugin_init_cb init; // bool (*)(ProxySQL_PluginServices *) proxysql_plugin_start_cb start; // bool (*)() proxysql_plugin_stop_cb stop; // bool (*)() proxysql_plugin_status_json_cb status_json; // const char *(*)() - proxysql_plugin_register_schemas_cb register_schemas; // ABI 2 only, optional + proxysql_plugin_register_schemas_cb register_schemas; // ABI 2+, optional }; ``` | Field | Type | Description | |--------------------|---------------|-----------------------------------------------------------| | `name` | `const char*` | Plugin identifier, used in logging. | -| `abi_version` | `uint32_t` | Set from `PROXYSQL_PLUGIN_ABI_VERSION`. Value `1` = pre-chassis descriptor (six fields). Value `2` = PROXYSQL40 descriptor (adds `register_schemas`). A v3/v3.1 ProxySQL core rejects `abi_version > 1`. | +| `abi_version` | `uint32_t` | Set from `PROXYSQL_PLUGIN_ABI_VERSION`. Value `1` = pre-chassis descriptor (six fields). Value `2` = adds `register_schemas` (four-phase lifecycle). Value `3` = same descriptor layout as `2`; `ProxySQL_PluginServices` adds a tail-appended `register_runtime_view`. A v3/v3.1 ProxySQL core rejects `abi_version > 1`; the current PROXYSQL40 core accepts `[1, 3]`. | | `init` | callback | Phase D — called with live services; register tables and commands here (or finish context setup if `register_schemas` already did it). | | `start` | callback | Phase E — start threads, open sockets, load config. | | `stop` | callback | Called on shutdown. Pairs with `init`, not `start`: if `init` returned true and `start` later failed, `stop` is still called so the plugin can release resources it allocated in `init`. | | `status_json` | callback | Return a static JSON string describing plugin status. | -| `register_schemas` | callback | Phase B (ABI 2 only). Optional; leave null to skip Phase B entirely. Services passed here have `register_table` / `register_command` / `register_command_alias` LIVE but DB-handle getters returning `nullptr`. | +| `register_schemas` | callback | Phase B (ABI 2+). Optional; leave null to skip Phase B entirely. Services passed here have `register_table` / `register_command` / `register_command_alias` / `register_runtime_view` LIVE but DB-handle getters returning `nullptr`. | All callbacks return `bool` (except `status_json` which returns `const char*`). Return `true` on success, `false` on failure. A `false` return from @@ -126,12 +128,16 @@ Return `true` on success, `false` on failure. A `false` return from #### ABI version -`include/ProxySQL_Plugin.h` exposes `PROXYSQL_PLUGIN_ABI_VERSION` (2 under +`include/ProxySQL_Plugin.h` exposes `PROXYSQL_PLUGIN_ABI_VERSION` (3 under PROXYSQL40, undefined in pre-chassis builds — the descriptor is then a legacy six-field struct with `abi_version = 1`). Plugins MUST assign `abi_version` from this macro rather than hard-coding a literal; the core's loader uses it to detect layout skew and reject plugins built -for an unsupported ABI. See `ProxySQL_Plugin.h` for the exact rules. +for an unsupported ABI. ABI 3 keeps the descriptor layout identical to +ABI 2 — the only addition is a tail-appended `register_runtime_view` +field on `ProxySQL_PluginServices` — so plugins that compile against +ABI 2 still load on the current core; the trailing services field is +simply invisible to them. See `ProxySQL_Plugin.h` for the exact rules. ### The Entry Point @@ -157,6 +163,12 @@ struct ProxySQL_PluginServices { proxysql_plugin_db_handle_cb get_admindb; proxysql_plugin_db_handle_cb get_configdb; proxysql_plugin_db_handle_cb get_statsdb; + // ABI 2 (PROXYSQL40) tail extensions: + proxysql_plugin_register_query_hook_cb register_query_hook; + proxysql_plugin_get_prometheus_registry_cb get_prometheus_registry; + proxysql_plugin_register_command_alias_cb register_command_alias; + // ABI 3 tail extension: + proxysql_plugin_register_runtime_view_cb register_runtime_view; }; ``` @@ -188,10 +200,15 @@ struct ProxySQL_PluginTableDef { | `stats_db` | In-memory | Statistics/metrics tables | **Convention**: For configuration tables that support the standard -memory↔runtime↔disk tier model, register the table in **both** `admin_db` and -`config_db`. Create a separate `runtime_`-prefixed table in `admin_db` only. -This is the pattern used by ProxySQL's built-in modules (e.g., `mysql_users` + -`runtime_mysql_users`). +memory↔runtime↔disk tier model, register the editable table in **both** +`admin_db` and `config_db`. Register a separate `runtime_`-prefixed +table in `admin_db` only — but treat it as an admin-side **projection**, +not as a tier the plugin maintains: declare it via `register_table`, +then declare a refresh callback for it via `register_runtime_view` (ABI +3+). The callback is invoked by the chassis before any admin SELECT +against the table. This mirrors the canonical core pattern (`mysql_users` ++ `runtime_mysql_users`, where `runtime_mysql_users` is repopulated from +the in-memory `MySQL_Authentication` state on demand). #### `register_command` @@ -254,6 +271,41 @@ Return a snapshot of ProxySQL's internal MySQL topology state. These allow a plugin to access the current user list, server list, or group replication hostgroups without directly coupling to internal data structures. +#### `register_runtime_view` (ABI 3+) + +```cpp +struct ProxySQL_PluginRuntimeView { + const char *table_name; + void (*refresh)(SQLite3DB *admindb, void *opaque); + void *opaque; +}; + +bool register_runtime_view(const ProxySQL_PluginRuntimeView &view); +``` + +Declare an admin-side **view** of plugin-module state. The named +`table_name` lives in `admin_db` (typically `runtime_`) and +holds no persistent rows — the chassis invokes `refresh(admindb, +opaque)` before any admin SELECT against it. The refresh callback is +expected to do (typically) `BEGIN; DELETE FROM ; INSERT/REPLACE +INTO
...; COMMIT;` from the module's own in-memory state. + +The chassis deep-copies `table_name` so the plugin need not keep the +pointed-to string alive after registration. The callback pointer must +have static lifetime (typically a free function in the plugin `.so`). +`opaque` is plugin-owned and passed back unchanged on each invocation; +plugins that don't need it should pass `nullptr`. + +Returns `true` on successful registration, `false` if `table_name` is +already registered (by this or another plugin) or if `refresh` is +`nullptr`. + +`register_runtime_view` is live both during `register_schemas` (Phase B) +and `init` (Phase D). Plugins typically register views alongside the +editable tables they project. See the separation-of-duties contract +under [Admin Integration Patterns](#admin-integration-patterns) below +for why this exists. + ## Admin Command Context and Result ### Context @@ -391,47 +443,84 @@ callbacks, not by linking against ProxySQL's SQLite wrapper. ## Admin Integration Patterns -### The Three-Tier Configuration Model +### Separation of duties: Admin, the module, and the runtime view -ProxySQL uses a three-tier configuration model: +ProxySQL's three-tier configuration model is, in storage terms: ``` -DISK (on-disk SQLite) ↔ MEMORY (in-memory admin tables) ↔ RUNTIME (live state) +DISK (config_db) ↔ MEMORY (admin_db editable tables) ↔ RUNTIME (in-module state) ``` -Plugins that manage configuration should follow this pattern: +The crucial point is that **only the first two are persistent SQLite tables**. +"RUNTIME" is the plugin module's in-memory state — typically an object +guarded by its own mutex (e.g. `MysqlxConfigStore`). The +`runtime_` table you register in `admin_db` is **not** module +storage; it is an admin-side **view** of module state, projected on +demand. + +Therefore the canonical division of work is: + +- **Admin** owns the editable tables (`` in both `admin_db` and `config_db`). +- **The plugin module** owns the runtime state (an in-memory object). +- The `runtime_` table in `admin_db` is repopulated by the plugin's + refresh callback registered via `services.register_runtime_view(...)`. + +Concretely: + +| Command | What it does | +|---------|--------------| +| `LOAD TO RUNTIME` | Plugin reads the editable `admin_db.` and hands rows to its module via a typed install API that swaps state under the module's lock. **Does not touch `runtime_`.** | +| `SAVE [FROM RUNTIME] TO MEMORY` | Plugin dumps its in-memory state and `REPLACE INTO`s the editable `admin_db.`. **Does not read `runtime_`.** | +| `LOAD FROM DISK` / `SAVE TO DISK` | Plain `BEGIN/DELETE/INSERT/COMMIT` between `config_db.` and `admin_db.`. No module involvement. | +| `SELECT ... FROM runtime_` (admin port) | Chassis fires the registered refresh callback, which wipes `runtime_` and re-projects the module's current state. | + +This mirrors the core's own `MySQL_Authentication` / `runtime_mysql_users` +pattern (see `lib/ProxySQL_Admin.cpp::save_mysql_users_runtime_to_database`). + +#### Disk-tier sync invariant + +The disk-tier copies (LOAD/SAVE FROM/TO DISK) are still subject to the +**empty-source-must-still-clear-destination** rule. Run the +`DELETE`+`INSERT` unconditionally inside a single transaction and check +each `execute()` return; an empty source means "no rows", not "leave the +destination alone". PR #5643 fixed an early implementation that had +this wrong on the disk path. -1. Register tables in both `config_db` (for disk persistence) and `admin_db` - (for in-memory configuration). -2. Register `runtime_`-prefixed tables in `admin_db` for the live runtime state. -3. Register admin commands for each tier transition: - - `LOAD MYPLUGIN TO RUNTIME` — copy from memory to runtime tables - - `SAVE MYPLUGIN TO MEMORY` — copy from runtime to memory tables +The runtime path does not need this discipline because the module-side +install API is a typed swap, not a copy: replacing the in-memory state +with an empty set is a single atomic operation. ### Registering Admin Commands -Commands are registered with the canonical form. Alias support (e.g., `TO RUN` -for `TO RUNTIME`, `FROM MEM` for `FROM MEMORY`) is handled in ProxySQL's -`Admin_Handler.cpp`, not in the plugin. If you need new aliases, you must modify -the ProxySQL core to add the alias vectors and dispatch mapping. +Commands are registered with the canonical form. Aliases (e.g., `TO +RUN` for `TO RUNTIME`, `FROM MEM` for `FROM MEMORY`) are registered by +the plugin via `register_command_alias` (ABI 2+); the chassis resolves +incoming admin SQL to the canonical form before invoking the plugin's +callback. There is no longer a hardcoded alias ladder in +`Admin_Handler.cpp`. -### Table Registration Patterns +### Table and view Registration Patterns ```cpp -// Configuration table: visible in both admin and config databases +// Editable configuration table: visible in both admin and config databases. void register_config_table(ProxySQL_PluginServices& services, const char* name, const char* def) { services.register_table({ProxySQL_PluginDBKind::admin_db, name, def}); services.register_table({ProxySQL_PluginDBKind::config_db, name, def}); } -// Runtime table: admin database only -void register_runtime_table(ProxySQL_PluginServices& services, - const char* name, const char* def) { +// Admin-side projection of module state. Declare the empty table in +// admin_db, then wire a refresh callback that reprojects from the +// module before any admin SELECT. +void register_runtime_view_table(ProxySQL_PluginServices& services, + const char* name, const char* def, + void (*refresh)(SQLite3DB*, void*), + void* opaque) { services.register_table({ProxySQL_PluginDBKind::admin_db, name, def}); + services.register_runtime_view({name, refresh, opaque}); } -// Stats table: stats database only +// Stats table: stats database only. void register_stats_table(ProxySQL_PluginServices& services, const char* name, const char* def) { services.register_table({ProxySQL_PluginDBKind::stats_db, name, def}); @@ -446,7 +535,7 @@ void register_stats_table(ProxySQL_PluginServices& services, - **No dependency resolution**: Plugins are loaded in the order listed in `proxysql.cnf`. If one plugin depends on another, the dependency must be listed first. -- **Single ABI version**: Only ABI version 1 is supported. +- **ABI version range**: The current core accepts `abi_version` values in `[1, 3]`. Newly built plugins should set `abi_version = PROXYSQL_PLUGIN_ABI_VERSION`. - **Compiler coupling**: Plugins must match the ProxySQL core's C++ compiler and standard library due to `std::string` in `ProxySQL_PluginCommandResult`. diff --git a/doc/plugin-chassis/ABI.md b/doc/plugin-chassis/ABI.md index c7b099421..c1c6b201d 100644 --- a/doc/plugin-chassis/ABI.md +++ b/doc/plugin-chassis/ABI.md @@ -59,11 +59,14 @@ The chassis (`lib/ProxySQL_PluginManager.cpp:324–383`) enforces: ### Current ABI version ```c -#define PROXYSQL_PLUGIN_ABI_VERSION 2 -#define PROXYSQL_PLUGIN_ABI_VERSION_MAX 2 +#define PROXYSQL_PLUGIN_ABI_VERSION 3 +#define PROXYSQL_PLUGIN_ABI_VERSION_MAX 3 ``` -ABI 1 and ABI 2 differ in **one field**: ABI 2 appends `register_schemas`. ABI-1 plugins skip Phase B entirely. +ABI evolution so far: + +- **ABI 1 → ABI 2:** appends `register_schemas` to the descriptor (four-phase lifecycle). ABI-1 plugins skip Phase B entirely. +- **ABI 2 → ABI 3:** descriptor layout is **unchanged**. The single addition is a `register_runtime_view` callback at the **tail of `ProxySQL_PluginServices`** (see §3 below). ABI-2 plugins keep loading on an ABI-3 core: their compiled-against `ProxySQL_PluginServices` simply ends one field earlier, and core never dereferences the trailing field for them. The accept range remains `[1, PROXYSQL_PLUGIN_ABI_VERSION_MAX]`. Future ABI versions append fields. The chassis bumps `PROXYSQL_PLUGIN_ABI_VERSION_MAX` and gates each new field's read on `abi_version >= N`. @@ -88,16 +91,18 @@ The services struct is the **same shape** in every phase, but some function poin | `get_statsdb` | **returns nullptr** | live | live | live | | `register_query_hook` | **returns false (warn)** | live | n/a | n/a | | `get_prometheus_registry` | live | live | live | live | +| `register_runtime_view` (ABI 3) | live | live | n/a | n/a | Reasons: - **DB handles in Phase B** — the admin module hasn't initialized yet, so the SQLite handles don't exist. Returning a nullptr from a stub is safer than not installing the function pointer (a misbehaving plugin sees a nullptr return instead of crashing on a null function pointer call). - **`register_query_hook` in Phase B** — query hooks are registered in `commands_` / `mysql_query_hook_` / `pgsql_query_hook_`, which Phase D writes and workers read lock-free. Phase B is too early; the plugin is told "no" and warned. -- **Registration after Phase D** — the chassis does not currently support live registration. Once Phase D returns, `register_table` / `register_command` are not called by anyone. This is by design — see §6 for the worker-thread visibility argument. +- **`register_runtime_view` in Phase B** — runtime views are typically declared alongside the editable tables they project, so the callback is wired live in BOTH `services_phase_b_` and `services_` (Phase D). Plugins may also register from `init`. +- **Registration after Phase D** — the chassis does not currently support live registration. Once Phase D returns, `register_table` / `register_command` / `register_runtime_view` are not called by anyone. This is by design — see §6 for the worker-thread visibility argument. ### Field stability -The services struct is **tail-extensible**. The chassis fills the struct in declaration order and the plugin reads what it knows about. A plugin compiled against ABI 2 that runs on a future ABI-3 chassis will simply not see the new fields; that's fine. +The services struct is **tail-extensible**. The chassis fills the struct in declaration order and the plugin reads what it knows about. A plugin compiled against ABI 2 still loads on the current ABI-3 chassis: its compiled-against `ProxySQL_PluginServices` ends at `register_command_alias` and the chassis simply doesn't dereference the trailing `register_runtime_view` for that plugin. Same rule applies for any future ABI-N additions. The reverse — a future plugin trying to call a field that doesn't exist on the current chassis — would crash. The chassis prevents this by rejecting plugins whose `abi_version > PROXYSQL_PLUGIN_ABI_VERSION_MAX`. @@ -136,15 +141,23 @@ The chassis loads with `RTLD_NOW | RTLD_LOCAL`. This means: --- -## 5. The empty-source-sync invariant +## 5. Separation of duties between Admin and the plugin module + +This is the central behavioural contract for chassis-driven LOAD/SAVE commands. It is documented in full at the bottom of `include/ProxySQL_Plugin.h` and at length in `doc/PLUGIN_API.md`. Briefly: + +- **Admin** owns the editable, persistent configuration tables (e.g. `mysqlx_users`, `mysqlx_routes`). +- The **plugin module** owns the runtime state — typically an in-memory snapshot kept under its own mutex (e.g. `MysqlxConfigStore`). +- The `runtime_` table in `admin_db` is **not module storage**; it is an admin-side **view** of module state, projected on demand by a callback the plugin registers via `services.register_runtime_view(...)`. -This is a behavioural invariant baked into how the chassis-driven LOAD/SAVE commands work. It's documented at the bottom of `include/ProxySQL_Plugin.h` and at length in `doc/PLUGIN_API.md`. Briefly: +Therefore: -When a plugin's `LOAD ... TO RUNTIME` command runs, it copies rows from a "source" table (e.g. `mysqlx_users`) to a "runtime" table (e.g. `runtime_mysqlx_users`). The convention is that this is a full replace: existing runtime rows are deleted, then the source rows are inserted. **Including when the source table is empty.** An empty source means "no users / no routes / nothing", not "leave the runtime alone". +- `LOAD TO RUNTIME` reads the editable admin table and hands the rows to the module via a typed install API that swaps state under the module's own lock. It MUST NOT touch `runtime_`. +- `SAVE [FROM RUNTIME] TO MEMORY` dumps the module's in-memory state and `REPLACE INTO`s the editable admin table. It MUST NOT read `runtime_`. +- `runtime_` is repopulated by the registered refresh callback before any admin SELECT touches it. Admin's pre-SELECT hook walks every registered view and invokes the callback for any whose table name appears in the SQL. -This is the convention the core's own LOAD/SAVE commands follow. Plugins must do the same. PR #5643 fixed an early mysqlx implementation that did NOT delete first; it caused stale-row bugs after `DELETE FROM mysqlx_users; LOAD MYSQLX USERS TO RUNTIME;` left the runtime table populated with the previous values. +Disk-tier copies (`LOAD FROM DISK`, `SAVE TO DISK`) are the exception: those DO copy between configdb and admindb persistent tables, and they remain plain `BEGIN/DELETE/INSERT/COMMIT` with checked rollback. For those, the **empty-source-must-still-clear-destination** rule still applies — a `DELETE FROM mysqlx_users; SAVE MYSQLX USERS TO DISK;` must leave the disk table empty, not preserve the previous rows. PR #5643 fixed an early mysqlx implementation that omitted the unconditional DELETE on the disk path. -The reference implementation is `plugins/mysqlx/src/mysqlx_admin_schema.cpp:copy_table()` — it does `BEGIN; DELETE FROM ; INSERT INTO SELECT * FROM ; COMMIT;` with checked rollback on any failure. +The reference for the runtime-view path is `plugins/mysqlx/src/mysqlx_admin_schema.cpp` (each `load__to_runtime` callback calls `MysqlxConfigStore::install__from_admin`; each `save__from_runtime` calls `save__to_admin_table`; four free `refresh__runtime_view` callbacks are wired via `services.register_runtime_view`). --- @@ -238,7 +251,7 @@ static bool my_stop(const ProxySQL_PluginServices* services) { static const ProxySQL_PluginDescriptor descriptor = { "my_plugin", // name - PROXYSQL_PLUGIN_ABI_VERSION, // abi_version (= 2) + PROXYSQL_PLUGIN_ABI_VERSION, // abi_version (= 3) my_init, // init (Phase D) my_start, // start (Phase E) my_stop, // stop diff --git a/doc/plugin-chassis/FILE_CHANGES.md b/doc/plugin-chassis/FILE_CHANGES.md index c8f81ad89..6df6ed6d5 100644 --- a/doc/plugin-chassis/FILE_CHANGES.md +++ b/doc/plugin-chassis/FILE_CHANGES.md @@ -18,15 +18,17 @@ Sections **A–G** cover the **chassis core**. Sections **H–O** cover the **my - **Purpose:** the C++ ABI a plugin compiles against; all types are file-wide guarded by `#ifdef PROXYSQL40` so v3.x TUs see nothing. - **Key contents:** - - `PROXYSQL_PLUGIN_ABI_VERSION` (currently `2`) and `_MAX`. ABI 1 = original 6-field descriptor; ABI 2 appends `register_schemas` for the four-phase lifecycle. + - `PROXYSQL_PLUGIN_ABI_VERSION` (currently `3`) and `_MAX` (also 3). ABI 1 = original 6-field descriptor; ABI 2 appends `register_schemas` for the four-phase lifecycle; ABI 3 keeps the descriptor layout unchanged from ABI 2 and adds a single `register_runtime_view` callback at the **tail of `ProxySQL_PluginServices`** so plugins can declare admin-side projections of module state. ABI-2 plugins still load on an ABI-3 core (loader range is `[1, 3]`; the trailing services field is invisible to the older compile). - `ProxySQL_PluginDescriptor` — 7-field struct `{name, abi_version, init, start, stop, status_json, register_schemas}` returned via `extern "C" proxysql_plugin_descriptor_v1()`. - - `ProxySQL_PluginServices` — services injected into the plugin: `register_table`, `register_command`, `register_command_alias`, three snapshot stubs (always nullptr today), `log_message`, three `get_*db` getters, `register_query_hook`, `get_prometheus_registry`. Tail-append discipline. + - `ProxySQL_PluginServices` — services injected into the plugin: `register_table`, `register_command`, `register_command_alias`, three snapshot stubs (always nullptr today), `log_message`, three `get_*db` getters, `register_query_hook`, `get_prometheus_registry`, `register_runtime_view` (ABI 3, tail-appended). Tail-append discipline. - Per-phase availability matrix is documented inline above the struct (Phase B: `register_table` live, DB getters return nullptr; Phase D: full services). - Query-hook ABI — `ProxySQL_PluginQueryHookPayload/Result/Action`; one hook per protocol per plugin. + - Runtime-view ABI — `ProxySQL_PluginRuntimeView{table_name, refresh, opaque}` plus `proxysql_plugin_register_runtime_view_cb`. The refresh callback gets a borrowed `SQLite3DB*` and re-projects module state into the named admin-db table on demand. - Header explicitly calls out the `std::string` / `prometheus-cpp` C++-ABI coupling: plugins MUST share toolchain with core. - - Long footer comment encodes the empty-source sync invariant (PR #5643 lesson). + - Footer comment block encodes the **separation-of-duties contract**: LOAD reads the editable admin table and hands rows to the module's typed install API (never touches `runtime_`); SAVE dumps module state back to the editable table; `runtime_` is an admin-side projection refreshed by the registered view callback. Disk-tier copies are still plain BEGIN/DELETE/INSERT/COMMIT and still subject to the empty-source-must-clear-destination rule (PR #5643). - **Spot-check:** - Verify `register_schemas` is only dereferenced when `abi_version >= 2`. + - Verify `register_runtime_view` lives at the END of `ProxySQL_PluginServices`; older plugins compiled against the ABI-2 layout never see the field. - Verify the loader rejects `abi_version > PROXYSQL_PLUGIN_ABI_VERSION_MAX` rather than reading past its struct. ### `include/ProxySQL_PluginManager.h` — NEW, 172 lines @@ -34,10 +36,10 @@ Sections **A–G** cover the **chassis core**. Sections **H–O** cover the **my - **Purpose:** declares the in-process loader/dispatcher (`ProxySQL_PluginManager`) plus the free-standing helpers main.cpp/Admin call. - **Key contents:** - Class is move-only-ish (copy ctor/assign deleted). - - Public lifecycle: `load`, `invoke_register_schemas_phase`, `init_all`, `start_all`, `stop_all`; plus `tables(kind)`, `dispatch_admin_command`, `register_command_alias`, `resolve_alias_to_canonical`, `register_query_hook`, `has_query_hook`, `dispatch_query_hook`. - - Free-standing API: `proxysql_get_plugin_manager` (atomic load), `proxysql_load_configured_plugins` (Phase A+B, **publishes** the manager), `proxysql_init_configured_plugins` (Phase D), `proxysql_start_configured_plugins`, `proxysql_stop_configured_plugins`, plus three dispatch helpers (`dispatch_configured_plugin_admin_command`, `dispatch_configured_plugin_query_hook`, lock-free `has_configured_plugin_query_hook`) and `resolve_configured_plugin_admin_alias`. - - Internal `plugin_handle_t` carries dlopen handle + `schemas_registered/initialized/started/stopped` state flags. - - Two services structs: `services_` (Phase D) and `services_phase_b_` (Phase B, with stubbed DB getters and stubbed `register_query_hook`). + - Public lifecycle: `load`, `invoke_register_schemas_phase`, `init_all`, `start_all`, `stop_all`; plus `tables(kind)`, `dispatch_admin_command`, `register_command_alias`, `resolve_alias_to_canonical`, `register_query_hook`, `has_query_hook`, `dispatch_query_hook`, `register_runtime_view`, `refresh_runtime_views_for_query`. + - Free-standing API: `proxysql_get_plugin_manager` (atomic load), `proxysql_load_configured_plugins` (Phase A+B, **publishes** the manager), `proxysql_init_configured_plugins` (Phase D), `proxysql_start_configured_plugins`, `proxysql_stop_configured_plugins`, plus three dispatch helpers (`dispatch_configured_plugin_admin_command`, `dispatch_configured_plugin_query_hook`, lock-free `has_configured_plugin_query_hook`), `resolve_configured_plugin_admin_alias`, and `proxysql_refresh_configured_plugin_runtime_views` (admin pre-SELECT hook). + - Internal `plugin_handle_t` carries dlopen handle + `schemas_registered/initialized/started/stopped` state flags. `runtime_views_` vector holds `registered_runtime_view_t{table_name, refresh, opaque}` entries. + - Two services structs: `services_` (Phase D, full services) and `services_phase_b_` (Phase B, with stubbed DB getters and stubbed `register_query_hook`). **`register_runtime_view` is wired in BOTH structs** — plugins typically declare runtime views alongside their tables in `register_schemas`, so the callback is live in Phase B as well as Phase D. - **Spot-check:** - Lifecycle invariant: `proxysql_init_configured_plugins` must run BEFORE any worker thread takes the lock-free read path. - `resolve_alias_to_canonical` returns `std::string` by value (not borrowed `c_str()`) so a concurrent reload between resolve and dispatch can't dangle a pointer. @@ -51,7 +53,8 @@ Sections **A–G** cover the **chassis core**. Sections **H–O** cover the **my - **Purpose:** dlopen orchestration, services trampolines, lifecycle state machine, dispatch. - **Key contents:** - **Concurrency model:** `g_active_plugin_manager` is a `std::atomic`; reads/writes coordinated via `g_active_plugin_manager_mutex` (`std::shared_mutex` — readers from dispatch/resolve paths share, publishers/unpublishers take unique). A separate `std::mutex g_plugin_lifecycle_mutex` serializes load/start/stop transitions. The shared-mutex change exists explicitly to keep query-hook dispatch from collapsing per-worker thread parallelism. - - **Service trampolines** (file-static): `register_table_service`, `register_command_service`, `register_command_alias_service`, `register_query_hook_service`, `get_admindb/configdb/statsdb_service`, `log_message_service`, `get_prometheus_registry_service`. Each writes to `g_registry_target` set by a `ScopedRegistryTarget` RAII guard around plugin callbacks; `note_registration_failure` records the first failure. + - **Service trampolines** (file-static): `register_table_service`, `register_command_service`, `register_command_alias_service`, `register_query_hook_service`, `register_runtime_view_service`, `get_admindb/configdb/statsdb_service`, `log_message_service`, `get_prometheus_registry_service`. Each writes to `g_registry_target` set by a `ScopedRegistryTarget` RAII guard around plugin callbacks; `note_registration_failure` records the first failure. + - **`sql_references_table_ci()`** is the matcher used by `refresh_runtime_views_for_query`: case-insensitive whole-identifier substring match treating `[A-Za-z0-9_]` as identifier characters. So `runtime_mysqlx_users` matches in `` SELECT * FROM `runtime_mysqlx_users` `` but NOT in `runtime_mysqlx_users_extra` or `stats_runtime_mysqlx_users`. - **`load()`** (lines 324–383): rejects duplicate paths; `dlopen(RTLD_NOW|RTLD_LOCAL)`; resolves `proxysql_plugin_descriptor_v1`; rejects null/empty `name` and `abi_version` outside `[1, PROXYSQL_PLUGIN_ABI_VERSION_MAX]`. - **`invoke_register_schemas_phase()`** (lines 386–461): Phase B; only reads `descriptor->register_schemas` when `abi_version >= 2`; snapshots `tables_*`/`commands_`/storage sizes and rolls back on failure to keep retries idempotent. - **`init_all()`** has the same snapshot/rollback contract. @@ -91,11 +94,12 @@ Sections **A–G** cover the **chassis core**. Sections **H–O** cover the **my ### `lib/ProxySQL_Admin.cpp` — MODIFIED, +42 lines -- **Purpose:** exposes admin/config/stats DB handles to the loader and adds the templated dispatcher. +- **Purpose:** exposes admin/config/stats DB handles to the loader, adds the templated dispatcher, and wires the pre-SELECT runtime-view dispatch. - **Key contents:** - Three free functions `proxysql_plugin_get_admindb/configdb/statsdb()` return `GloAdmin->{admindb,configdb,statsdb}` if `GloAdmin` is non-null. Gated by `PROXYSQL40` so v3.x exports no plugin-aware symbols at all. - `ProxySQL_Admin::dispatch_plugin_admin_command(sess, sql)` builds a `ProxySQL_PluginCommandContext{admindb,configdb,statsdb}`, calls `proxysql_dispatch_configured_plugin_admin_command`, and translates `result` into `send_ok_msg_to_client` / `send_error_msg_to_client`. - Explicit template instantiations for `MySQL_Session` and `PgSQL_Session`. + - **Pre-SELECT runtime-view dispatch in `GenericRefreshStatistics`** (around line 1654): `proxysql_refresh_configured_plugin_runtime_views(query_no_space, admindb)` is called for **every admin-port query**, gated only on `if (admin)` and placed **outside** the existing `if (refresh==true)` block. The chassis itself decides whether any plugin's refresh callback fires, by per-view substring match against the query — a query that touches no registered view is a cheap no-op (one shared lock + N substring scans). The `refresh` flag is left untouched; it gates a separate set of core-only refreshes (stats_mysql_processlist, runtime_mysql_users, etc.). ### `include/proxysql_admin.h` — MODIFIED, +8 lines @@ -181,7 +185,7 @@ The harness plugin source; one .cpp builds two .sos (`fake_plugin` and `fake_plu - Implements the `ProxySQL_PluginDescriptor` entry exported via `proxysql_plugin_descriptor_v1()`. - Four-phase lifecycle: `mysqlx_register_schemas` (Phase B, services without DB), `mysqlx_init` (Phase D, DB live), `mysqlx_start`, `mysqlx_stop`. - `replace_table_atomically()` — every `admindb.execute()` return is checked; defensive ROLLBACK on any failure including post-COMMIT, addresses the v3.0 silent-wipe bug. -- `mysqlx_start()` does disk→memory→runtime sync, loads config store, clamps pool size 1..64, then drives `mysqlx_reconcile_listeners()` on the same path used by `LOAD MYSQLX ROUTES TO RUNTIME`. +- `mysqlx_start()` does `sync_disk_to_memory()` (configdb → editable mysqlx_* tables) then four `install__from_admin` calls to populate the in-memory `MysqlxConfigStore` directly from the editable admin tables — no `runtime_mysqlx_*` write happens here, since runtime tables are now admin-side projections, not a tier the plugin maintains. Then clamps pool size 1..64 and drives `mysqlx_reconcile_listeners()` on the same path used by `LOAD MYSQLX ROUTES TO RUNTIME`. - `parse_bind_addr()` handles both `host:port` and `[ipv6]:port`; default port 33060. - Descriptor is constexpr-ish and pinned to `PROXYSQL_PLUGIN_ABI_VERSION`. @@ -192,42 +196,45 @@ The harness plugin source; one .cpp builds two .sos (`fake_plugin` and `fake_plu ### `plugins/mysqlx/include/mysqlx_admin_schema.h` — NEW, 8 lines Single-symbol header declaring `mysqlx_register_admin_schema()`. -### `plugins/mysqlx/src/mysqlx_admin_schema.cpp` — NEW, 565 lines +### `plugins/mysqlx/src/mysqlx_admin_schema.cpp` — NEW - Registers DDL for all `mysqlx_*` tables and the `LOAD/SAVE` admin commands. - Four config-table pairs (memory + runtime): `mysqlx_users`, `mysqlx_routes`, `mysqlx_backend_endpoints`, `mysqlx_variables`, all with `JSON_VALID(attributes)` CHECK constraints. - Two stats-only tables: `stats_mysqlx_routes`, `stats_mysqlx_processlist`. - 8 LOAD/SAVE TO RUNTIME commands plus 8 disk variants; each has alias group (`FROM MEMORY`, `FROM MEM`, `TO RUN`, etc.) registered via `register_command_alias`. -- `copy_table()` is BEGIN/DELETE/INSERT/COMMIT with checked rollback. +- LOAD/SAVE callbacks no longer copy between admin tables. Each `load__to_runtime` callback invokes `MysqlxConfigStore::install__from_admin(admindb, err)` (read editable mysqlx_, swap into the module's in-memory state under its own lock). Each `save__from_runtime` callback invokes `MysqlxConfigStore::save__to_admin_table(admindb)` (dump module state into the editable mysqlx_ table). Disk-tier copies (LOAD/SAVE FROM/TO DISK) remain plain BEGIN/DELETE/INSERT/COMMIT between configdb and admindb. +- Four free functions `refresh_users_runtime_view` / `refresh_routes_runtime_view` / `refresh_endpoints_runtime_view` / `refresh_variables_runtime_view` are registered at schema-registration time via `services.register_runtime_view({"runtime_mysqlx_", &refresh__runtime_view, nullptr})`. Each calls `MysqlxConfigStore::project__to_runtime_view(admindb)` to wipe the destination admin-db table and refill it from the module's in-memory state. The chassis fires the relevant callback before any admin SELECT against the projected table. - `load_routes_to_runtime` calls `mysqlx_reconcile_listeners` weak-pointer if non-null. --- ## J. Mysqlx plugin: configuration runtime -### `plugins/mysqlx/include/mysqlx_config_store.h` — NEW, 101 lines -### `plugins/mysqlx/src/mysqlx_config_store.cpp` — NEW, 382 lines +### `plugins/mysqlx/include/mysqlx_config_store.h` — NEW +### `plugins/mysqlx/src/mysqlx_config_store.cpp` — NEW -- Thread-safe in-memory view of runtime tables consumed by sessions/threads. -- `MysqlxResolvedIdentity` (was `MysqlxCredentials` — renamed in `84bbdfdca`). +- **Authoritative in-memory source of truth** for X-protocol routing/identity state. Sessions/threads consume it directly; the `runtime_mysqlx_*` tables in admin_db are admin-side projections of this store, refilled on demand by the plugin's runtime-view refresh callbacks (see §I). +- `MysqlxResolvedIdentity` (was `MysqlxCredentials` — renamed in `84bbdfdca`); now carries a `comment` field. +- `MysqlxRoute` also carries a `comment` field. - `MysqlxBackendAuthMode` enum: `mapped` / `service_account` / `pass_through`. -- API: `load_from_runtime`, `resolve_identity`, `pick_endpoint`, `route_hostgroup`, `route_exists`. -- `mutable std::shared_mutex mutex_` — readers (resolve_identity, pick_endpoint, route_exists) take shared locks; load_from_runtime takes exclusive. -- `load_from_runtime()` reads `runtime_mysql_users` (canonical password/hostgroup) then `runtime_mysqlx_users` (X-specific overrides), `runtime_mysqlx_routes`, `runtime_mysqlx_backend_endpoints`, `runtime_mysql_servers WHERE status='ONLINE'`, `runtime_mysqlx_variables`. -- Endpoint overrides (mysqlx_port, use_ssl, attributes) are keyed by `(hostname, mysql_port)` and applied on top of `runtime_mysql_servers`. +- Public struct `MysqlxBackendEndpointOverride{hostname, mysql_port, mysqlx_port, use_ssl, attributes, comment}` — promoted from a file-local `MysqlxEndpointOverride` so SAVE / projection can round-trip the per-(hostname, mysql_port) overrides. +- API: per-entity install / save / project triplets — `install_users_from_admin(db, err)` / `save_users_to_admin_table(db)` / `project_users_to_runtime_view(db)`, same shape for routes / endpoints / variables; convenience `install_all_from_admin(db, err)` (test-only fixture helper); `snapshot_active_routes()` returning `vector>` for the listener reconciler; plus the read-side `resolve_identity`, `pick_endpoint`, `route_hostgroup`, `route_exists`. +- `mutable std::shared_mutex mutex_` — readers (resolve_identity, pick_endpoint, route_exists, snapshot_active_routes, project_*_to_runtime_view) take shared locks; install_*_from_admin takes exclusive. Each install is independent: LOAD MYSQLX USERS does not touch routes/endpoints/variables. +- `install_users_from_admin()` reads the editable `mysqlx_users` plus cross-module `runtime_mysql_users` (canonical password/hostgroup); `install_endpoints_from_admin()` reads the editable `mysqlx_backend_endpoints` and cross-module `runtime_mysql_servers WHERE status='ONLINE'`. None of the install paths touch any `runtime_mysqlx_*` table — those are projections, not inputs. +- `endpoint_overrides_` (the public-struct overrides) is preserved across install_endpoints calls so SAVE can round-trip and the runtime-view projection faithfully reflects what was loaded. - `pick_from_hostgroup()` supports `first_available` and `round_robin[_with_fallback]`; round-robin uses a separate `rr_mutex_` so RR state doesn't interfere with config swaps. -- Test-only `install_for_test` is unconditionally available — note it bypasses `load_from_runtime`. +- Test-only `install_for_test` is unconditionally available — note it bypasses `install_*_from_admin`. --- ## K. Mysqlx plugin: listener reconciliation -### `plugins/mysqlx/src/mysqlx_listener_reconcile.cpp` — NEW, 151 lines +### `plugins/mysqlx/src/mysqlx_listener_reconcile.cpp` — NEW -- Desired-state reconciler from `runtime_mysqlx_routes` to per-thread listener fds. +- Desired-state reconciler from the `MysqlxConfigStore` (the authoritative in-memory state) to per-thread listener fds. Signature is `mysqlx_reconcile_listeners_impl(const MysqlxConfigStore& store, ...)` — no DB handle. - 3-step algorithm: snapshot desired set → remove stale/mis-bound listeners → add missing listeners (RR over thread pool). - Bind-change detection compares `threads[tidx]->get_listener_addr_for_route()` to `host:port` string; mismatch is treated as remove+add. -- Single-admin-thread assumption is documented inline; DB snapshot is taken outside `route_to_thread_mutex`. +- Single-admin-thread assumption is documented inline; the desired set comes from `store.snapshot_active_routes()` (taken under the store's shared lock, dropped before listener fd manipulation) rather than a SELECT against `runtime_mysqlx_routes`. An inline comment calls out why we deliberately do NOT read `runtime_mysqlx_routes` here: that table is an on-demand admin-side projection of the store, only populated when admin runs a SELECT against it; reading it from the LOAD path would see empty/stale data on every startup and every `LOAD MYSQLX ROUTES TO RUNTIME` call. - RR cursor `next_rr_index` is wrapped via `((next_rr_index % pool) + pool) % pool` — defensive against negative. - **Note:** if `add_listener` fails (port in use), the route is silently not added to `route_to_thread`; operator gets no error feedback beyond the chassis-level log. Future enhancement opportunity. diff --git a/doc/plugin-chassis/REVIEW_GUIDE.md b/doc/plugin-chassis/REVIEW_GUIDE.md index c46214e00..bc243a1de 100644 --- a/doc/plugin-chassis/REVIEW_GUIDE.md +++ b/doc/plugin-chassis/REVIEW_GUIDE.md @@ -54,7 +54,7 @@ Pick one based on your time budget. Each is cumulative — the 2-hour pass conti Goal: convince yourself the chassis ABI and lifecycle are sane and the v3.x invisibility is real. Don't read mysqlx; trust that it's a consumer. 1. Read **§3 (ABI surface)** below. Cross-check against [`ABI.md`](./ABI.md). -2. Skim `include/ProxySQL_Plugin.h` (324 LOC) — note `PROXYSQL_PLUGIN_ABI_VERSION = 2`, the descriptor struct, the services struct. +2. Skim `include/ProxySQL_Plugin.h` — note `PROXYSQL_PLUGIN_ABI_VERSION = 3`, the descriptor struct (unchanged from ABI 2), the services struct (ABI 3 appends `register_runtime_view` at the tail). 3. Read **§4 (Four-phase lifecycle)** below. 4. Skim `lib/ProxySQL_PluginManager.cpp` lines **324–461** (load + register_schemas + abi_version gating) and **548–576** (stop_all pairs with init). 5. Check the v3.x invisibility claim: @@ -93,9 +93,9 @@ The chassis is defined by **two headers** plus the loader implementation. The co **Key types (in `include/ProxySQL_Plugin.h`):** -- `PROXYSQL_PLUGIN_ABI_VERSION` (currently `2`) — what newly-built plugins target. ABI 1 was the original 6-field descriptor; ABI 2 appends `register_schemas` for the four-phase lifecycle. +- `PROXYSQL_PLUGIN_ABI_VERSION` (currently `3`) — what newly-built plugins target. ABI 1 was the original 6-field descriptor; ABI 2 appends `register_schemas` for the four-phase lifecycle; ABI 3 keeps the descriptor unchanged and appends `register_runtime_view` at the tail of `ProxySQL_PluginServices` so plugins can declare admin-side projections of module state. - `ProxySQL_PluginDescriptor` — 7-field struct returned via `extern "C" proxysql_plugin_descriptor_v1()`. The single mandatory entry point a plugin must export. -- `ProxySQL_PluginServices` — services struct injected into the plugin: table/command/query-hook registration, log helper, three DB getters, prometheus registry. Tail-append discipline preserves ABI compatibility. +- `ProxySQL_PluginServices` — services struct injected into the plugin: table/command/query-hook registration, log helper, three DB getters, prometheus registry, runtime-view registration. Tail-append discipline preserves ABI compatibility (ABI-2 plugins compile against the smaller layout and still load on an ABI-3 core). **The contract of the descriptor is:** - `name` is a non-null, non-empty C string. The loader rejects anything else. @@ -191,10 +191,11 @@ client ProxySQL The mysqlx plugin demonstrates every chassis affordance: -- **Phase B** — `mysqlx_register_schemas` declares 8 admin-schema tables (`mysqlx_users`, `mysqlx_routes`, `mysqlx_backend_endpoints`, `mysqlx_variables`, plus their `runtime_*` mirrors and `stats_mysqlx_*` tables) and 16 admin commands (`LOAD MYSQLX USERS TO RUNTIME` and the 7 cousins, plus `SAVE` variants and aliases like `FROM MEMORY` / `FROM MEM` / `TO RUN`). -- **Phase D** — `mysqlx_init` performs disk-to-runtime sync of the mysqlx tables on first boot. -- **Phase E** — `mysqlx_start` clamps the thread-pool size, drives the listener reconciler from `runtime_mysqlx_routes`, and spawns N worker threads. -- **Admin command dispatch** — every `LOAD MYSQLX … TO RUNTIME` / `SAVE` lands as a callback on `mysqlx_admin_schema.cpp:copy_table()`. +- **Phase B** — `mysqlx_register_schemas` declares 8 admin-schema tables (`mysqlx_users`, `mysqlx_routes`, `mysqlx_backend_endpoints`, `mysqlx_variables`, plus their `runtime_*` admin-side projections and `stats_mysqlx_*` tables), 16 admin commands (`LOAD MYSQLX USERS TO RUNTIME` and the 7 cousins, plus `SAVE` variants and aliases like `FROM MEMORY` / `FROM MEM` / `TO RUN`), and four runtime-view refresh callbacks via `services.register_runtime_view` (one per `runtime_mysqlx_` table). +- **Phase D** — `mysqlx_init` performs disk-to-memory sync of the mysqlx tables on first boot, then loads the in-memory `MysqlxConfigStore` directly from the editable admin tables via four `install__from_admin` calls. +- **Phase E** — `mysqlx_start` clamps the thread-pool size, drives the listener reconciler from `MysqlxConfigStore::snapshot_active_routes()`, and spawns N worker threads. +- **Admin command dispatch** — `LOAD MYSQLX TO RUNTIME` lands on a callback that invokes `MysqlxConfigStore::install__from_admin`; `SAVE MYSQLX [FROM RUNTIME] TO MEMORY` invokes `MysqlxConfigStore::save__to_admin_table`. The disk-tier variants (LOAD/SAVE FROM/TO DISK) remain a plain BEGIN/DELETE/INSERT/COMMIT between configdb and admindb. +- **Runtime-view projections** — when admin runs `SELECT * FROM runtime_mysqlx_users` (or one of the four cousins), the chassis fires the registered refresh callback, which calls `MysqlxConfigStore::project__to_runtime_view(admindb)` to wipe and refill the admin table from current module state. - **Identity callbacks** — each `MysqlxSession` is given an `identity_lookup_` closure that calls back into `MysqlxConfigStore::resolve_identity()` for the username the client sends. If you can convince yourself the chassis can host the mysqlx plugin coherently, the chassis is in good shape. From ef32d9df87f7bcaa0f77923b37b95df56e5cd1ab Mon Sep 17 00:00:00 2001 From: Rene Cannao Date: Fri, 1 May 2026 04:01:23 +0000 Subject: [PATCH 2/3] ci: register plugin_runtime_views_unit-t in groups.json MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Lint failure surfaced on PR #5690's CI run (job 73893199128) — the chassis runtime-view test added in PR #5688 was never registered in test/tap/groups/groups.json, and check_groups.py treats any unlisted TAP source as a lint error. Same group entry as the sibling plugin-chassis tests (plugin_dispatch _unit-t, plugin_manager_unit-t, etc.): "plugin_runtime_views_unit-t" : [ "unit-tests-g1","@proxysql_min_version:4.0" ] unit-tests-g1 is the standard host-only TAP group (no docker backend required); the @proxysql_min_version:4.0 attribute keeps it from running against pre-chassis builds. Trivial fix that belongs on the same PR as the doc updates because the lint failure is what surfaces on every PR opened against plugin-chassis until this lands. --- test/tap/groups/groups.json | 1 + 1 file changed, 1 insertion(+) diff --git a/test/tap/groups/groups.json b/test/tap/groups/groups.json index 037ff4b40..d0294de3a 100644 --- a/test/tap/groups/groups.json +++ b/test/tap/groups/groups.json @@ -202,6 +202,7 @@ "plugin_prometheus_unit-t" : [ "unit-tests-g1","@proxysql_min_version:4.0" ], "plugin_query_hook_unit-t" : [ "unit-tests-g1","@proxysql_min_version:4.0" ], "plugin_registry_unit-t" : [ "unit-tests-g1","@proxysql_min_version:4.0" ], + "plugin_runtime_views_unit-t" : [ "unit-tests-g1","@proxysql_min_version:4.0" ], "prepare_statement_err3024-t" : [ "legacy-g1","mysql-auto_increment_delay_multiplex=0-g1","mysql-multiplexing=false-g1","mysql-query_digests=0-g1","mysql-query_digests_keep_comment=1-g1","mysql84-g1","mysql90-g1","mysql95-g1" ], "prepare_statement_err3024_async-t" : [ "legacy-g1","mysql-auto_increment_delay_multiplex=0-g1","mysql-multiplexing=false-g1","mysql-query_digests=0-g1","mysql-query_digests_keep_comment=1-g1","mysql84-g1","mysql90-g1","mysql95-g1" ], "prepare_statement_err3024_libmysql-t" : [ "legacy-g1","mysql-auto_increment_delay_multiplex=0-g1","mysql-multiplexing=false-g1","mysql-query_digests=0-g1","mysql-query_digests_keep_comment=1-g1","mysql84-g1","mysql90-g1","mysql95-g1" ], From 29ee30daf9cf449fd96b5d1c03ebe1e17f24567c Mon Sep 17 00:00:00 2001 From: Rene Cannao Date: Fri, 1 May 2026 04:15:24 +0000 Subject: [PATCH 3/3] docs(plugin-chassis): address PR-#5690 review feedback MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CodeRabbit + Gemini review pass surfaced four small doc-precision items, all legitimate. None were code bugs. # CodeRabbit (markdownlint nit) - doc/PLUGIN_API.md:455: fenced code block at "DISK ↔ MEMORY ↔ RUNTIME" lacked a language specifier. Added `text` to satisfy MD040 / fenced-code-language. # Gemini (3x same root: clarify the matching algorithm) The chassis dispatch matcher is case-insensitive whole-identifier substring match, but three docs described it loosely. Plugin authors deciding what table name to register need to know that runtime_X does not match runtime_X_extra or stats_runtime_X. - doc/PLUGIN_API.md:289 ("any admin SELECT against it") -> spell out the whole-identifier rule with the longer-prefix / longer-suffix rejection examples. - doc/plugin-chassis/ABI.md:156 ("appears in the SQL") -> "is referenced as a whole identifier in the SQL query (case-insensitive; identifier-aware, so runtime_X_extra or stats_runtime_X do not match runtime_X)". - doc/plugin-chassis/FILE_CHANGES.md:102 ("substring match") -> "case-insensitive whole-identifier substring match", matching the precise wording already used in section B for the sql_references_table_ci helper. Cross-section consistency. No source code touched. No new content sections, only existing prose tightened. --- doc/PLUGIN_API.md | 12 ++++++++---- doc/plugin-chassis/ABI.md | 2 +- doc/plugin-chassis/FILE_CHANGES.md | 2 +- 3 files changed, 10 insertions(+), 6 deletions(-) diff --git a/doc/PLUGIN_API.md b/doc/PLUGIN_API.md index 27c6fe06a..89198ab6a 100644 --- a/doc/PLUGIN_API.md +++ b/doc/PLUGIN_API.md @@ -286,9 +286,13 @@ bool register_runtime_view(const ProxySQL_PluginRuntimeView &view); Declare an admin-side **view** of plugin-module state. The named `table_name` lives in `admin_db` (typically `runtime_`) and holds no persistent rows — the chassis invokes `refresh(admindb, -opaque)` before any admin SELECT against it. The refresh callback is -expected to do (typically) `BEGIN; DELETE FROM
; INSERT/REPLACE -INTO
...; COMMIT;` from the module's own in-memory state. +opaque)` before any admin `SELECT` that references the table as a +whole identifier. The match is case-insensitive but identifier-aware: +a query against `runtime_
_extra` (longer suffix) or +`stats_runtime_
` (longer prefix) does NOT trigger the refresh +for `runtime_
`. The refresh callback is expected to do +(typically) `BEGIN; DELETE FROM
; INSERT/REPLACE INTO +
...; COMMIT;` from the module's own in-memory state. The chassis deep-copies `table_name` so the plugin need not keep the pointed-to string alive after registration. The callback pointer must @@ -447,7 +451,7 @@ callbacks, not by linking against ProxySQL's SQLite wrapper. ProxySQL's three-tier configuration model is, in storage terms: -``` +```text DISK (config_db) ↔ MEMORY (admin_db editable tables) ↔ RUNTIME (in-module state) ``` diff --git a/doc/plugin-chassis/ABI.md b/doc/plugin-chassis/ABI.md index c1c6b201d..32e46e0ff 100644 --- a/doc/plugin-chassis/ABI.md +++ b/doc/plugin-chassis/ABI.md @@ -153,7 +153,7 @@ Therefore: - `LOAD TO RUNTIME` reads the editable admin table and hands the rows to the module via a typed install API that swaps state under the module's own lock. It MUST NOT touch `runtime_`. - `SAVE [FROM RUNTIME] TO MEMORY` dumps the module's in-memory state and `REPLACE INTO`s the editable admin table. It MUST NOT read `runtime_`. -- `runtime_` is repopulated by the registered refresh callback before any admin SELECT touches it. Admin's pre-SELECT hook walks every registered view and invokes the callback for any whose table name appears in the SQL. +- `runtime_` is repopulated by the registered refresh callback before any admin SELECT touches it. Admin's pre-SELECT hook walks every registered view and invokes the callback for any view whose table name is referenced as a whole identifier in the SQL query (case-insensitive; identifier-aware, so `runtime__extra` or `stats_runtime_` do not match `runtime_`). Disk-tier copies (`LOAD FROM DISK`, `SAVE TO DISK`) are the exception: those DO copy between configdb and admindb persistent tables, and they remain plain `BEGIN/DELETE/INSERT/COMMIT` with checked rollback. For those, the **empty-source-must-still-clear-destination** rule still applies — a `DELETE FROM mysqlx_users; SAVE MYSQLX USERS TO DISK;` must leave the disk table empty, not preserve the previous rows. PR #5643 fixed an early mysqlx implementation that omitted the unconditional DELETE on the disk path. diff --git a/doc/plugin-chassis/FILE_CHANGES.md b/doc/plugin-chassis/FILE_CHANGES.md index 6df6ed6d5..d66d4cd8b 100644 --- a/doc/plugin-chassis/FILE_CHANGES.md +++ b/doc/plugin-chassis/FILE_CHANGES.md @@ -99,7 +99,7 @@ Sections **A–G** cover the **chassis core**. Sections **H–O** cover the **my - Three free functions `proxysql_plugin_get_admindb/configdb/statsdb()` return `GloAdmin->{admindb,configdb,statsdb}` if `GloAdmin` is non-null. Gated by `PROXYSQL40` so v3.x exports no plugin-aware symbols at all. - `ProxySQL_Admin::dispatch_plugin_admin_command(sess, sql)` builds a `ProxySQL_PluginCommandContext{admindb,configdb,statsdb}`, calls `proxysql_dispatch_configured_plugin_admin_command`, and translates `result` into `send_ok_msg_to_client` / `send_error_msg_to_client`. - Explicit template instantiations for `MySQL_Session` and `PgSQL_Session`. - - **Pre-SELECT runtime-view dispatch in `GenericRefreshStatistics`** (around line 1654): `proxysql_refresh_configured_plugin_runtime_views(query_no_space, admindb)` is called for **every admin-port query**, gated only on `if (admin)` and placed **outside** the existing `if (refresh==true)` block. The chassis itself decides whether any plugin's refresh callback fires, by per-view substring match against the query — a query that touches no registered view is a cheap no-op (one shared lock + N substring scans). The `refresh` flag is left untouched; it gates a separate set of core-only refreshes (stats_mysql_processlist, runtime_mysql_users, etc.). + - **Pre-SELECT runtime-view dispatch in `GenericRefreshStatistics`** (around line 1654): `proxysql_refresh_configured_plugin_runtime_views(query_no_space, admindb)` is called for **every admin-port query**, gated only on `if (admin)` and placed **outside** the existing `if (refresh==true)` block. The chassis itself decides whether any plugin's refresh callback fires, by case-insensitive whole-identifier substring match against the query (matching the wording used in section B for the `sql_references_table_ci` matcher) — a query that touches no registered view is a cheap no-op (one shared lock + N substring scans). The `refresh` flag is left untouched; it gates a separate set of core-only refreshes (stats_mysql_processlist, runtime_mysql_users, etc.). ### `include/proxysql_admin.h` — MODIFIED, +8 lines