diff --git a/doc/PLUGIN_API.md b/doc/PLUGIN_API.md index 259a3d706..27c6fe06a 100644 --- a/doc/PLUGIN_API.md +++ b/doc/PLUGIN_API.md @@ -43,22 +43,24 @@ startup phase before the database takes precedence). ProxySQL uses a **four-phase** plugin lifecycle. Every phase but Phase B is mandatory; Phase B is optional via the `register_schemas` descriptor -field and only enabled when the plugin declares ABI version 2. +field and only enabled when the plugin declares ABI version 2 or higher. 1. **Phase A — load.** ProxySQL parses `proxysql.cnf` and populates the `plugins` list. For each plugin path, ProxySQL calls `dlopen()`, resolves the `proxysql_plugin_descriptor_v1` symbol, and validates the descriptor (`abi_version`, `name`, callback pointers). -2. **Phase B — register_schemas (optional, ABI 2 only).** If the +2. **Phase B — register_schemas (optional, ABI 2+).** If the descriptor wires `register_schemas`, the loader invokes it with a `ProxySQL_PluginServices` whose `register_table` / - `register_command` / `register_command_alias` entries are LIVE but - whose DB-handle getters (`get_admindb`, `get_configdb`, - `get_statsdb`) are non-null stubs that return `nullptr`. The plugin - declares the tables it owns and (optionally) its admin commands; it - MUST NOT touch DB handles here. Plugins that leave - `register_schemas` null (or that declare ABI 1) skip this phase - entirely and do all their setup in Phase D. + `register_command` / `register_command_alias` / + `register_runtime_view` (ABI 3+) entries are LIVE but whose + DB-handle getters (`get_admindb`, `get_configdb`, `get_statsdb`) + are non-null stubs that return `nullptr`. The plugin declares the + tables it owns, its admin commands, and any admin-side runtime + views it wants the chassis to project from module state; it MUST + NOT touch DB handles here. Plugins that leave `register_schemas` + null (or that declare ABI 1) skip this phase entirely and do all + their setup in Phase D. 3. **Phase C — admin materialization.** The admin module initializes and materializes the SQLite schemas collected during Phase B (`merge_plugin_tables` + `CREATE TABLE`). On DDL failure ProxySQL @@ -101,24 +103,24 @@ All types are defined in `include/ProxySQL_Plugin.h`: ```cpp struct ProxySQL_PluginDescriptor { const char *name; // Human-readable plugin name - uint32_t abi_version; // PROXYSQL_PLUGIN_ABI_VERSION (1 or 2) + uint32_t abi_version; // PROXYSQL_PLUGIN_ABI_VERSION (1, 2, or 3) proxysql_plugin_init_cb init; // bool (*)(ProxySQL_PluginServices *) proxysql_plugin_start_cb start; // bool (*)() proxysql_plugin_stop_cb stop; // bool (*)() proxysql_plugin_status_json_cb status_json; // const char *(*)() - proxysql_plugin_register_schemas_cb register_schemas; // ABI 2 only, optional + proxysql_plugin_register_schemas_cb register_schemas; // ABI 2+, optional }; ``` | Field | Type | Description | |--------------------|---------------|-----------------------------------------------------------| | `name` | `const char*` | Plugin identifier, used in logging. | -| `abi_version` | `uint32_t` | Set from `PROXYSQL_PLUGIN_ABI_VERSION`. Value `1` = pre-chassis descriptor (six fields). Value `2` = PROXYSQL40 descriptor (adds `register_schemas`). A v3/v3.1 ProxySQL core rejects `abi_version > 1`. | +| `abi_version` | `uint32_t` | Set from `PROXYSQL_PLUGIN_ABI_VERSION`. Value `1` = pre-chassis descriptor (six fields). Value `2` = adds `register_schemas` (four-phase lifecycle). Value `3` = same descriptor layout as `2`; `ProxySQL_PluginServices` adds a tail-appended `register_runtime_view`. A v3/v3.1 ProxySQL core rejects `abi_version > 1`; the current PROXYSQL40 core accepts `[1, 3]`. | | `init` | callback | Phase D — called with live services; register tables and commands here (or finish context setup if `register_schemas` already did it). | | `start` | callback | Phase E — start threads, open sockets, load config. | | `stop` | callback | Called on shutdown. Pairs with `init`, not `start`: if `init` returned true and `start` later failed, `stop` is still called so the plugin can release resources it allocated in `init`. | | `status_json` | callback | Return a static JSON string describing plugin status. | -| `register_schemas` | callback | Phase B (ABI 2 only). Optional; leave null to skip Phase B entirely. Services passed here have `register_table` / `register_command` / `register_command_alias` LIVE but DB-handle getters returning `nullptr`. | +| `register_schemas` | callback | Phase B (ABI 2+). Optional; leave null to skip Phase B entirely. Services passed here have `register_table` / `register_command` / `register_command_alias` / `register_runtime_view` LIVE but DB-handle getters returning `nullptr`. | All callbacks return `bool` (except `status_json` which returns `const char*`). Return `true` on success, `false` on failure. A `false` return from @@ -126,12 +128,16 @@ Return `true` on success, `false` on failure. A `false` return from #### ABI version -`include/ProxySQL_Plugin.h` exposes `PROXYSQL_PLUGIN_ABI_VERSION` (2 under +`include/ProxySQL_Plugin.h` exposes `PROXYSQL_PLUGIN_ABI_VERSION` (3 under PROXYSQL40, undefined in pre-chassis builds — the descriptor is then a legacy six-field struct with `abi_version = 1`). Plugins MUST assign `abi_version` from this macro rather than hard-coding a literal; the core's loader uses it to detect layout skew and reject plugins built -for an unsupported ABI. See `ProxySQL_Plugin.h` for the exact rules. +for an unsupported ABI. ABI 3 keeps the descriptor layout identical to +ABI 2 — the only addition is a tail-appended `register_runtime_view` +field on `ProxySQL_PluginServices` — so plugins that compile against +ABI 2 still load on the current core; the trailing services field is +simply invisible to them. See `ProxySQL_Plugin.h` for the exact rules. ### The Entry Point @@ -157,6 +163,12 @@ struct ProxySQL_PluginServices { proxysql_plugin_db_handle_cb get_admindb; proxysql_plugin_db_handle_cb get_configdb; proxysql_plugin_db_handle_cb get_statsdb; + // ABI 2 (PROXYSQL40) tail extensions: + proxysql_plugin_register_query_hook_cb register_query_hook; + proxysql_plugin_get_prometheus_registry_cb get_prometheus_registry; + proxysql_plugin_register_command_alias_cb register_command_alias; + // ABI 3 tail extension: + proxysql_plugin_register_runtime_view_cb register_runtime_view; }; ``` @@ -188,10 +200,15 @@ struct ProxySQL_PluginTableDef { | `stats_db` | In-memory | Statistics/metrics tables | **Convention**: For configuration tables that support the standard -memory↔runtime↔disk tier model, register the table in **both** `admin_db` and -`config_db`. Create a separate `runtime_`-prefixed table in `admin_db` only. -This is the pattern used by ProxySQL's built-in modules (e.g., `mysql_users` + -`runtime_mysql_users`). +memory↔runtime↔disk tier model, register the editable table in **both** +`admin_db` and `config_db`. Register a separate `runtime_`-prefixed +table in `admin_db` only — but treat it as an admin-side **projection**, +not as a tier the plugin maintains: declare it via `register_table`, +then declare a refresh callback for it via `register_runtime_view` (ABI +3+). The callback is invoked by the chassis before any admin SELECT +against the table. This mirrors the canonical core pattern (`mysql_users` ++ `runtime_mysql_users`, where `runtime_mysql_users` is repopulated from +the in-memory `MySQL_Authentication` state on demand). #### `register_command` @@ -254,6 +271,41 @@ Return a snapshot of ProxySQL's internal MySQL topology state. These allow a plugin to access the current user list, server list, or group replication hostgroups without directly coupling to internal data structures. +#### `register_runtime_view` (ABI 3+) + +```cpp +struct ProxySQL_PluginRuntimeView { + const char *table_name; + void (*refresh)(SQLite3DB *admindb, void *opaque); + void *opaque; +}; + +bool register_runtime_view(const ProxySQL_PluginRuntimeView &view); +``` + +Declare an admin-side **view** of plugin-module state. The named +`table_name` lives in `admin_db` (typically `runtime_`) and +holds no persistent rows — the chassis invokes `refresh(admindb, +opaque)` before any admin SELECT against it. The refresh callback is +expected to do (typically) `BEGIN; DELETE FROM ; INSERT/REPLACE +INTO
...; COMMIT;` from the module's own in-memory state. + +The chassis deep-copies `table_name` so the plugin need not keep the +pointed-to string alive after registration. The callback pointer must +have static lifetime (typically a free function in the plugin `.so`). +`opaque` is plugin-owned and passed back unchanged on each invocation; +plugins that don't need it should pass `nullptr`. + +Returns `true` on successful registration, `false` if `table_name` is +already registered (by this or another plugin) or if `refresh` is +`nullptr`. + +`register_runtime_view` is live both during `register_schemas` (Phase B) +and `init` (Phase D). Plugins typically register views alongside the +editable tables they project. See the separation-of-duties contract +under [Admin Integration Patterns](#admin-integration-patterns) below +for why this exists. + ## Admin Command Context and Result ### Context @@ -391,47 +443,84 @@ callbacks, not by linking against ProxySQL's SQLite wrapper. ## Admin Integration Patterns -### The Three-Tier Configuration Model +### Separation of duties: Admin, the module, and the runtime view -ProxySQL uses a three-tier configuration model: +ProxySQL's three-tier configuration model is, in storage terms: ``` -DISK (on-disk SQLite) ↔ MEMORY (in-memory admin tables) ↔ RUNTIME (live state) +DISK (config_db) ↔ MEMORY (admin_db editable tables) ↔ RUNTIME (in-module state) ``` -Plugins that manage configuration should follow this pattern: +The crucial point is that **only the first two are persistent SQLite tables**. +"RUNTIME" is the plugin module's in-memory state — typically an object +guarded by its own mutex (e.g. `MysqlxConfigStore`). The +`runtime_` table you register in `admin_db` is **not** module +storage; it is an admin-side **view** of module state, projected on +demand. + +Therefore the canonical division of work is: + +- **Admin** owns the editable tables (`` in both `admin_db` and `config_db`). +- **The plugin module** owns the runtime state (an in-memory object). +- The `runtime_` table in `admin_db` is repopulated by the plugin's + refresh callback registered via `services.register_runtime_view(...)`. + +Concretely: + +| Command | What it does | +|---------|--------------| +| `LOAD TO RUNTIME` | Plugin reads the editable `admin_db.` and hands rows to its module via a typed install API that swaps state under the module's lock. **Does not touch `runtime_`.** | +| `SAVE [FROM RUNTIME] TO MEMORY` | Plugin dumps its in-memory state and `REPLACE INTO`s the editable `admin_db.`. **Does not read `runtime_`.** | +| `LOAD FROM DISK` / `SAVE TO DISK` | Plain `BEGIN/DELETE/INSERT/COMMIT` between `config_db.` and `admin_db.`. No module involvement. | +| `SELECT ... FROM runtime_` (admin port) | Chassis fires the registered refresh callback, which wipes `runtime_` and re-projects the module's current state. | + +This mirrors the core's own `MySQL_Authentication` / `runtime_mysql_users` +pattern (see `lib/ProxySQL_Admin.cpp::save_mysql_users_runtime_to_database`). + +#### Disk-tier sync invariant + +The disk-tier copies (LOAD/SAVE FROM/TO DISK) are still subject to the +**empty-source-must-still-clear-destination** rule. Run the +`DELETE`+`INSERT` unconditionally inside a single transaction and check +each `execute()` return; an empty source means "no rows", not "leave the +destination alone". PR #5643 fixed an early implementation that had +this wrong on the disk path. -1. Register tables in both `config_db` (for disk persistence) and `admin_db` - (for in-memory configuration). -2. Register `runtime_`-prefixed tables in `admin_db` for the live runtime state. -3. Register admin commands for each tier transition: - - `LOAD MYPLUGIN TO RUNTIME` — copy from memory to runtime tables - - `SAVE MYPLUGIN TO MEMORY` — copy from runtime to memory tables +The runtime path does not need this discipline because the module-side +install API is a typed swap, not a copy: replacing the in-memory state +with an empty set is a single atomic operation. ### Registering Admin Commands -Commands are registered with the canonical form. Alias support (e.g., `TO RUN` -for `TO RUNTIME`, `FROM MEM` for `FROM MEMORY`) is handled in ProxySQL's -`Admin_Handler.cpp`, not in the plugin. If you need new aliases, you must modify -the ProxySQL core to add the alias vectors and dispatch mapping. +Commands are registered with the canonical form. Aliases (e.g., `TO +RUN` for `TO RUNTIME`, `FROM MEM` for `FROM MEMORY`) are registered by +the plugin via `register_command_alias` (ABI 2+); the chassis resolves +incoming admin SQL to the canonical form before invoking the plugin's +callback. There is no longer a hardcoded alias ladder in +`Admin_Handler.cpp`. -### Table Registration Patterns +### Table and view Registration Patterns ```cpp -// Configuration table: visible in both admin and config databases +// Editable configuration table: visible in both admin and config databases. void register_config_table(ProxySQL_PluginServices& services, const char* name, const char* def) { services.register_table({ProxySQL_PluginDBKind::admin_db, name, def}); services.register_table({ProxySQL_PluginDBKind::config_db, name, def}); } -// Runtime table: admin database only -void register_runtime_table(ProxySQL_PluginServices& services, - const char* name, const char* def) { +// Admin-side projection of module state. Declare the empty table in +// admin_db, then wire a refresh callback that reprojects from the +// module before any admin SELECT. +void register_runtime_view_table(ProxySQL_PluginServices& services, + const char* name, const char* def, + void (*refresh)(SQLite3DB*, void*), + void* opaque) { services.register_table({ProxySQL_PluginDBKind::admin_db, name, def}); + services.register_runtime_view({name, refresh, opaque}); } -// Stats table: stats database only +// Stats table: stats database only. void register_stats_table(ProxySQL_PluginServices& services, const char* name, const char* def) { services.register_table({ProxySQL_PluginDBKind::stats_db, name, def}); @@ -446,7 +535,7 @@ void register_stats_table(ProxySQL_PluginServices& services, - **No dependency resolution**: Plugins are loaded in the order listed in `proxysql.cnf`. If one plugin depends on another, the dependency must be listed first. -- **Single ABI version**: Only ABI version 1 is supported. +- **ABI version range**: The current core accepts `abi_version` values in `[1, 3]`. Newly built plugins should set `abi_version = PROXYSQL_PLUGIN_ABI_VERSION`. - **Compiler coupling**: Plugins must match the ProxySQL core's C++ compiler and standard library due to `std::string` in `ProxySQL_PluginCommandResult`. diff --git a/doc/plugin-chassis/ABI.md b/doc/plugin-chassis/ABI.md index c7b099421..c1c6b201d 100644 --- a/doc/plugin-chassis/ABI.md +++ b/doc/plugin-chassis/ABI.md @@ -59,11 +59,14 @@ The chassis (`lib/ProxySQL_PluginManager.cpp:324–383`) enforces: ### Current ABI version ```c -#define PROXYSQL_PLUGIN_ABI_VERSION 2 -#define PROXYSQL_PLUGIN_ABI_VERSION_MAX 2 +#define PROXYSQL_PLUGIN_ABI_VERSION 3 +#define PROXYSQL_PLUGIN_ABI_VERSION_MAX 3 ``` -ABI 1 and ABI 2 differ in **one field**: ABI 2 appends `register_schemas`. ABI-1 plugins skip Phase B entirely. +ABI evolution so far: + +- **ABI 1 → ABI 2:** appends `register_schemas` to the descriptor (four-phase lifecycle). ABI-1 plugins skip Phase B entirely. +- **ABI 2 → ABI 3:** descriptor layout is **unchanged**. The single addition is a `register_runtime_view` callback at the **tail of `ProxySQL_PluginServices`** (see §3 below). ABI-2 plugins keep loading on an ABI-3 core: their compiled-against `ProxySQL_PluginServices` simply ends one field earlier, and core never dereferences the trailing field for them. The accept range remains `[1, PROXYSQL_PLUGIN_ABI_VERSION_MAX]`. Future ABI versions append fields. The chassis bumps `PROXYSQL_PLUGIN_ABI_VERSION_MAX` and gates each new field's read on `abi_version >= N`. @@ -88,16 +91,18 @@ The services struct is the **same shape** in every phase, but some function poin | `get_statsdb` | **returns nullptr** | live | live | live | | `register_query_hook` | **returns false (warn)** | live | n/a | n/a | | `get_prometheus_registry` | live | live | live | live | +| `register_runtime_view` (ABI 3) | live | live | n/a | n/a | Reasons: - **DB handles in Phase B** — the admin module hasn't initialized yet, so the SQLite handles don't exist. Returning a nullptr from a stub is safer than not installing the function pointer (a misbehaving plugin sees a nullptr return instead of crashing on a null function pointer call). - **`register_query_hook` in Phase B** — query hooks are registered in `commands_` / `mysql_query_hook_` / `pgsql_query_hook_`, which Phase D writes and workers read lock-free. Phase B is too early; the plugin is told "no" and warned. -- **Registration after Phase D** — the chassis does not currently support live registration. Once Phase D returns, `register_table` / `register_command` are not called by anyone. This is by design — see §6 for the worker-thread visibility argument. +- **`register_runtime_view` in Phase B** — runtime views are typically declared alongside the editable tables they project, so the callback is wired live in BOTH `services_phase_b_` and `services_` (Phase D). Plugins may also register from `init`. +- **Registration after Phase D** — the chassis does not currently support live registration. Once Phase D returns, `register_table` / `register_command` / `register_runtime_view` are not called by anyone. This is by design — see §6 for the worker-thread visibility argument. ### Field stability -The services struct is **tail-extensible**. The chassis fills the struct in declaration order and the plugin reads what it knows about. A plugin compiled against ABI 2 that runs on a future ABI-3 chassis will simply not see the new fields; that's fine. +The services struct is **tail-extensible**. The chassis fills the struct in declaration order and the plugin reads what it knows about. A plugin compiled against ABI 2 still loads on the current ABI-3 chassis: its compiled-against `ProxySQL_PluginServices` ends at `register_command_alias` and the chassis simply doesn't dereference the trailing `register_runtime_view` for that plugin. Same rule applies for any future ABI-N additions. The reverse — a future plugin trying to call a field that doesn't exist on the current chassis — would crash. The chassis prevents this by rejecting plugins whose `abi_version > PROXYSQL_PLUGIN_ABI_VERSION_MAX`. @@ -136,15 +141,23 @@ The chassis loads with `RTLD_NOW | RTLD_LOCAL`. This means: --- -## 5. The empty-source-sync invariant +## 5. Separation of duties between Admin and the plugin module + +This is the central behavioural contract for chassis-driven LOAD/SAVE commands. It is documented in full at the bottom of `include/ProxySQL_Plugin.h` and at length in `doc/PLUGIN_API.md`. Briefly: + +- **Admin** owns the editable, persistent configuration tables (e.g. `mysqlx_users`, `mysqlx_routes`). +- The **plugin module** owns the runtime state — typically an in-memory snapshot kept under its own mutex (e.g. `MysqlxConfigStore`). +- The `runtime_` table in `admin_db` is **not module storage**; it is an admin-side **view** of module state, projected on demand by a callback the plugin registers via `services.register_runtime_view(...)`. -This is a behavioural invariant baked into how the chassis-driven LOAD/SAVE commands work. It's documented at the bottom of `include/ProxySQL_Plugin.h` and at length in `doc/PLUGIN_API.md`. Briefly: +Therefore: -When a plugin's `LOAD ... TO RUNTIME` command runs, it copies rows from a "source" table (e.g. `mysqlx_users`) to a "runtime" table (e.g. `runtime_mysqlx_users`). The convention is that this is a full replace: existing runtime rows are deleted, then the source rows are inserted. **Including when the source table is empty.** An empty source means "no users / no routes / nothing", not "leave the runtime alone". +- `LOAD TO RUNTIME` reads the editable admin table and hands the rows to the module via a typed install API that swaps state under the module's own lock. It MUST NOT touch `runtime_`. +- `SAVE [FROM RUNTIME] TO MEMORY` dumps the module's in-memory state and `REPLACE INTO`s the editable admin table. It MUST NOT read `runtime_`. +- `runtime_` is repopulated by the registered refresh callback before any admin SELECT touches it. Admin's pre-SELECT hook walks every registered view and invokes the callback for any whose table name appears in the SQL. -This is the convention the core's own LOAD/SAVE commands follow. Plugins must do the same. PR #5643 fixed an early mysqlx implementation that did NOT delete first; it caused stale-row bugs after `DELETE FROM mysqlx_users; LOAD MYSQLX USERS TO RUNTIME;` left the runtime table populated with the previous values. +Disk-tier copies (`LOAD FROM DISK`, `SAVE TO DISK`) are the exception: those DO copy between configdb and admindb persistent tables, and they remain plain `BEGIN/DELETE/INSERT/COMMIT` with checked rollback. For those, the **empty-source-must-still-clear-destination** rule still applies — a `DELETE FROM mysqlx_users; SAVE MYSQLX USERS TO DISK;` must leave the disk table empty, not preserve the previous rows. PR #5643 fixed an early mysqlx implementation that omitted the unconditional DELETE on the disk path. -The reference implementation is `plugins/mysqlx/src/mysqlx_admin_schema.cpp:copy_table()` — it does `BEGIN; DELETE FROM ; INSERT INTO SELECT * FROM ; COMMIT;` with checked rollback on any failure. +The reference for the runtime-view path is `plugins/mysqlx/src/mysqlx_admin_schema.cpp` (each `load__to_runtime` callback calls `MysqlxConfigStore::install__from_admin`; each `save__from_runtime` calls `save__to_admin_table`; four free `refresh__runtime_view` callbacks are wired via `services.register_runtime_view`). --- @@ -238,7 +251,7 @@ static bool my_stop(const ProxySQL_PluginServices* services) { static const ProxySQL_PluginDescriptor descriptor = { "my_plugin", // name - PROXYSQL_PLUGIN_ABI_VERSION, // abi_version (= 2) + PROXYSQL_PLUGIN_ABI_VERSION, // abi_version (= 3) my_init, // init (Phase D) my_start, // start (Phase E) my_stop, // stop diff --git a/doc/plugin-chassis/FILE_CHANGES.md b/doc/plugin-chassis/FILE_CHANGES.md index c8f81ad89..6df6ed6d5 100644 --- a/doc/plugin-chassis/FILE_CHANGES.md +++ b/doc/plugin-chassis/FILE_CHANGES.md @@ -18,15 +18,17 @@ Sections **A–G** cover the **chassis core**. Sections **H–O** cover the **my - **Purpose:** the C++ ABI a plugin compiles against; all types are file-wide guarded by `#ifdef PROXYSQL40` so v3.x TUs see nothing. - **Key contents:** - - `PROXYSQL_PLUGIN_ABI_VERSION` (currently `2`) and `_MAX`. ABI 1 = original 6-field descriptor; ABI 2 appends `register_schemas` for the four-phase lifecycle. + - `PROXYSQL_PLUGIN_ABI_VERSION` (currently `3`) and `_MAX` (also 3). ABI 1 = original 6-field descriptor; ABI 2 appends `register_schemas` for the four-phase lifecycle; ABI 3 keeps the descriptor layout unchanged from ABI 2 and adds a single `register_runtime_view` callback at the **tail of `ProxySQL_PluginServices`** so plugins can declare admin-side projections of module state. ABI-2 plugins still load on an ABI-3 core (loader range is `[1, 3]`; the trailing services field is invisible to the older compile). - `ProxySQL_PluginDescriptor` — 7-field struct `{name, abi_version, init, start, stop, status_json, register_schemas}` returned via `extern "C" proxysql_plugin_descriptor_v1()`. - - `ProxySQL_PluginServices` — services injected into the plugin: `register_table`, `register_command`, `register_command_alias`, three snapshot stubs (always nullptr today), `log_message`, three `get_*db` getters, `register_query_hook`, `get_prometheus_registry`. Tail-append discipline. + - `ProxySQL_PluginServices` — services injected into the plugin: `register_table`, `register_command`, `register_command_alias`, three snapshot stubs (always nullptr today), `log_message`, three `get_*db` getters, `register_query_hook`, `get_prometheus_registry`, `register_runtime_view` (ABI 3, tail-appended). Tail-append discipline. - Per-phase availability matrix is documented inline above the struct (Phase B: `register_table` live, DB getters return nullptr; Phase D: full services). - Query-hook ABI — `ProxySQL_PluginQueryHookPayload/Result/Action`; one hook per protocol per plugin. + - Runtime-view ABI — `ProxySQL_PluginRuntimeView{table_name, refresh, opaque}` plus `proxysql_plugin_register_runtime_view_cb`. The refresh callback gets a borrowed `SQLite3DB*` and re-projects module state into the named admin-db table on demand. - Header explicitly calls out the `std::string` / `prometheus-cpp` C++-ABI coupling: plugins MUST share toolchain with core. - - Long footer comment encodes the empty-source sync invariant (PR #5643 lesson). + - Footer comment block encodes the **separation-of-duties contract**: LOAD reads the editable admin table and hands rows to the module's typed install API (never touches `runtime_`); SAVE dumps module state back to the editable table; `runtime_` is an admin-side projection refreshed by the registered view callback. Disk-tier copies are still plain BEGIN/DELETE/INSERT/COMMIT and still subject to the empty-source-must-clear-destination rule (PR #5643). - **Spot-check:** - Verify `register_schemas` is only dereferenced when `abi_version >= 2`. + - Verify `register_runtime_view` lives at the END of `ProxySQL_PluginServices`; older plugins compiled against the ABI-2 layout never see the field. - Verify the loader rejects `abi_version > PROXYSQL_PLUGIN_ABI_VERSION_MAX` rather than reading past its struct. ### `include/ProxySQL_PluginManager.h` — NEW, 172 lines @@ -34,10 +36,10 @@ Sections **A–G** cover the **chassis core**. Sections **H–O** cover the **my - **Purpose:** declares the in-process loader/dispatcher (`ProxySQL_PluginManager`) plus the free-standing helpers main.cpp/Admin call. - **Key contents:** - Class is move-only-ish (copy ctor/assign deleted). - - Public lifecycle: `load`, `invoke_register_schemas_phase`, `init_all`, `start_all`, `stop_all`; plus `tables(kind)`, `dispatch_admin_command`, `register_command_alias`, `resolve_alias_to_canonical`, `register_query_hook`, `has_query_hook`, `dispatch_query_hook`. - - Free-standing API: `proxysql_get_plugin_manager` (atomic load), `proxysql_load_configured_plugins` (Phase A+B, **publishes** the manager), `proxysql_init_configured_plugins` (Phase D), `proxysql_start_configured_plugins`, `proxysql_stop_configured_plugins`, plus three dispatch helpers (`dispatch_configured_plugin_admin_command`, `dispatch_configured_plugin_query_hook`, lock-free `has_configured_plugin_query_hook`) and `resolve_configured_plugin_admin_alias`. - - Internal `plugin_handle_t` carries dlopen handle + `schemas_registered/initialized/started/stopped` state flags. - - Two services structs: `services_` (Phase D) and `services_phase_b_` (Phase B, with stubbed DB getters and stubbed `register_query_hook`). + - Public lifecycle: `load`, `invoke_register_schemas_phase`, `init_all`, `start_all`, `stop_all`; plus `tables(kind)`, `dispatch_admin_command`, `register_command_alias`, `resolve_alias_to_canonical`, `register_query_hook`, `has_query_hook`, `dispatch_query_hook`, `register_runtime_view`, `refresh_runtime_views_for_query`. + - Free-standing API: `proxysql_get_plugin_manager` (atomic load), `proxysql_load_configured_plugins` (Phase A+B, **publishes** the manager), `proxysql_init_configured_plugins` (Phase D), `proxysql_start_configured_plugins`, `proxysql_stop_configured_plugins`, plus three dispatch helpers (`dispatch_configured_plugin_admin_command`, `dispatch_configured_plugin_query_hook`, lock-free `has_configured_plugin_query_hook`), `resolve_configured_plugin_admin_alias`, and `proxysql_refresh_configured_plugin_runtime_views` (admin pre-SELECT hook). + - Internal `plugin_handle_t` carries dlopen handle + `schemas_registered/initialized/started/stopped` state flags. `runtime_views_` vector holds `registered_runtime_view_t{table_name, refresh, opaque}` entries. + - Two services structs: `services_` (Phase D, full services) and `services_phase_b_` (Phase B, with stubbed DB getters and stubbed `register_query_hook`). **`register_runtime_view` is wired in BOTH structs** — plugins typically declare runtime views alongside their tables in `register_schemas`, so the callback is live in Phase B as well as Phase D. - **Spot-check:** - Lifecycle invariant: `proxysql_init_configured_plugins` must run BEFORE any worker thread takes the lock-free read path. - `resolve_alias_to_canonical` returns `std::string` by value (not borrowed `c_str()`) so a concurrent reload between resolve and dispatch can't dangle a pointer. @@ -51,7 +53,8 @@ Sections **A–G** cover the **chassis core**. Sections **H–O** cover the **my - **Purpose:** dlopen orchestration, services trampolines, lifecycle state machine, dispatch. - **Key contents:** - **Concurrency model:** `g_active_plugin_manager` is a `std::atomic`; reads/writes coordinated via `g_active_plugin_manager_mutex` (`std::shared_mutex` — readers from dispatch/resolve paths share, publishers/unpublishers take unique). A separate `std::mutex g_plugin_lifecycle_mutex` serializes load/start/stop transitions. The shared-mutex change exists explicitly to keep query-hook dispatch from collapsing per-worker thread parallelism. - - **Service trampolines** (file-static): `register_table_service`, `register_command_service`, `register_command_alias_service`, `register_query_hook_service`, `get_admindb/configdb/statsdb_service`, `log_message_service`, `get_prometheus_registry_service`. Each writes to `g_registry_target` set by a `ScopedRegistryTarget` RAII guard around plugin callbacks; `note_registration_failure` records the first failure. + - **Service trampolines** (file-static): `register_table_service`, `register_command_service`, `register_command_alias_service`, `register_query_hook_service`, `register_runtime_view_service`, `get_admindb/configdb/statsdb_service`, `log_message_service`, `get_prometheus_registry_service`. Each writes to `g_registry_target` set by a `ScopedRegistryTarget` RAII guard around plugin callbacks; `note_registration_failure` records the first failure. + - **`sql_references_table_ci()`** is the matcher used by `refresh_runtime_views_for_query`: case-insensitive whole-identifier substring match treating `[A-Za-z0-9_]` as identifier characters. So `runtime_mysqlx_users` matches in `` SELECT * FROM `runtime_mysqlx_users` `` but NOT in `runtime_mysqlx_users_extra` or `stats_runtime_mysqlx_users`. - **`load()`** (lines 324–383): rejects duplicate paths; `dlopen(RTLD_NOW|RTLD_LOCAL)`; resolves `proxysql_plugin_descriptor_v1`; rejects null/empty `name` and `abi_version` outside `[1, PROXYSQL_PLUGIN_ABI_VERSION_MAX]`. - **`invoke_register_schemas_phase()`** (lines 386–461): Phase B; only reads `descriptor->register_schemas` when `abi_version >= 2`; snapshots `tables_*`/`commands_`/storage sizes and rolls back on failure to keep retries idempotent. - **`init_all()`** has the same snapshot/rollback contract. @@ -91,11 +94,12 @@ Sections **A–G** cover the **chassis core**. Sections **H–O** cover the **my ### `lib/ProxySQL_Admin.cpp` — MODIFIED, +42 lines -- **Purpose:** exposes admin/config/stats DB handles to the loader and adds the templated dispatcher. +- **Purpose:** exposes admin/config/stats DB handles to the loader, adds the templated dispatcher, and wires the pre-SELECT runtime-view dispatch. - **Key contents:** - Three free functions `proxysql_plugin_get_admindb/configdb/statsdb()` return `GloAdmin->{admindb,configdb,statsdb}` if `GloAdmin` is non-null. Gated by `PROXYSQL40` so v3.x exports no plugin-aware symbols at all. - `ProxySQL_Admin::dispatch_plugin_admin_command(sess, sql)` builds a `ProxySQL_PluginCommandContext{admindb,configdb,statsdb}`, calls `proxysql_dispatch_configured_plugin_admin_command`, and translates `result` into `send_ok_msg_to_client` / `send_error_msg_to_client`. - Explicit template instantiations for `MySQL_Session` and `PgSQL_Session`. + - **Pre-SELECT runtime-view dispatch in `GenericRefreshStatistics`** (around line 1654): `proxysql_refresh_configured_plugin_runtime_views(query_no_space, admindb)` is called for **every admin-port query**, gated only on `if (admin)` and placed **outside** the existing `if (refresh==true)` block. The chassis itself decides whether any plugin's refresh callback fires, by per-view substring match against the query — a query that touches no registered view is a cheap no-op (one shared lock + N substring scans). The `refresh` flag is left untouched; it gates a separate set of core-only refreshes (stats_mysql_processlist, runtime_mysql_users, etc.). ### `include/proxysql_admin.h` — MODIFIED, +8 lines @@ -181,7 +185,7 @@ The harness plugin source; one .cpp builds two .sos (`fake_plugin` and `fake_plu - Implements the `ProxySQL_PluginDescriptor` entry exported via `proxysql_plugin_descriptor_v1()`. - Four-phase lifecycle: `mysqlx_register_schemas` (Phase B, services without DB), `mysqlx_init` (Phase D, DB live), `mysqlx_start`, `mysqlx_stop`. - `replace_table_atomically()` — every `admindb.execute()` return is checked; defensive ROLLBACK on any failure including post-COMMIT, addresses the v3.0 silent-wipe bug. -- `mysqlx_start()` does disk→memory→runtime sync, loads config store, clamps pool size 1..64, then drives `mysqlx_reconcile_listeners()` on the same path used by `LOAD MYSQLX ROUTES TO RUNTIME`. +- `mysqlx_start()` does `sync_disk_to_memory()` (configdb → editable mysqlx_* tables) then four `install__from_admin` calls to populate the in-memory `MysqlxConfigStore` directly from the editable admin tables — no `runtime_mysqlx_*` write happens here, since runtime tables are now admin-side projections, not a tier the plugin maintains. Then clamps pool size 1..64 and drives `mysqlx_reconcile_listeners()` on the same path used by `LOAD MYSQLX ROUTES TO RUNTIME`. - `parse_bind_addr()` handles both `host:port` and `[ipv6]:port`; default port 33060. - Descriptor is constexpr-ish and pinned to `PROXYSQL_PLUGIN_ABI_VERSION`. @@ -192,42 +196,45 @@ The harness plugin source; one .cpp builds two .sos (`fake_plugin` and `fake_plu ### `plugins/mysqlx/include/mysqlx_admin_schema.h` — NEW, 8 lines Single-symbol header declaring `mysqlx_register_admin_schema()`. -### `plugins/mysqlx/src/mysqlx_admin_schema.cpp` — NEW, 565 lines +### `plugins/mysqlx/src/mysqlx_admin_schema.cpp` — NEW - Registers DDL for all `mysqlx_*` tables and the `LOAD/SAVE` admin commands. - Four config-table pairs (memory + runtime): `mysqlx_users`, `mysqlx_routes`, `mysqlx_backend_endpoints`, `mysqlx_variables`, all with `JSON_VALID(attributes)` CHECK constraints. - Two stats-only tables: `stats_mysqlx_routes`, `stats_mysqlx_processlist`. - 8 LOAD/SAVE TO RUNTIME commands plus 8 disk variants; each has alias group (`FROM MEMORY`, `FROM MEM`, `TO RUN`, etc.) registered via `register_command_alias`. -- `copy_table()` is BEGIN/DELETE/INSERT/COMMIT with checked rollback. +- LOAD/SAVE callbacks no longer copy between admin tables. Each `load__to_runtime` callback invokes `MysqlxConfigStore::install__from_admin(admindb, err)` (read editable mysqlx_, swap into the module's in-memory state under its own lock). Each `save__from_runtime` callback invokes `MysqlxConfigStore::save__to_admin_table(admindb)` (dump module state into the editable mysqlx_ table). Disk-tier copies (LOAD/SAVE FROM/TO DISK) remain plain BEGIN/DELETE/INSERT/COMMIT between configdb and admindb. +- Four free functions `refresh_users_runtime_view` / `refresh_routes_runtime_view` / `refresh_endpoints_runtime_view` / `refresh_variables_runtime_view` are registered at schema-registration time via `services.register_runtime_view({"runtime_mysqlx_", &refresh__runtime_view, nullptr})`. Each calls `MysqlxConfigStore::project__to_runtime_view(admindb)` to wipe the destination admin-db table and refill it from the module's in-memory state. The chassis fires the relevant callback before any admin SELECT against the projected table. - `load_routes_to_runtime` calls `mysqlx_reconcile_listeners` weak-pointer if non-null. --- ## J. Mysqlx plugin: configuration runtime -### `plugins/mysqlx/include/mysqlx_config_store.h` — NEW, 101 lines -### `plugins/mysqlx/src/mysqlx_config_store.cpp` — NEW, 382 lines +### `plugins/mysqlx/include/mysqlx_config_store.h` — NEW +### `plugins/mysqlx/src/mysqlx_config_store.cpp` — NEW -- Thread-safe in-memory view of runtime tables consumed by sessions/threads. -- `MysqlxResolvedIdentity` (was `MysqlxCredentials` — renamed in `84bbdfdca`). +- **Authoritative in-memory source of truth** for X-protocol routing/identity state. Sessions/threads consume it directly; the `runtime_mysqlx_*` tables in admin_db are admin-side projections of this store, refilled on demand by the plugin's runtime-view refresh callbacks (see §I). +- `MysqlxResolvedIdentity` (was `MysqlxCredentials` — renamed in `84bbdfdca`); now carries a `comment` field. +- `MysqlxRoute` also carries a `comment` field. - `MysqlxBackendAuthMode` enum: `mapped` / `service_account` / `pass_through`. -- API: `load_from_runtime`, `resolve_identity`, `pick_endpoint`, `route_hostgroup`, `route_exists`. -- `mutable std::shared_mutex mutex_` — readers (resolve_identity, pick_endpoint, route_exists) take shared locks; load_from_runtime takes exclusive. -- `load_from_runtime()` reads `runtime_mysql_users` (canonical password/hostgroup) then `runtime_mysqlx_users` (X-specific overrides), `runtime_mysqlx_routes`, `runtime_mysqlx_backend_endpoints`, `runtime_mysql_servers WHERE status='ONLINE'`, `runtime_mysqlx_variables`. -- Endpoint overrides (mysqlx_port, use_ssl, attributes) are keyed by `(hostname, mysql_port)` and applied on top of `runtime_mysql_servers`. +- Public struct `MysqlxBackendEndpointOverride{hostname, mysql_port, mysqlx_port, use_ssl, attributes, comment}` — promoted from a file-local `MysqlxEndpointOverride` so SAVE / projection can round-trip the per-(hostname, mysql_port) overrides. +- API: per-entity install / save / project triplets — `install_users_from_admin(db, err)` / `save_users_to_admin_table(db)` / `project_users_to_runtime_view(db)`, same shape for routes / endpoints / variables; convenience `install_all_from_admin(db, err)` (test-only fixture helper); `snapshot_active_routes()` returning `vector>` for the listener reconciler; plus the read-side `resolve_identity`, `pick_endpoint`, `route_hostgroup`, `route_exists`. +- `mutable std::shared_mutex mutex_` — readers (resolve_identity, pick_endpoint, route_exists, snapshot_active_routes, project_*_to_runtime_view) take shared locks; install_*_from_admin takes exclusive. Each install is independent: LOAD MYSQLX USERS does not touch routes/endpoints/variables. +- `install_users_from_admin()` reads the editable `mysqlx_users` plus cross-module `runtime_mysql_users` (canonical password/hostgroup); `install_endpoints_from_admin()` reads the editable `mysqlx_backend_endpoints` and cross-module `runtime_mysql_servers WHERE status='ONLINE'`. None of the install paths touch any `runtime_mysqlx_*` table — those are projections, not inputs. +- `endpoint_overrides_` (the public-struct overrides) is preserved across install_endpoints calls so SAVE can round-trip and the runtime-view projection faithfully reflects what was loaded. - `pick_from_hostgroup()` supports `first_available` and `round_robin[_with_fallback]`; round-robin uses a separate `rr_mutex_` so RR state doesn't interfere with config swaps. -- Test-only `install_for_test` is unconditionally available — note it bypasses `load_from_runtime`. +- Test-only `install_for_test` is unconditionally available — note it bypasses `install_*_from_admin`. --- ## K. Mysqlx plugin: listener reconciliation -### `plugins/mysqlx/src/mysqlx_listener_reconcile.cpp` — NEW, 151 lines +### `plugins/mysqlx/src/mysqlx_listener_reconcile.cpp` — NEW -- Desired-state reconciler from `runtime_mysqlx_routes` to per-thread listener fds. +- Desired-state reconciler from the `MysqlxConfigStore` (the authoritative in-memory state) to per-thread listener fds. Signature is `mysqlx_reconcile_listeners_impl(const MysqlxConfigStore& store, ...)` — no DB handle. - 3-step algorithm: snapshot desired set → remove stale/mis-bound listeners → add missing listeners (RR over thread pool). - Bind-change detection compares `threads[tidx]->get_listener_addr_for_route()` to `host:port` string; mismatch is treated as remove+add. -- Single-admin-thread assumption is documented inline; DB snapshot is taken outside `route_to_thread_mutex`. +- Single-admin-thread assumption is documented inline; the desired set comes from `store.snapshot_active_routes()` (taken under the store's shared lock, dropped before listener fd manipulation) rather than a SELECT against `runtime_mysqlx_routes`. An inline comment calls out why we deliberately do NOT read `runtime_mysqlx_routes` here: that table is an on-demand admin-side projection of the store, only populated when admin runs a SELECT against it; reading it from the LOAD path would see empty/stale data on every startup and every `LOAD MYSQLX ROUTES TO RUNTIME` call. - RR cursor `next_rr_index` is wrapped via `((next_rr_index % pool) + pool) % pool` — defensive against negative. - **Note:** if `add_listener` fails (port in use), the route is silently not added to `route_to_thread`; operator gets no error feedback beyond the chassis-level log. Future enhancement opportunity. diff --git a/doc/plugin-chassis/REVIEW_GUIDE.md b/doc/plugin-chassis/REVIEW_GUIDE.md index c46214e00..bc243a1de 100644 --- a/doc/plugin-chassis/REVIEW_GUIDE.md +++ b/doc/plugin-chassis/REVIEW_GUIDE.md @@ -54,7 +54,7 @@ Pick one based on your time budget. Each is cumulative — the 2-hour pass conti Goal: convince yourself the chassis ABI and lifecycle are sane and the v3.x invisibility is real. Don't read mysqlx; trust that it's a consumer. 1. Read **§3 (ABI surface)** below. Cross-check against [`ABI.md`](./ABI.md). -2. Skim `include/ProxySQL_Plugin.h` (324 LOC) — note `PROXYSQL_PLUGIN_ABI_VERSION = 2`, the descriptor struct, the services struct. +2. Skim `include/ProxySQL_Plugin.h` — note `PROXYSQL_PLUGIN_ABI_VERSION = 3`, the descriptor struct (unchanged from ABI 2), the services struct (ABI 3 appends `register_runtime_view` at the tail). 3. Read **§4 (Four-phase lifecycle)** below. 4. Skim `lib/ProxySQL_PluginManager.cpp` lines **324–461** (load + register_schemas + abi_version gating) and **548–576** (stop_all pairs with init). 5. Check the v3.x invisibility claim: @@ -93,9 +93,9 @@ The chassis is defined by **two headers** plus the loader implementation. The co **Key types (in `include/ProxySQL_Plugin.h`):** -- `PROXYSQL_PLUGIN_ABI_VERSION` (currently `2`) — what newly-built plugins target. ABI 1 was the original 6-field descriptor; ABI 2 appends `register_schemas` for the four-phase lifecycle. +- `PROXYSQL_PLUGIN_ABI_VERSION` (currently `3`) — what newly-built plugins target. ABI 1 was the original 6-field descriptor; ABI 2 appends `register_schemas` for the four-phase lifecycle; ABI 3 keeps the descriptor unchanged and appends `register_runtime_view` at the tail of `ProxySQL_PluginServices` so plugins can declare admin-side projections of module state. - `ProxySQL_PluginDescriptor` — 7-field struct returned via `extern "C" proxysql_plugin_descriptor_v1()`. The single mandatory entry point a plugin must export. -- `ProxySQL_PluginServices` — services struct injected into the plugin: table/command/query-hook registration, log helper, three DB getters, prometheus registry. Tail-append discipline preserves ABI compatibility. +- `ProxySQL_PluginServices` — services struct injected into the plugin: table/command/query-hook registration, log helper, three DB getters, prometheus registry, runtime-view registration. Tail-append discipline preserves ABI compatibility (ABI-2 plugins compile against the smaller layout and still load on an ABI-3 core). **The contract of the descriptor is:** - `name` is a non-null, non-empty C string. The loader rejects anything else. @@ -191,10 +191,11 @@ client ProxySQL The mysqlx plugin demonstrates every chassis affordance: -- **Phase B** — `mysqlx_register_schemas` declares 8 admin-schema tables (`mysqlx_users`, `mysqlx_routes`, `mysqlx_backend_endpoints`, `mysqlx_variables`, plus their `runtime_*` mirrors and `stats_mysqlx_*` tables) and 16 admin commands (`LOAD MYSQLX USERS TO RUNTIME` and the 7 cousins, plus `SAVE` variants and aliases like `FROM MEMORY` / `FROM MEM` / `TO RUN`). -- **Phase D** — `mysqlx_init` performs disk-to-runtime sync of the mysqlx tables on first boot. -- **Phase E** — `mysqlx_start` clamps the thread-pool size, drives the listener reconciler from `runtime_mysqlx_routes`, and spawns N worker threads. -- **Admin command dispatch** — every `LOAD MYSQLX … TO RUNTIME` / `SAVE` lands as a callback on `mysqlx_admin_schema.cpp:copy_table()`. +- **Phase B** — `mysqlx_register_schemas` declares 8 admin-schema tables (`mysqlx_users`, `mysqlx_routes`, `mysqlx_backend_endpoints`, `mysqlx_variables`, plus their `runtime_*` admin-side projections and `stats_mysqlx_*` tables), 16 admin commands (`LOAD MYSQLX USERS TO RUNTIME` and the 7 cousins, plus `SAVE` variants and aliases like `FROM MEMORY` / `FROM MEM` / `TO RUN`), and four runtime-view refresh callbacks via `services.register_runtime_view` (one per `runtime_mysqlx_` table). +- **Phase D** — `mysqlx_init` performs disk-to-memory sync of the mysqlx tables on first boot, then loads the in-memory `MysqlxConfigStore` directly from the editable admin tables via four `install__from_admin` calls. +- **Phase E** — `mysqlx_start` clamps the thread-pool size, drives the listener reconciler from `MysqlxConfigStore::snapshot_active_routes()`, and spawns N worker threads. +- **Admin command dispatch** — `LOAD MYSQLX TO RUNTIME` lands on a callback that invokes `MysqlxConfigStore::install__from_admin`; `SAVE MYSQLX [FROM RUNTIME] TO MEMORY` invokes `MysqlxConfigStore::save__to_admin_table`. The disk-tier variants (LOAD/SAVE FROM/TO DISK) remain a plain BEGIN/DELETE/INSERT/COMMIT between configdb and admindb. +- **Runtime-view projections** — when admin runs `SELECT * FROM runtime_mysqlx_users` (or one of the four cousins), the chassis fires the registered refresh callback, which calls `MysqlxConfigStore::project__to_runtime_view(admindb)` to wipe and refill the admin table from current module state. - **Identity callbacks** — each `MysqlxSession` is given an `identity_lookup_` closure that calls back into `MysqlxConfigStore::resolve_identity()` for the username the client sends. If you can convince yourself the chassis can host the mysqlx plugin coherently, the chassis is in good shape.