feat(chassis): add register_runtime_view ABI for module-owned state

Plugins that declare admin-side runtime views of their own in-memory
state need a way to register a projection callback the chassis can
invoke when admin SELECTs against the registered table. Without this
hook, the only way for a plugin to surface its runtime state to admin
operators was to *persist* a duplicate copy of the data into admin_db,
which violates the separation of duties between Admin (owns
configuration tables and views) and the module (owns runtime state).

Refs #5687.

ABI surface (in include/ProxySQL_Plugin.h):
  - struct ProxySQL_PluginRuntimeView { table_name, refresh, opaque }
  - new services.register_runtime_view callback
  - bumps PROXYSQL_PLUGIN_ABI_VERSION to 3

The new field is appended at the end of ProxySQL_PluginServices, so
ABI-2 plugins keep working — they neither set nor read past the
previous layout. ABI 3 plugins set abi_version=3 and use the new
field. Loader accepts ABI 1, 2, and 3.

Plumbing (in lib/ProxySQL_PluginManager.cpp):
  - register_runtime_view_service() free fn wires through to the
    manager's runtime_views_ vector
  - ProxySQL_PluginManager::register_runtime_view() rejects empty
    table names, null callbacks, and duplicate registrations
  - sql_references_table_ci() does whole-identifier substring match,
    so a SELECT on `runtime_mysqlx_users` doesn't fire the refresh
    for `runtime_mysqlx_users_extra` if both ever coexist
  - refresh_runtime_views_for_query() iterates registered views and
    invokes the matching refresh callbacks
  - proxysql_refresh_configured_plugin_runtime_views() is the
    Admin-callable wrapper that takes the manager shared lock

Admin-handler integration (in lib/ProxySQL_Admin.cpp):
  - the existing pre-SELECT refresh block (where runtime_mysql_users
    triggers save_mysql_users_runtime_to_database(true), etc.) now
    also calls proxysql_refresh_configured_plugin_runtime_views().
    This is the SAME refresh point the canonical core tables use, so
    plugin views are guaranteed to refresh before the admin query
    actually executes against admindb.

Doc block at the bottom of ProxySQL_Plugin.h rewritten to spell out
the separation-of-duties contract explicitly, replacing the previous
"copy_table guidance" that incorrectly endorsed plugins persisting
their runtime state to admin_db tables.

This commit is the foundation only — no plugin uses the new API yet.
The follow-up commits convert the mysqlx plugin's four entity pairs
(users / routes / endpoints / variables) to the canonical pattern.
fix/mysqlx-runtime-views-separation
Rene Cannao 4 weeks ago
parent e68f72cf99
commit f42c3ee1ab

@ -23,13 +23,14 @@ namespace prometheus { class Registry; }
// ABI 1: original 6-field descriptor (name, abi_version, init, start,
// stop, status_json). Pre-chassis build.
// ABI 2: appends `register_schemas` (four-phase lifecycle, PROXYSQL40).
//
// A v3 core (built without PROXYSQL40) only understands ABI 1 and MUST
// reject ABI>=2 plugins — it would read past the end of its own struct
// definition. A v4 core accepts ABI 1 plugins by treating the
// register_schemas field as if null (never dereferenced on ABI 1).
#define PROXYSQL_PLUGIN_ABI_VERSION 2u
#define PROXYSQL_PLUGIN_ABI_VERSION_MAX 2u
// ABI 3: same descriptor layout as ABI 2; ProxySQL_PluginServices grows
// a `register_runtime_view` field at the end so plugins can
// declare admin-side projections of module state. Plugins that
// stay on ABI 2 keep working — they simply don't see the new
// field in their compiled-against struct, and core never
// dereferences past the ABI-2 layout for them.
#define PROXYSQL_PLUGIN_ABI_VERSION 3u
#define PROXYSQL_PLUGIN_ABI_VERSION_MAX 3u
enum class ProxySQL_PluginDBKind : uint8_t {
admin_db = 0,
@ -166,6 +167,48 @@ using proxysql_plugin_register_query_hook_cb =
// scraped immediately.
using proxysql_plugin_get_prometheus_registry_cb =
prometheus::Registry* (*)();
// Runtime-view projection (ABI 3 extension).
//
// Mirrors the canonical mysql_users / runtime_mysql_users pattern: Admin
// owns the editable table (e.g. mysqlx_users), the plugin module owns
// the runtime state (e.g. MysqlxConfigStore::identities_), and the
// runtime_<table> in admin_db is an Admin-side *view* projected on
// demand from the module. The view holds no persistent rows — it is
// wiped and refilled by the plugin's refresh callback before any admin
// SELECT touches it.
//
// Plugins call register_runtime_view(view) during register_schemas or
// init. Each registration ties an admin-db table name (e.g.
// "runtime_mysqlx_users") to a refresh callback. Admin's pre-SELECT
// hook walks every registered view and invokes the callback for any
// that the SQL references.
//
// The refresh callback gets a borrowed admindb handle and is expected
// to do (typically) `BEGIN; DELETE FROM <table>; INSERT/REPLACE INTO
// <table> ...; COMMIT;` from the module's in-memory state. The chassis
// passes admindb explicitly (rather than the plugin reaching for it via
// services.get_admindb) because the same callback might be invoked
// against snapshot DBs in the future without the global-getter detour.
//
// Lifetime: the chassis deep-copies `table_name`, so the plugin need
// not keep the pointed-to string alive. The callback pointer itself
// must point at a function with static lifetime (typically a free
// function in the plugin .so). `opaque` is plugin-owned; it is passed
// back unchanged on each invocation. Plugins that don't need it should
// pass nullptr.
//
// Returns true on successful registration, false if `table_name` is
// already registered (by this or another plugin) or if `refresh` is
// nullptr.
struct ProxySQL_PluginRuntimeView {
const char *table_name;
void (*refresh)(SQLite3DB *admindb, void *opaque);
void *opaque;
};
using proxysql_plugin_register_runtime_view_cb =
bool (*)(const ProxySQL_PluginRuntimeView &);
#endif /* PROXYSQL40 */
// Services provided to plugins across the four-phase lifecycle.
@ -219,6 +262,16 @@ struct ProxySQL_PluginServices {
// plugins avoid the MYSQLX-specific hardcoded alias ladder that
// previously lived in lib/Admin_Handler.cpp.
proxysql_plugin_register_command_alias_cb register_command_alias;
// ABI-3 extension: declare an admin-side view of module state
// (canonical pattern, mirrors Admin's own runtime_mysql_users /
// save_mysql_users_runtime_to_database flow). See the contract
// next to ProxySQL_PluginRuntimeView above. May be nullptr in
// services_phase_b_ — register_runtime_view is live both during
// register_schemas() and init(); plugins typically register views
// at the same point they register their tables, so the callback
// is wired in both phases.
proxysql_plugin_register_runtime_view_cb register_runtime_view;
#endif /* PROXYSQL40 */
};
@ -285,39 +338,43 @@ struct ProxySQL_PluginDescriptor {
using proxysql_plugin_descriptor_v1_t = const ProxySQL_PluginDescriptor *(*)();
// ---------------------------------------------------------------------------
// ABI guidance: disk/memory/runtime sync — empty-source MUST still clear
// the destination.
//
// Plugins that implement a three-tier storage model (disk ↔ memory ↔
// runtime, mirroring proxysql_admin) typically copy rows from one tier
// into another via some variant of:
//
// BEGIN;
// DELETE FROM dest;
// INSERT INTO dest SELECT * FROM source;
// COMMIT;
// ABI guidance: separation of duties between Admin and the plugin module.
//
// Do NOT short-circuit this on `SELECT COUNT(*) FROM source == 0`. Early
// return on an empty source leaves stale rows in the destination — the
// exact bug that motivated PR #5643 on the mysqlx plugin, where
// Admin owns the editable, persistent configuration tables (e.g.
// mysqlx_users, mysqlx_routes). The plugin module owns the runtime
// state — typically an in-memory snapshot kept under its own mutex.
// The runtime_<table> in admin_db is NOT module storage; it is an
// Admin-side *view* of module state, projected on demand.
//
// if (disk_cnt == 0) continue; // stale rows in runtime
// if (cnt == 0) continue; // stale rows in memory
// LOAD <X> TO RUNTIME callbacks should:
// - read the editable admin table directly,
// - hand the rows to the module via a typed install API that swaps
// state under the module's own lock,
// - NOT touch runtime_<X> at all.
//
// caused "I deleted every row from mysqlx_users on disk then reloaded,
// but the runtime still has the old users" behavior across restarts.
// SAVE <X> [FROM RUNTIME] TO MEMORY callbacks should:
// - dump the module's in-memory state,
// - REPLACE INTO the editable admin table,
// - NOT read runtime_<X>.
//
// The correct invariant: after sync, `dest` contains exactly the rows
// from `source` at the moment of the transaction. An empty source
// produces an empty destination. Atomicity with ROLLBACK on any
// intermediate error keeps the destination in a well-defined state on
// failure.
// The runtime_<X> view is repopulated by a refresh callback registered
// via services.register_runtime_view(...). Admin invokes the callback
// before any SELECT against the registered table; the callback wipes
// the table and re-projects from the module's in-memory state. This
// matches the canonical MySQL_Authentication / runtime_mysql_users
// pattern in core (lib/ProxySQL_Admin.cpp::save_mysql_users_runtime_to_
// database). It does duplicate the data briefly (in module + in the
// projected admin table during a query), and that is the point: the
// module is isolated from Admin's persistence, and operators see a
// faithful snapshot whenever they query.
//
// Applies to every LOAD/SAVE command a plugin registers. If you
// register admin commands that copy between tiers (LOAD X TO RUNTIME,
// SAVE X TO DISK, etc.), their callbacks MUST NOT skip the sync on an
// empty source — run the DELETE+INSERT unconditionally inside a single
// transaction and check each execute() return.
// Disk-tier copies (LOAD X FROM DISK, SAVE X TO DISK) DO copy between
// configdb and admindb persistent tables and remain a plain
// BEGIN/DELETE/INSERT/COMMIT — there is no module involvement and no
// view projection on that path. For those copies, the empty-source-
// must-still-clear-destination rule still applies (see PR #5643): run
// the DELETE+INSERT unconditionally inside a single transaction and
// check each execute() return.
// ---------------------------------------------------------------------------
#endif /* PROXYSQL40 (file-wide) */

@ -65,6 +65,20 @@ public:
bool dispatch_query_hook(ProxySQL_PluginProtocol proto,
const ProxySQL_PluginQueryHookPayload& payload,
ProxySQL_PluginQueryHookResult& result) const;
// Runtime-view (admin-side projection of module state) plumbing.
// register_runtime_view returns false if the table is already
// registered or if the refresh callback is null.
bool register_runtime_view(const ProxySQL_PluginRuntimeView& view);
// Refresh every registered view whose table_name appears as a
// case-insensitive substring of `sql`. Each refresh callback is
// invoked exactly once per call, regardless of how many times its
// table is mentioned. Best-effort: a callback that throws or
// otherwise misbehaves is logged but does not stop other views from
// refreshing. Caller supplies admindb so the chassis does not have
// to reach into the global admin module.
void refresh_runtime_views_for_query(const std::string& sql, SQLite3DB* admindb) const;
#endif /* PROXYSQL40 */
size_t size() const;
@ -115,6 +129,18 @@ private:
// At most one hook per protocol; nullptr means "no hook".
proxysql_plugin_query_hook_cb mysql_query_hook_ { nullptr };
proxysql_plugin_query_hook_cb pgsql_query_hook_ { nullptr };
// Runtime-view registry: one entry per admin-side projection of
// module state. Stored alongside an owned table_name copy so
// callers may free the input string after registration. The
// refresh callback pointer and opaque are plugin-owned with
// static lifetime (the .so isn't unloaded while a view is live).
struct registered_runtime_view_t {
std::string table_name {};
void (*refresh)(SQLite3DB*, void*) { nullptr };
void* opaque { nullptr };
};
std::vector<registered_runtime_view_t> runtime_views_ {};
#endif /* PROXYSQL40 */
};
@ -141,6 +167,14 @@ bool proxysql_has_configured_plugin_query_hook(ProxySQL_PluginProtocol proto);
// release the manager lock before dispatching without risking pointer
// invalidation on concurrent reload.
std::string proxysql_resolve_configured_plugin_admin_alias(const std::string& sql);
// Admin-side helper: invoke every plugin runtime-view refresh callback
// whose registered table is referenced by `sql`. Used by Admin's
// pre-SELECT path, mirroring the way runtime_mysql_users is refreshed
// before its SELECTs. No-op if no plugin manager is active or no views
// match. Caller supplies admindb (typically the same handle Admin uses
// for its own runtime_mysql_users refresh).
void proxysql_refresh_configured_plugin_runtime_views(const std::string& sql, SQLite3DB* admindb);
// Phase A + B of the four-phase lifecycle: dlopen() each module, read its
// descriptor, then call register_schemas() on plugins that opted in. On
// success, `manager` is populated AND installed as the active manager so

@ -1872,6 +1872,17 @@ bool ProxySQL_Admin::GenericRefreshStatistics(const char *query_no_space, unsign
}
#endif /* PROXYSQLCLICKHOUSE */
#ifdef PROXYSQL40
// Plugin-registered runtime views (canonical pattern, mirrors
// the runtime_mysql_users refresh above): each plugin declares
// its admin-side view of module state via
// services.register_runtime_view(); the chassis dispatcher
// invokes the registered refresh callback for any view whose
// table name is referenced in this query, before the query
// runs against admindb.
proxysql_refresh_configured_plugin_runtime_views(query_no_space, admindb);
#endif /* PROXYSQL40 */
}
if (monitor_mysql_server_group_replication_log) {
if (GloMyMon) {

@ -161,6 +161,21 @@ bool register_query_hook_service(ProxySQL_PluginProtocol proto,
}
return true;
}
bool register_runtime_view_service(const ProxySQL_PluginRuntimeView& view) {
if (g_registry_target == nullptr) {
proxy_warning("Plugin runtime-view registration attempted outside init/register_schemas phase\n");
return false;
}
if (!g_registry_target->register_runtime_view(view)) {
note_registration_failure("plugin runtime view",
view.table_name != nullptr ? view.table_name : "(null)");
proxy_warning("Plugin runtime-view registration failed for %s\n",
view.table_name != nullptr ? view.table_name : "(null)");
return false;
}
return true;
}
#endif /* PROXYSQL40 */
SQLite3DB* get_admindb_service() {
@ -286,6 +301,7 @@ ProxySQL_PluginManager::ProxySQL_PluginManager() {
services_.register_query_hook = &register_query_hook_service;
services_.get_prometheus_registry = &get_prometheus_registry_service;
services_.register_command_alias = &register_command_alias_service;
services_.register_runtime_view = &register_runtime_view_service;
// Phase-B (register_schemas) services: same layout as init(), but DB
// handle getters and the query-hook registrar are stubbed -- see the
@ -306,6 +322,11 @@ ProxySQL_PluginManager::ProxySQL_PluginManager() {
// register_command() first, then register aliases. Since register_command
// is also available during Phase B, so is register_command_alias.
services_phase_b_.register_command_alias = &register_command_alias_service;
// Runtime-view registration is live during register_schemas: views are
// declared alongside tables, well before init() runs. The actual
// refresh callback won't fire until Admin handles a SELECT, by which
// point admin module bootstrap has long since completed.
services_phase_b_.register_runtime_view = &register_runtime_view_service;
#endif /* PROXYSQL40 */
}
@ -739,6 +760,65 @@ bool ProxySQL_PluginManager::dispatch_query_hook(ProxySQL_PluginProtocol proto,
result = cb(payload);
return true;
}
bool ProxySQL_PluginManager::register_runtime_view(const ProxySQL_PluginRuntimeView& view) {
if (view.table_name == nullptr || *view.table_name == '\0' || view.refresh == nullptr) {
return false;
}
for (const auto& existing : runtime_views_) {
if (strcasecmp(existing.table_name.c_str(), view.table_name) == 0) {
return false;
}
}
registered_runtime_view_t entry;
entry.table_name = view.table_name;
entry.refresh = view.refresh;
entry.opaque = view.opaque;
runtime_views_.push_back(std::move(entry));
return true;
}
namespace {
// Case-insensitive substring check matching whole identifier-like
// occurrences. We don't want a SELECT against `runtime_mysqlx_users`
// to also fire the refresh for `runtime_mysqlx_users_extra` if
// someone ever registers both. The match treats `[A-Za-z0-9_]` as
// identifier characters and requires the surrounding chars (if any)
// to be non-identifier — same convention as the sql_equals_ci
// canonicaliser used elsewhere in this file.
bool is_ident_char(unsigned char c) {
return (c >= 'A' && c <= 'Z') || (c >= 'a' && c <= 'z') ||
(c >= '0' && c <= '9') || c == '_';
}
bool sql_references_table_ci(const std::string& sql, const std::string& table_name) {
if (table_name.empty() || table_name.size() > sql.size()) {
return false;
}
for (size_t i = 0; i + table_name.size() <= sql.size(); i++) {
if (strncasecmp(sql.data() + i, table_name.data(), table_name.size()) != 0) {
continue;
}
const bool left_ok = (i == 0) || !is_ident_char(static_cast<unsigned char>(sql[i - 1]));
const size_t after = i + table_name.size();
const bool right_ok = (after == sql.size()) || !is_ident_char(static_cast<unsigned char>(sql[after]));
if (left_ok && right_ok) {
return true;
}
}
return false;
}
} // namespace
void ProxySQL_PluginManager::refresh_runtime_views_for_query(const std::string& sql, SQLite3DB* admindb) const {
for (const auto& view : runtime_views_) {
if (view.refresh == nullptr) continue;
if (!sql_references_table_ci(sql, view.table_name)) continue;
view.refresh(admindb, view.opaque);
}
}
#endif /* PROXYSQL40 */
bool ProxySQL_PluginManager::register_command(const char* sql, proxysql_plugin_admin_command_cb cb) {
@ -891,6 +971,20 @@ bool proxysql_has_configured_plugin_query_hook(ProxySQL_PluginProtocol proto) {
}
return mgr->has_query_hook(proto);
}
void proxysql_refresh_configured_plugin_runtime_views(const std::string& sql, SQLite3DB* admindb) {
// Reader: shared lock. The refresh callbacks themselves write to
// admindb (DELETE+REPLACE INTO the projected runtime_<plugin>
// table) and read from the plugin's own in-memory store under that
// store's own lock. They must not call back into the plugin
// manager (no nested lock acquisition).
std::shared_lock<std::shared_mutex> lock(g_active_plugin_manager_mutex);
ProxySQL_PluginManager* mgr = g_active_plugin_manager.load();
if (mgr == nullptr) {
return;
}
mgr->refresh_runtime_views_for_query(sql, admindb);
}
#endif /* PROXYSQL40 */
bool proxysql_load_configured_plugins(

Loading…
Cancel
Save