99 Commits (6d8dff293973edd1be9cc67be03aac400e85cb4c)

Author	SHA1	Message	Date
Rene Cannao	e535a66ee1	fix(mysqlx): enforce per-user require_tls and allowed_auth_methods ... Two MysqlxResolvedIdentity fields were loaded from runtime_mysqlx_users into the in-memory store but never consulted by the auth path: identity_->require_tls // per-user "must be over TLS" identity_->allowed_auth_methods // per-user mechanism whitelist Net effect: - require_tls=1 did not reject MYSQL41-over-plaintext. Operators setting this column expected a hardening guarantee they did not actually get; an attacker bypassing TLS could still authenticate via MYSQL41 (which is challenge-response over plaintext, but leaks the user's full SHA1(SHA1(pw)) on a passive eavesdrop and is replayable for the duration of the auth_challenge_). - allowed_auth_methods='MYSQL41' did not reject PLAIN attempts (and vice versa). Operators expected to be able to lock down a user to a specific mechanism; the column was decorative. # Fix New helper MysqlxSession::enforce_identity_policy() called from both auth resolution sites (handle_auth_plain at line ~539, handler_auth_ challenge_sent at line ~664) immediately after identity_ resolves and before credential verification. It: - rejects with 1045 "User requires a TLS connection" when identity_->require_tls && !client_ds_.is_encrypted(). - rejects with 1045 "Authentication mechanism not allowed for user" when identity_->allowed_auth_methods is non-empty and auth_method_ is not in the comma-separated list (case- insensitive, whitespace-trimmed). Empty allowed_auth_methods preserves the historical "any wired method" default so existing rows don't require a backfill — the column already defaulted to '' in the table DDL. # What this commit does NOT cover backend_auth_mode is the third field in the same review finding (reviewer's #2). It's partially implemented today by side-effect: the backend_username being non-empty selects the service_account codepath, empty selects mapped. The pass_through mode would require forwarding the frontend AuthStart frame unmodified to the backend and is not implemented; it remains TBD pending issue #5693 (asymmetric TLS / AsClient) which has overlapping protocol-shape concerns. This commit deliberately scopes to the two policy checks that are unambiguously broken; backend_auth_mode pass_through gets its own commit when #5693 lands. # Verified 21 / 21 mysqlx unit tests still green. Caught by an external review pass, finding #2.	3 weeks ago
Rene Cannao	4e32f44196	fix(mysqlx): backend TLS honors endpoint use_ssl flag ... mysqlx_backend_endpoints.use_ssl is documented as forcing TLS on the proxy↔backend connection regardless of frontend TLS state. The flag was loaded into MysqlxBackendEndpoint at install_endpoints_from_admin time (mysqlx_config_store.cpp::load_endpoint_overrides applies the override on top of runtime_mysql_servers), but resolve_backend_target then copied only hostname and mysqlx_port into the session's per- target fields. The use_ssl flag was silently dropped. Backend TLS at line 1186 was gated only on client_ds_.is_encrypted() — frontend state. So: - plaintext client + endpoint use_ssl=1 → backend was plaintext (BUG) - TLS client + endpoint use_ssl=0 → backend was TLS (BUG, less severe — the client opted into TLS but the operator chose not to) The operator's intent (`mysqlx_backend_endpoints.use_ssl`) was simply not honored. # Fix Capture ep.use_ssl into a new target_use_ssl_ session field at resolve_backend_target. Reset it to false alongside target_address_/ target_port_ in the existing two reset paths. At the backend-auth setup site, gate the SSL_CTX install on `target_use_ssl_ || client_ds_.is_encrypted()`. Both signals independently force backend TLS: - target_use_ssl_ is the operator-mandated posture (network-zone / compliance / cert-pinning concerns). - client_ds_.is_encrypted() is the existing "match what the client did" heuristic. The two are AND-required together would have been wrong (operators mandating backend TLS shouldn't have plaintext clients defeat it); the OR is the right combination. # Out of scope - Asymmetric TLS / AsClient mode where the proxy mirrors the client's choice on the backend with no operator override is tracked separately in #5693. - TLS passthrough (forward raw TLS records without decryption) is #5692. - The full multi-mode mysqlx_tls_backend_mode (DISABLED, PREFERRED, REQUIRED, AS_CLIENT) is also #5693 — this commit just stops the silent drop of the per-endpoint flag, restoring the documented behaviour. Other modes remain TBD. # Verified 21 / 21 mysqlx unit tests still green. Caught by an external review pass, finding #3.	3 weeks ago
Rene Cannao	bd49725a6e	fix(mysqlx): pool reuse decision must precede reset() ... Mysqlx_Thread::return_connection_to_cache called conn->reset() *before* deciding whether to keep the connection. MysqlxConnection::reset() unconditionally sets in_transaction_=false, has_prepared_stmt_=false, reusable_=true, then returns. Net effect: - A session that called set_has_prepared_statement(true) on its backend (mysqlx_session.cpp:762, the PREPARE_PREPARE handler) had that signal wiped seconds later when return_backend_to_pool ran. The pool then handed the same backend — still holding the prepared statement — to the next session that asked for a backend with matching {hostgroup, user, schema}. The next session would see "ER_UNKNOWN_STMT_HANDLER" or worse, a misrouted execute. - The dead-socket paths (handler_waiting_server_msg lines 884-895 after EOF or read error, handler_session_reset_waiting lines 945-951) all called return_backend_to_pool() unconditionally. reset() didn't mark the connection dead, so a closed fd ended up in the cache. The next checkout would hit ECONNRESET / EBADF on its first read. # Fix 1. Reorder: return_connection_to_cache now consults is_reusable() FIRST. If false, delete; only if true do we reset() and cache. 2. Tighten is_reusable() with state-machine disqualifiers that catch failure paths callers might have forgotten to translate into set_reusable(false): state_ == ERROR_STATE / CLOSED and auth_state_ == BACKEND_AUTH_ERROR. The previous version checked only in_transaction_ and has_prepared_stmt_ as soft disqualifiers plus the reusable_ flag. 3. Mark backend_conn_->set_reusable(false) explicitly at every dead-socket call site in MysqlxSession (lines that previously just called return_backend_to_pool when the backend fd was -1 or read_from_net returned 0 / -1). Belt-and-suspenders: even if a future is_reusable() refactor narrows the state-based check, the explicit caller signal still correctly disqualifies dead sockets. # Why is_reusable doesn't check fd_<0 Several unit-test fixtures construct MysqlxConnection objects with fd_=-1 placeholders (e.g. test_backend_reset_clears_auth, test_ connection_multiplexing). Adding fd_<0 to is_reusable would have broken those tests without catching any production path that the state checks don't already cover — production set_reusable(false) calls + state_ updates on the failure paths are the right source of truth for "this connection cannot be reused." # Verified 21 / 21 mysqlx unit tests still green, including the connection- multiplexing and pool-reuse fixtures that would surface a regression in is_reusable() shape. Caught by an external review pass, finding #1.	3 weeks ago
Rene Cannao	9c84ae4550	fix(mysqlx): clear data-stream revents after handler runs ... process_all_sessions() in plugins/mysqlx/src/mysqlx_thread.cpp:252 gates handler() invocation on whether a poll event has landed: short c_rev = sess->client_ds().get_revents(); short s_rev = sess->server_ds().get_revents(); bool fd_ready = (c_rev != 0) || (s_rev != 0); if (fd_ready || buffered || sess->to_process) { sess->to_process = true; rc = sess->handler(); } The intent of this gate (per the inline comment immediately above) is "only invoke handler() when there is real work — forcing to_process on every tick burned the CPU at large session counts." But there was no code path that cleared the per-stream revents_ field after handler() consumed the event. The only existing reset is MysqlxDataStream::reset() (full stream tear- down). So once a session received one POLLIN, the same revents bitmask stayed set for the rest of its life: every poll cycle would re-enter handler() and hit EAGAIN, defeating the entire CPU-saving change. The hot loop scaled with session count. Fix: set_revents(0) on both streams immediately after handler returns. The next iteration only fires on a fresh poll event (or buffered work / explicit to_process), which is what the inline comment promised. Caught by an external review pass, finding #4. Verified locally: 21/21 mysqlx unit tests still green, including mysqlx_concurrent_unit-t (the 20-client burst that originally caught the listener O_NONBLOCK regression — would also surface here if the readiness clear introduced a stall).	3 weeks ago
Rene Cannao	f08206a2f9	build(plugins/mysqlx): mirror top-level feature-tier cascade ... A standalone `make -C plugins/mysqlx PROXYSQLGENAI=1` previously set only -DPROXYSQLGENAI; PROXYSQL40, PROXYSQL31, PROXYSQLFFTO, PROXYSQLTSDB all stayed unset because the four `ifeq ($(PROXYSQL_X),1)` blocks were independent. The resulting .so saw a different ProxySQL_PluginServices / ProxySQL_PluginDescriptor layout than libproxysql.a (which set every flag via the top-level cascade), and the chassis loader then read past the end of the plugin's static descriptor. The Makefile comment already warned about this and asked the caller to export every flag. CI works because workflow YAMLs list all five explicitly. The footgun was the standalone path. Mirror the top-level Makefile's cascade ladder before the per-flag -DPROXYSQL_X assignments: PROXYSQLGENAI=1 ⇒ PROXYSQL40=1 ⇒ PROXYSQL31=1 ⇒ PROXYSQLFFTO=1 + PROXYSQLTSDB=1 Now `make -C plugins/mysqlx PROXYSQLGENAI=1` produces the right five-flag set, matching the top-level build. Caught by an external review pass, finding #6.	3 weeks ago
Rene Cannao	becbf09ffa	fix(mysqlx): plugin_descriptor visibility + admin_tables test + dead decl ... Three issues code-review caught. # Plugin-descriptor visibility plugins/mysqlx/Makefile builds the .so with -fvisibility=hidden. The loader resolves proxysql_plugin_descriptor_v1 via dlsym(); without an explicit visibility-default override on this single exported function, the symbol stays hidden and ProxySQL_PluginManager::load() fails with "undefined symbol: proxysql_plugin_descriptor_v1". Add the __attribute__((visibility("default"))) override so the .so exports exactly one symbol — the descriptor entry point — and no others. Latent regression unblocked here (test_mysqlx_plugin_load-t was also affected). # test_mysqlx_admin_tables-t.cpp rewrite This integration-style TAP test (registered as unit-tests-g1, @proxysql_min_version:4.0) used to assert on runtime_mysqlx_<X> contents after each LOAD/SAVE call: mgr.dispatch_admin_command(ctx, "LOAD MYSQLX USERS TO RUNTIME", result); ok(admin_db.return_one_int("SELECT COUNT(*) FROM runtime_mysqlx_users") == 1, ...); Under the new architecture, LOAD updates MysqlxConfigStore and never writes runtime_mysqlx_users. The runtime view is repopulated only via the chassis pre-SELECT hook in ProxySQL_Admin::GenericRefreshStatistics — a path the test bypasses by reaching admin_db directly. The test also seeded only mysqlx_users without seeding runtime_mysql_users, so the cross-module-join in install_users_from_admin would silently drop every user. And the SAVE-after-modifying-runtime tests were asserting on a no-op (SAVE no longer reads runtime_mysqlx_users). Rewrite: - LOAD assertions now go via rows_affected from the callback and DELETE editable + SAVE roundtrip to confirm the in-memory store really held the rows. If SAVE re-materialises the rows the operator deleted from the editable table, they came from the store — exactly the contract LOAD is meant to install. - Cross-module canonical tables seeded explicitly: helper seed_canonical_tables creates runtime_mysql_users + runtime_mysql_servers via the canonical ADMIN_SQLITE_RUNTIME_MYSQL_USERS / ADMIN_SQLITE_TABLE_RUNTIME _MYSQL_SERVERS DDL. - SAVE-after-runtime-mutation tests (lines 263+) rewritten: instead of "modify runtime_mysqlx_users, then SAVE, expect change in mysqlx_users", they affirmatively assert the new contract — runtime-view edits do NOT propagate to mysqlx_users on SAVE. - backend_auth_mode values normalised to canonical enum values (service_account, pass_through, mapped) since mysqlx_backend_auth_mode_from_string collapses unknowns to `mapped` and would have hidden any round-trip bug otherwise. 45 / 0 asserts after rewrite. # Dead declaration MysqlxConfigStore::rebuild_hostgroup_endpoints_locked() declared in the header but never defined or called. Drop it. Code review found this in the same pass.	4 weeks ago
Rene Cannao	b4127156ed	fix(mysqlx): listener reconciler reads MysqlxConfigStore, not runtime view ... Code-review finding on PR #5688. mysqlx_reconcile_listeners_impl in plugins/mysqlx/src/mysqlx_listener_reconcile.cpp was still issuing a SELECT against runtime_mysqlx_routes to build its desired route set: SELECT name, bind FROM runtime_mysqlx_routes WHERE active=1 After the previous commit decoupled module state from the runtime_mysqlx_<X> projection, that table is empty until an admin SELECT triggers the projection callback. The reconciler runs from two non-admin paths: - mysqlx_plugin.cpp::mysqlx_start() at process startup, after install_routes_from_admin populates the store - load_routes_to_runtime() admin command, after install_routes_from_admin populates the store Both paths fire BEFORE any admin SELECT has projected the runtime view, so the reconciler would see ZERO desired routes. Net effect in production: no mysqlx listeners bind on startup, and LOAD MYSQLX ROUTES TO RUNTIME silently removes any listeners that were already mapped (the reconciler treats every mapped route as "no longer desired"). The unit tests didn't catch this because the weak symbol mysqlx_reconcile_listeners is null in unit-test binaries (the listener-reconcile pure variant is the one tested, and previously it was reading a pre-populated SQLite table). # Fix Read the desired route set from MysqlxConfigStore directly via a new public method: std::vector<std::pair<std::string,std::string>> MysqlxConfigStore::snapshot_active_routes() const It returns a (name, bind) pair for every active route under a shared lock. The reconciler consumes the snapshot the same way the previous SQL-driven version consumed result rows. The pure variant signature changes from void mysqlx_reconcile_listeners_impl(SQLite3DB& admindb, ...) to void mysqlx_reconcile_listeners_impl(const MysqlxConfigStore&, ...) which both restores the function's "no global state" purity (it no longer needs to reach into mysqlx_context() and never has) and makes the data-flow obvious: the store is the input. The strong weak-symbol entry mysqlx_reconcile_listeners(SQLite3DB&) is unchanged externally — it still takes admindb (now ignored, kept for ABI stability) and grabs the store from mysqlx_context() internally. Doc comment on the weak hook in include/mysqlx_plugin.h updated to describe the new data flow and to spell out explicitly why we do NOT read runtime_mysqlx_routes. # Test updates The two robustness tests that exercise the reconciler (test_listener_reconciliation, test_listener_reconciliation_bind_ change) used to set up a SQLite admindb with runtime_mysqlx_routes rows. They now construct a MysqlxConfigStore directly and populate it via install_for_test() — a smaller, more honest fixture that matches what the reconciler actually consumes. Note: MysqlxConfigStore is non-copyable/non-movable, so the bind-change test installs into two stack-local stores back-to- back via a helper lambda rather than rebuilding by value. # Status - All 21 mysqlx unit tests still green (646 ok, 0 not_ok). - Listener reconciler now sees the right state in both startup and LOAD MYSQLX ROUTES TO RUNTIME paths. Closes the BLOCKER from the PR #5688 review. Other concerns the review surfaced were all dismissed as not-bugs (ABI 1/2/3 loader compat, sql_references_table_ci whole-identifier match, lock ordering, sqlite_quote correctness, save semantics matching the canonical pattern).	4 weeks ago
Rene Cannao	9da7300afe	fix(mysqlx): Admin/module separation for runtime config tables ... Previously, plugin-chassis stored authoritative mysqlx runtime state inside admin-db tables (runtime_mysqlx_users / _routes / _backend_ endpoints / _variables). LOAD/SAVE commands shuffled rows between mysqlx_<X> and runtime_mysqlx_<X> via plain INSERT ... SELECT, and MysqlxConfigStore::load_from_runtime read the runtime_<X> tables back out into its in-memory map. The data lived in three places (editable mysqlx_<X>, persistent runtime_mysqlx_<X>, in-memory store) with no detection of skew between them. Issue #5687. The canonical pattern (mysql_users / GloMyAuth / runtime_mysql_users in lib/ProxySQL_Admin.cpp::__refresh_users / save_mysql_users_runtime _to_database) keeps Admin and the module strictly separated: Admin owns the editable configuration table and provides a runtime_<X> view of module state Module owns runtime state in its own data structures runtime_<X> is rebuilt on demand from module state, not persistent storage This commit restructures the mysqlx plugin to match. # MysqlxConfigStore: per-entity install / save / project triplets Replaces the monolithic load_from_runtime() with three independent operations per entity (users / routes / endpoints / variables): install_<X>_from_admin(db, err) LOAD <X> TO RUNTIME path. SELECT the editable mysqlx_<X> table (and the cross-module runtime_mysql_users / runtime_mysql_servers projections where applicable), build a new local representation, atomically swap into the in-memory store under the store's mutex. save_<X>_to_admin_table(db) SAVE <X> [FROM RUNTIME] TO MEMORY path. Mirror save_mysql_ users_runtime_to_database(false): mark all rows in mysqlx_<X> inactive, then upsert the live store contents with active=1. Inactive rows the operator deactivated but didn't delete are preserved. project_<X>_to_runtime_view(db) Runtime-view refresh path invoked by the chassis before any admin SELECT touches runtime_mysqlx_<X>. Mirror save_mysql_ users_runtime_to_database(true): DELETE the projected table, then INSERT live store contents. install_all_from_admin() is a convenience wrapper that runs all four in sequence; production code calls the per-entity methods so each LOAD command only touches its own slice of state, and unit tests have a single entry point that exercises the whole pipeline. # MysqlxConfigStore data-model additions - MysqlxResolvedIdentity gains `comment` (preserved through round- trip; the canonical mysql_users path also preserves comments). - MysqlxRoute gains `comment` for the same reason. - New public MysqlxBackendEndpointOverride struct (replaces the file-local MysqlxEndpointOverride that used to be in the .cpp). - New endpoint_overrides_ map: per-(hostname,mysql_port) overrides preserved verbatim across LOAD calls so SAVE can round-trip and so the runtime-view projection can faithfully reflect what was loaded. Previously these overrides were dropped after being folded into hostgroup_endpoints_. # Plugin: register_runtime_view + rewritten LOAD/SAVE callbacks In mysqlx_admin_schema.cpp: - Removes copy_table() and reload_config_store(); they were the INSERT...SELECT shovel between editable and runtime tables that encoded the architectural mistake. - Each load_<X>_to_runtime callback now calls install_<X>_from_ admin(*ctx.admindb, err) and never touches runtime_mysqlx_<X>. - Each save_<X>_from_runtime callback now calls save_<X>_to_admin_ table(*ctx.admindb) and never reads runtime_mysqlx_<X>. - rows_affected on LOAD now reports the active row count in the editable table (the source); on SAVE it reports the row count in the editable table after the dump (the destination). - Adds four refresh_<X>_runtime_view free functions and registers them via services.register_runtime_view() during schema registration. The chassis invokes these before any admin SELECT against the projected table. In mysqlx_plugin.cpp::mysqlx_start(): - Drops copy_to_runtime() entirely (the old "copy editable mysqlx_<X> to runtime_mysqlx_<X> at startup" step that no longer has a purpose). - Replaces the single load_from_runtime call with the four install_<X>_from_admin calls so a failure in one entity is surfaced individually in the log. - Keeps sync_disk_to_memory() unchanged. Disk-tier persistence is legitimate admin behaviour; only the runtime-tier copy was wrong. # Net effect at the SQL level - INSERT INTO runtime_mysqlx_<X> ... no longer happens on any LOAD or SAVE. The only writes to runtime_mysqlx_<X> are the on-demand projections, triggered by the chassis pre-SELECT refresh path. - SELECT FROM runtime_mysqlx_<X> always reflects current MysqlxConfigStore state, even if the operator never ran LOAD since the last edit to mysqlx_<X>. - mysqlx_<X> remains the editable table the operator writes to. The bug-stay-out-of-runtime contract documented in include/ProxySQL_Plugin.h (the rewritten doc block) now matches what this plugin actually does.	4 weeks ago
Rene Cannao	6844a1cf4c	fix(mysqlx): set listener fd to non-blocking ... Mysqlx_Thread::add_listener() called socket() / bind() / listen() but never set O_NONBLOCK on the resulting fd. The accept-drain loop in accept_new_connection() (line 213) is structured as while (true) { int client_fd = accept(listener_fd, ...); if (client_fd < 0) { if (errno == EAGAIN || errno == EWOULDBLOCK) break; ... } // ... create session ... } i.e. it relies on accept() returning EAGAIN once the kernel's accept queue is empty. With a blocking listener fd the syscall instead blocks inside the kernel waiting for the next connection, freezing the poll-driven thread: - already-accepted sessions never get their POLLIN serviced, so CapabilitiesGet/handshake bytes the client wrote sit unread in the kernel receive buffer (Recv-Q=5 in `ss -tnp`) - the signal_pipe wakeup written by stop() is never observed, so the listener never shuts down cleanly under load. Reproducer: mysqlx_concurrent_unit-t (20 simultaneous CapabilitiesGet clients). Without this fix, hangs deterministically — accept() drains ~6 of the 20 queued connections then blocks; the test eventually times out at the join() loop. With it, the test completes 20 handshakes in ~600 ms. Stress-tested locally: 16,000 handshakes across 10 runs of a custom harness (3x200 back-to-back bursts + a 1000-client mega-burst + a mid-burst stop()), zero failures, zero deadlocks. signal_pipe_[0] already gets the same O_NONBLOCK treatment in init() — this just extends the same protocol to the listener fd. Production impact: not just a test bug. Any burst of clients arriving faster than one round of poll() -> accept_new_connection() -> process_all_sessions() can drain would freeze the thread until the *next* incoming connection unblocks accept(); under steady-state load the freeze persists arbitrarily long, starving every accepted-but-not- yet-handshaken session, and stop() from the admin path cannot make the thread exit.	4 weeks ago
Rene Cannao	2b0c2fdcc4	fix(mysqlx): four minor security findings from issue #5676 ... Addresses the four Minor / Nit findings deferred when the Important re-auth bug was fixed in commit 09c15d6d5. ## (1) Reject TLS upgrade post-auth or on encrypted channel `handler_capabilities_set` previously honored `tls=true` regardless of session state, so a logged-in client could ship CapabilitiesSet with tls=true and the server would dutifully start an SSL handshake on an already-authenticated session — or, worse, a second handshake on an already-encrypted channel that left the state machine in "expecting fresh handshake" while the client kept its existing TLS session. Both cases are spec violations (the X Protocol forbids TLS negotiation post-AuthenticateOk) and produce a confused-client kill. Fix: reject with code 5052 (FATAL) when client_ds_.is_encrypted() is true OR status_ has progressed past the pre-auth states. The TLS branch only runs from CONNECTING_CLIENT / X_CAPABILITIES_GET / X_CAPABILITIES_SET now. ## (2) Cap leading-NOTICE drain in backend auth `read_auth_frame` loops while the backend ships NOTICE frames, on the assumption that any legitimate backend emits at most a couple during auth. A compromised or malicious backend could pin the worker thread by streaming NOTICEs continuously. Add a cap of 64 leading notices (MAX_LEADING_NOTICES); on the 65th, treat as auth failure (BACKEND_AUTH_ERROR) and let the chassis tear the session down. ## (3) Per-session pre-auth CapabilitiesGet/Set replay cap A client can reset back to CONNECTING_CLIENT after each CapabilitiesGet/Set without ever authenticating, and each iteration allocates a fresh protobuf and runs the full handler. Without a cap this is a free CPU/memory amplifier (bounded only by the OS recv buffer). Add a per-session counter (pre_auth_cap_msgs_) bumped on every CapabilitiesGet/Set seen while status_ != WAITING_CLIENT_XMSG. Threshold MAX_PRE_AUTH_CAP_MSGS = 64 (legitimate clients send 1-3); on overflow, send 5008 + healthy=false. Counter resets on session reset() and is bypassed once auth completes (post-auth introspection of capabilities is legitimate and unbounded). ## (4) MysqlxDataStream::enqueue_frame: surface oversize as parse error `if (body_len + 1 > X_MAX_PAYLOAD_SIZE) return;` previously dropped oversize bodies silently, leaving the client expecting a frame that never arrives. Set parse_error_=true so the session loop tears the connection down cleanly. Reaching this branch is an internal bug (server-side senders are responsible for never producing a >16MiB body), but a clean teardown is materially better than a half-working session. ## Verified After this commit (PROXYSQLGENAI=1 NOJEMALLOC=1 WITHASAN=1): - mysqlx_session_unit-t 60 / 0 (was 60 / 0) - mysqlx_robustness_unit-t 74 / 0 (was 74 / 0) - mysqlx_compression_unit-t 64 / 0 (was 64 / 0) - mysqlx_protocol_unit-t 42 / 0 (was 42 / 0) - mysqlx_data_stream_unit-t 18 / 0 (was 18 / 0) - ASAN: 0 errors These are defense-in-depth changes; none addresses an exploitable crash. All five tests pass with no regressions.	4 weeks ago
Rene Cannao	09c15d6d54	fix(mysqlx): reject re-auth on active session; fix two pre-existing test issues ... ## (A) Re-authentication on an active session is now rejected Addresses the Important finding from the security re-review of PR #5651 (see https://github.com/sysown/proxysql/issues/5676). After a successful login the session is in WAITING_CLIENT_XMSG. Before this commit, dispatch_client_message routed `SESS_AUTHENTICATE_START` and `SESS_AUTHENTICATE_CONTINUE` into the auth handlers unconditionally. The handlers overwrote `username_`, `identity_`, `target_hostgroup_`, `target_address_`, `target_port_` — but they did NOT tear down `backend_conn_`. The next StmtExecute was forwarded over the previous user's pooled backend connection. The proxy then audited the query as user B while the backend executed it as user A's role — a real identity-coherence / audit hazard. The X Protocol uses `Mysqlx::Session::Reset` for re-auth on the same connection, not direct re-auth. Reject `SESS_AUTHENTICATE_START` / `SESS_AUTHENTICATE_CONTINUE` when `status_ == WAITING_CLIENT_XMSG` with code 1845 (FATAL) and drop the session. Conformant with the spec. Also clears `auth_challenge_` on successful auth (defense in depth): the verified challenge is no longer reachable by a stale AuthenticateContinue replay even before the dispatch-level guard fires. The unit test `test_error_severity_non_fatal` previously took a shortcut: it manually set `status_=WAITING_CLIENT_XMSG` then sent AUTHENTICATE_START to drive the auth flow. With the re-auth rejection now in place, that shortcut hits the new guard and the test hangs waiting for an auth challenge that never comes. Updated the test to drive the auth flow naturally from CONNECTING_CLIENT (the state init() leaves the session in), which exercises the same code paths without the now-invalid pre-auth status_ override. ## (B) mysqlx_config_store_concurrent_unit-t: missing variables DDL This was the source of 6 pre-existing test failures surfaced during the ASAN run on PR #5651 (4, 11, 12, 13, 14, 15). All asserted that load_from_runtime atomically replaces previously-loaded data. They failed because the test fixture's create_runtime_db() did not create a `runtime_mysqlx_variables` table, but `load_from_runtime` queries that table near the end: SELECT variable_name, variable_value FROM runtime_mysqlx_variables When the table doesn't exist, fetch_result returns false and load_from_runtime short-circuits BEFORE swapping the new identities/ routes/endpoints into place. Every "second load replaces first" assertion silently failed because the second load never actually replaced anything — the swap never happened. This is the same bug that was fixed in `mysqlx_config_store_pure_unit-t.cpp` in commit 017496bc4. The sibling test was missed at the time. Add the DDL here too with a comment pointing at the underlying invariant. After this fix the concurrent test reports 15/15 ok. ## Pre-existing test bug: X_FAST_FORWARD reference `mysqlx_message_dispatch_unit-t.cpp:424` referenced `MysqlxSession::X_FAST_FORWARD`, which was retired together with the dormant MysqlxWorker path in commit 79cac4c97. The test failed to compile with `-DMYSQLX_TEST_BUILD` once that macro was added. Replace the comparison with `CONNECTING_CLIENT` (a still-extant earlier-than- TLS state) so the assertion's intent — "TLS states come after the basic states" — is preserved. (The other failures in this file remain pre-existing and are not addressed here; they are out of scope and tracked under issue #5679.) Verified locally with WITHASAN=1: mysqlx_session_unit-t 60 oks (2 pre-existing not-ok #33,34), mysqlx_robustness_unit-t 74/74, mysqlx_config_store_concurrent_unit-t 15/15.	4 weeks ago
Rene Cannao	55e90d1a76	fix(plugin-chassis,mysqlx): chassis read-path scaling, graceful shutdown, hardening ... Six independent items from the independent review of PR #5651, batched together because each one alone is small. 1) lib/ProxySQL_PluginManager.cpp: replace g_active_plugin_manager_mutex with a std::shared_mutex. Readers (dispatch_admin_command, dispatch_query_hook, resolve_alias_to_canonical) take a shared lock so multiple worker threads can be inside plugin callbacks concurrently; writers (publish/unpublish in load/stop) take the unique lock. The previous std::mutex serialized every plugin-callback dispatch on one mutex. Once a plugin actually wires a query hook into MySQL_Thread / PgSQL_Thread, every concurrent client query on every worker would have queued behind that mutex — silently negating ProxySQL's per-worker parallelism. The lock-free proxysql_has_configured_plugin_query_hook() didn't help, since the actual dispatch still took the lock. Switching to shared_mutex on the read path lets dispatch scale linearly. 2) plugins/mysqlx/include/mysqlx_session.h + plugins/mysqlx/src/mysqlx_session.cpp + plugins/mysqlx/src/mysqlx_thread.cpp: add MysqlxSession::shutdown_notify_client() and call it from Mysqlx_Thread::run() on the way out of the worker loop. Previously Mysqlx_Thread::stop() flipped running_=false and joined. The destructor then deleted sessions, closing their fds. Connected clients saw an unannounced TCP RST mid-response and a torn TLS record. Now the worker, on its way out, walks sessions_ and for each live session: enqueues a Mysqlx::Error frame (code 1053, "Server is shutting down", FATAL severity); flushes one write_to_net pass; if TLS is up, calls SSL_set_quiet_shutdown(1) + SSL_shutdown so the peer's TLS stack sees a proper close_notify rather than a torn record. Best-effort throughout — never blocks on unresponsive peers because the process is exiting. 3) plugins/mysqlx/include/mysqlx_session.h + plugins/mysqlx/src/mysqlx_session.cpp: remove the dead TLS_PASSTHROUGH enum value and the two corresponding branches. handler_tls_accept_init's first three lines and the `tls_mode_ != TLS_PASSTHROUGH` predicate in handler_connecting_server only ran when set_tls_mode(TLS_PASSTHROUGH) had been called, which never happened in production — the `mysqlx_tls_mode` config column is never plumbed into a session. Worse, the PASSTHROUGH branch did not actually implement an opaque pipe (it just skipped TLS termination and resumed clear-text X-Protocol parsing, which would desync any real client). Drop the value rather than carry a misleading enum that suggests a feature exists. Future passthrough work should reintroduce a properly-wired implementation. 4) plugins/mysqlx/src/mysqlx_connection.cpp: check the return value of inet_pton in start_connect; fail fast on anything that isn't a valid IPv4 dotted-quad. Previously the return was discarded. inet_pton on a hostname (or IPv6 literal, or empty string) silently left sin_addr at 0.0.0.0 — producing a connect to 0.0.0.0/INADDR_ANY rather than the intended target. Real footgun because mysqlx_backend_endpoints.hostname accepts arbitrary strings. Now: fail with ERROR_STATE so the misconfig surfaces instead of routing traffic to the wrong target. Hostname resolution is still the upstream pipeline's job; start_connect deliberately stays narrow. 5) plugins/mysqlx/src/mysqlx_data_stream.cpp: move do_ssl_handshake's 64 KiB scratch buffer from the stack to a thread_local static. ASan-instrumented builds and large-thread-pool configs can run with thread stacks tight enough that a stack-allocated 64 KiB local straddles the limit. Each Mysqlx_Thread owns its own thread_local instance so the buffer is not shared between threads. 6) plugins/mysqlx/src/mysqlx_thread.cpp: document the listener-removal semantics on remove_listener_for_route. Document that already-accepted sessions on a listener that's being removed continue running against their existing target_hostgroup_ / target_address_ / target_port_ until they finish or hit idle timeout. That matches surrounding MySQL behaviour (DROP TABLE doesn't cancel in-flight queries; ALTER doesn't kick off open prepared statements). Future change can call shutdown_notify_client on matching sessions if active disconnection becomes desirable. NOT changed: the agent-flagged "compression overshoot" issue at plugins/mysqlx/src/mysqlx_session.cpp:1283. The zstd-stream loop already caps the resize at `cap` on line 1369 (`if (new_sz > cap) new_sz = cap;`) before the resize, so decompressed never grows past the cap; there is no overshoot. Verified by tracing the loop. Skipped. Verified locally: - plugin .so builds clean with PROXYSQL40=1 and the implied tier flags. - libproxysql.a and src/proxysql build clean. - plugin chassis tests (plugin_lifecycle_unit-t, plugin_dispatch_unit-t, plugin_manager_unit-t, plugin_query_hook_unit-t) build and pass with the new shared_mutex read path. plugin_manager_unit-t shows the same 2 pre-existing destructor-related failures it had before this commit (verified by stash + rebuild). - mysqlx_robustness_unit-t passes 74/74. - mysqlx_session_unit-t has the same pre-existing failures at 33-34.	4 weeks ago
Rene Cannao	04bccec51e	chore(plugin-chassis): tighten gating, drop dead paths, gate forgery setters ... Five clean-up items from the independent review of PR #5651. None of these change behaviour on a normal run; they each fix a concrete way the current code is misleading or unnecessarily exposed. 1) lib/ProxySQL_Admin.cpp: gate the three plugin-DB-handle getters (proxysql_plugin_get_admindb / _configdb / _statsdb) under #ifdef PROXYSQL40. Previously these were defined unconditionally and emitted symbols into v3.0/v3.1 binaries. The chassis is a v4.0 feature; the user explicitly required that v3.x builds carry no plugin-aware code. Wrap in PROXYSQL40 so they are entirely absent from v3.x linkage. ProxySQL_PluginManager.cpp's extern declarations (lines 23-25) are already inside the file-wide PROXYSQL40 gate and so resolve only when the gate is active. 2) lib/ProxySQL_PluginManager.cpp + include/ProxySQL_PluginManager.h: delete the dead `#else /* !PROXYSQL40 */` branches. Both files are wrapped in a top-level `#ifdef PROXYSQL40` covering the entire body. Inner `#ifdef PROXYSQL40 ... #else ... #endif` blocks therefore had unreachable `#else` arms — 30+ lines of "pre-chassis two-phase loader" in the .cpp, plus a redundant declaration of proxysql_load_configured_plugins in the .h. The dead arms read as load-bearing alternative implementations on review and that is exactly the wrong signal. Drop them; the single PROXYSQL40 path is the only one. 3) lib/Admin_Bootstrap.cpp + include/proxysql_admin.h + src/main.cpp: remove ProxySQL_Admin::materialize_plugin_tables(). `Admin::init()` already merges plugin-declared schemas into tables_defs_{admin,config,stats} (~line 944) and runs the DDL via check_and_build_standard_tables() (~line 994), all on the same first-boot/reload code path the core tables use. The follow-up call to GloAdmin->materialize_plugin_tables() in main.cpp ran a second name-dedup pass that found everything already present and produced an empty new-rows set — i.e. the post-init helper was a no-op disguised as load-bearing infrastructure. Delete the helper, the header declaration, and the main.cpp call site, and update the comments in main.cpp + ProxySQL_PluginManager.cpp to point at Admin::init() as the single canonical materialization site. Leave a NOTE in Admin_Bootstrap.cpp at the old call site so anyone re-adding a similar helper sees why the prior one was removed. 4) plugins/mysqlx/include/mysqlx_session.h + plugins/mysqlx/src/mysqlx_session.cpp: gate the two genuine forgery vectors behind MYSQLX_TEST_BUILD. inject_identity_for_test() bypasses the full auth flow — no credential check, no capability negotiation, just sets identity_ to a caller-supplied MysqlxResolvedIdentity. resolve_backend_target_ for_test() drives a private routing helper without an authenticated identity. Both are necessary for unit tests but should not be reachable at all in shipped binaries; an in-process exploit reaching the session can call inject_identity_for_test() to forge an authenticated identity. Wrap them in #ifdef MYSQLX_TEST_BUILD; define -DMYSQLX_TEST_BUILD in the test Makefile only. The remaining target_*_for_test() getters are read-only state observers and are left unconditional — they cannot mutate the session and a debugger could observe the same state regardless. 5) test/tap/tests/unit/Makefile: define -DMYSQLX_TEST_BUILD on every unit test compile line via OPT. This is the test-only knob that re-enables the gated forgery methods so unit tests still compile. plugins/mysqlx/Makefile does NOT define this macro, so the production .so does not compile in the entry points. Verified locally: - plugins/mysqlx/ProxySQL_MySQLX_Plugin.so builds clean with PROXYSQL40=1 PROXYSQL31=1 PROXYSQLFFTO=1 PROXYSQLTSDB=1 PROXYSQLGENAI=1 (no MYSQLX_TEST_BUILD). - test/tap/tests/unit/mysqlx_robustness_unit-t builds and runs: 74/74 assertions pass, including the ones that exercise inject_identity_for_test (visible only because the test Makefile defines MYSQLX_TEST_BUILD).	4 weeks ago
Rene Cannao	4bd4b462be	fix(mysqlx): three blocking protocol/pool correctness bugs ... Three independent issues, fixed together because they all sit on the data-plane critical path and each one alone is enough to corrupt the session state machine. 1) MysqlxConnection::reset() did not scrub backend_ds_ buffers. The pooled-connection reuse path (Mysqlx_Thread::return_connection_ to_cache → reset → next session's get_connection_from_cache) calls MysqlxConnection::reset() between sessions. The previous version of reset() cleared in_transaction_, has_prepared_stmt_, reusable_, and auth_state_ but left the underlying MysqlxDataStream's read_buf_, write_buf_, ssl_write_buf_, and complete_frames_ untouched. Any straggler frame the prior session left in flight — for example a NOTICE that arrived after the terminal frame, an unread row that the prior session abandoned, or a half-parsed frame whose body had not finished arriving — would be served to the next session as if it were the response to its first query. That is a real cross-session data leak, not just a robustness issue. Adds MysqlxDataStream::clear_io_buffers() which scrubs the I/O queues without touching fd_, ssl_, rbio_ssl_, wbio_ssl_, or encrypted_. Preserving the SSL state is critical: rebuilding the SSL* would force a fresh TLS handshake on every pool checkout and would discard ALPN / cipher negotiation already done. Calls it from MysqlxConnection::reset(). 2) handler_capabilities_set silently accepted unparseable CapabilitiesSet messages. In plugins/mysqlx/src/mysqlx_session.cpp the previous logic was `if (cap_set.ParseFromArray(...))` { ... } and on parse failure fell through to the unconditional `pop_frame(); send_ok(); status_=CONNECTING_CLIENT;`. A buggy or hostile client that ships an unparseable capability payload would be told the negotiation succeeded — the server then proceeded with default capabilities while the client believed it had selected something. This is the exact kind of confused-state bug the X Protocol error codes exist to prevent. Returns send_error(5051, "Invalid CapabilitiesSet payload") on parse failure. 5051 is the X Protocol convention for an unrecognized/unparseable capability message; 5052 is reserved for "supported capability with rejected value" and is already used for the compression-value-not-supported path two blocks down. 3) step_auth_authenticate_start_sent ignored the return of AuthenticateContinue::ParseFromArray. In plugins/mysqlx/src/mysqlx_connection.cpp the backend-side auth state machine called `cont.ParseFromArray(frame->data() + 5, frame->size() - 5)` and discarded the bool. A backend (or a MITM that has bypassed TLS, or a misbehaving X-Protocol-aware proxy between us and the backend) returning a malformed AuthenticateContinue would produce an empty auth_data() field; the resulting empty challenge was then handed to mysqlx_mysql41_scramble() which would compute a scramble against a zero-length challenge — undefined-input territory. Backend auth would still fail eventually, but only after operating on uninitialized protobuf fields. Now: on parse failure, mark BACKEND_AUTH_ERROR and return -1 so the failure surfaces immediately and we never trust the half-parsed message body. Verified: plugins/mysqlx builds cleanly with PROXYSQL40=1 PROXYSQL31=1 PROXYSQLFFTO=1 PROXYSQLTSDB=1 PROXYSQLGENAI=1. Existing unit tests for protocol parsing and auth state are unchanged in expected behaviour (the new code paths trip only on malformed inputs that no current test exercises).	4 weeks ago
Rene Cannao	9f5ed235b8	fix(build,test groups): unblock CI on plugin-chassis ... Two unrelated CI blockers, fixed together because each one alone leaves the pipeline red and they are trivially independent: 1) `make cleanbuild` (and any other goal that recurses into plugins/mysqlx on a v3.0/v3.1 box without libprotobuf-dev) failed in plugins/mysqlx/Makefile because the protobuf-3.x ABI check fires at parse time. The check is correct for *building* the plugin — running pre-generated .pb.cc against an ABI-incompatible libprotobuf would produce a .so that links cleanly and crashes on first virtual dispatch — but it has no business firing for `clean`/`cleanall`, which only delete object files. Wrap the check in `ifeq ($(filter clean cleanall,$(MAKECMDGOALS)),)` so the safety guarantee is preserved on build paths and clean is now usable on a bare host. CI-builds was failing every job at Makefile:540 cleanbuild for this reason, and the failure cascaded through every dependent test workflow. 2) `mysqlx_compression_unit-t` was added in the X-Protocol Phase-1/2/3 compression commits but never registered in `test/tap/groups/groups.json`. The lint workflow `check_groups.py --source` flagged it as "executable test missing from groups.json" and exited 1, blocking the entire CI run. Add the entry to `unit-tests-g1` with the `@proxysql_min_version:4.0` tag, matching every other mysqlx unit test in the file. Also restore six MySQL test entries that lost their `mysql90-g3`, `mysql95-g3` tags during the d2e03fae3 conflict-resolution commit, plus `test_noise_injection-t` which lost the same two. Confirmed against origin/v3.0 — these were valid groupings on v3.0 that should have been preserved through the merge but were silently dropped. Verified locally: - `cd plugins/mysqlx && make clean` succeeds without libprotobuf installed (was: fatal `$(error)` at parse time). - `python3 test/tap/groups/lint_groups_json.py` → "OK (419 entries, sorted, compact)". - `python3 test/tap/groups/check_groups.py` → "OK: All 29 executable tests are registered in groups.json".	4 weeks ago
Rene Cannao	aef01ef0be	feat(mysqlx): compress outbound server frames (Phase 3) ... Phase 3 of three-phase X Protocol compression support: server→client compression on outbound frames. Wraps up the MVP — CapabilitiesSet negotiation (Phase 1), inbound decompression (Phase 2), and now outbound compression all roundtrip. What this commit does ===================== forward_frame_to_client() — the chokepoint for backend→client frame forwarding — now routes through send_to_client_compressed(), which: 1. Returns early to plain client_ds_.enqueue_frame() when compression is not negotiated OR the body is below COMPRESSION_MIN_OUTPUT_BYTES (50 bytes — same threshold the upstream MySQL X plugin uses; per- message envelope + framing overhead would otherwise dwarf savings). 2. With combine_mixed_messages = false: emits one Compression frame per body. The protobuf has server_messages set to the original frame's msg_type and uncompressed_size set to the original body length, so a spec-compliant client can decompress in place. 3. With combine_mixed_messages = true: appends a fully-framed copy of the body to compress_batch_framed_ and bumps compress_batch_count_. Once the count reaches max_combine_messages (defaulting to 64 if the client didn't supply one), or flush_compression_batch() is invoked at a natural boundary, all buffered frames are emitted as one Compression message with NEITHER server_messages nor client_messages set — payload is the concatenated stream of framed X messages, matching the spec's third payload shape. The flush points are: - handler_waiting_server_msg() when it sees a terminal frame (end of a result set / final OK / etc.) — the response is "done" so anything still buffered must reach the wire before we go back to WAITING_CLIENT_XMSG. - handler_session_reset_waiting() on the ERROR path before write_to_net(), same reasoning. Mid-response we deliberately do NOT flush — that would defeat the combine_mixed_messages benefit by emitting a Compression message per batch hit, which is exactly what we're trying to avoid for streaming result sets. The count cap bounds how long a single batch can grow. Compressor selection mirrors Phase 2: - zstd_stream uses ZSTD_compressCCtx() with a per-session ZSTD_CCtx that's lazily allocated and freed in reset_compression_state(). - lz4_message uses LZ4_compress_default() one-shot. On compressor failure (ZSTD_isError or lz4 returning <= 0) the helper falls back to enqueueing the body uncompressed — losing compression benefit beats dropping the message. The session itself stays healthy. Tests ===== mysqlx_compression_unit-t now plans 64 sub-tests, all passing: - Phase 1: 22 sub-tests for capability negotiation - Phase 2: 17 sub-tests for inbound decompression - Phase 3 (new, 25 sub-tests): - zstd round-trip: frame on the wire is COMPRESSION with server_messages = NOTICE; payload decompresses back to the exact original 200-byte body - lz4 round-trip, same shape - below-threshold passthrough: 20-byte body emitted as plain NOTICE, not COMPRESSION (verifies the fast path) - combine_mixed_messages with max_combine_messages=3: three successive sends produce ONE Compression frame whose decompressed payload contains all three NOTICE bodies in order, with neither server_messages nor client_messages set - compression-disabled passthrough: when no negotiation, the helper acts as a plain enqueue_frame() (proves a client that never opts in is unaffected) A few new test-only accessors expose the batch state and send_to_client_compressed() entry point so tests can exercise the output path without wiring a fake backend (which would block on connect() in the dispatcher anyway). Verification ============ Top-level `PROXYSQLGENAI=1 make` builds cleanly. mysqlx_compression_unit-t passes 64/64. plugin_lifecycle_unit-t (26/26) passes unchanged. mysqlx_session_unit-t still has its two pre-existing failures at sub-tests 33-34 (auth flow), unrelated to compression and present on this branch before the Phase 1 commit.	4 weeks ago
Rene Cannao	b1fd6b31fc	feat(mysqlx): decompress incoming Compression messages (Phase 2) ... Phase 2 of three-phase X Protocol compression support: client→server decompression. Compression on the server→client path (Phase 3) still goes out uncompressed; CapabilitiesSet stores the negotiation in Phase 1 and this commit consumes it. What this commit does ===================== When the client sends a Mysqlx.Connection.Compression message AND compression has been negotiated, we now: 1. Parse the Compression protobuf — payload bytes, optional uncompressed_size hint, optional client_messages tag. 2. Decompress the payload using the negotiated algorithm: - zstd_stream: streaming decompression via a ZSTD_DCtx that persists across messages on the same session (per spec — successive Compression frames may continue a single zstd stream, so we cannot recreate the context per frame). - lz4_message: one-shot LZ4_decompress_safe. The X spec defines lz4_message as one independent frame per message, so no persistent context is needed. 3. Feed the decompressed bytes back into client_ds_.feed_bytes() so the existing frame parser picks them up. Two payload shapes per spec are handled: - client_messages set: payload is one decompressed body of that type — we re-frame with a 5-byte X header in front. - neither set: payload is already a sequence of fully-framed X messages — fed verbatim. 4. The dispatch loop re-enters on the same handler tick (to_process set), so a Compression-wrapped StmtExecute runs end-to-end without an extra network roundtrip. If client_messages is unset and server_messages IS set, the message is rejected (5008): server_messages on the c→s path is always wrong. Anti-bomb / bounds ================== COMPRESSION_MAX_DECOMPRESSED_BYTES = 16 MiB caps how much output a single Compression message can produce, mirroring MysqlxDataStream's existing on-the-wire X_MAX_PAYLOAD_SIZE so a Compression frame never expands beyond what the rest of the data plane can handle anyway. If the client provides an uncompressed_size hint smaller than 16 MiB we honor that as the tighter bound. Hint of 0 with non-empty payload is treated as malformed. For zstd, the decompression loop re-checks the cap before each ZSTD_decompressStream call and bails (with 5008) the moment the output buffer would exceed it. A no-progress condition (zout.pos == 0 with input remaining) also bails — that catches malformed streams that would otherwise spin forever. For lz4, LZ4_decompress_safe(dstCapacity = cap) naturally enforces the limit: the decompressor refuses to write past dstCapacity and returns an error code we map to 5008. Linkage / build =============== The plugin .so is dlopen'd with RTLD_LOCAL, so symbols from libzstd / liblz4 that the proxysql binary may already pull in are NOT visible to the plugin. This commit links the static deps/zstd/zstd/lib/libzstd.a + deps/lz4/lz4/lib/liblz4.a archives directly into the .so so the plugin is self-contained. The session header forward-declares ZSTD_DCtx / ZSTD_CCtx so the zstd headers don't leak through include/mysqlx_session.h into other translation units. Tests ===== mysqlx_compression_unit-t now covers Phase 2 end-to-end (39 sub- tests total, all passing): - Phase 1: capability negotiation (22 sub-tests, unchanged) - decompress zstd_stream client_messages=SQL_STMT_EXECUTE, inner StmtExecute reaches dispatch (no 5008, session moves past WAITING_CLIENT_XMSG) - same for lz4_message - oversize bomb attempt: lie about uncompressed_size, send a 1 MiB-of-zeros payload — rejected with 5008 - garbage compressed payload: rejected with 5008 - sanity: COMPRESSION before negotiation still 5008 Verification ============ Top-level `PROXYSQLGENAI=1 make` builds cleanly; the plugin .so links against the static zstd + lz4 archives. plugin_lifecycle_unit-t (26/26) passes unchanged. mysqlx_session_unit-t still has its two pre-existing failures at sub-tests 33-34, identical to behavior before this commit (verified by stashing only the Phase 2 edits to plugins/mysqlx/{src,include}/* + Makefile and rebuilding). The mysqlx_thread_unit-t / mysqlx_concurrent_unit-t hangs are also pre-existing and reproduce on the unmodified branch.	4 weeks ago
Rene Cannao	f19be5f3a0	feat(mysqlx): negotiate X Protocol compression capability (Phase 1) ... Phase 1 of three-phase X Protocol compression support: capability negotiation only. The COMPRESSION message itself is still rejected with error 5008 in dispatch_client_message(); Phase 2 wires up decompression on input and Phase 3 wires up compression on output. What this commit does ===================== - send_capabilities() now advertises a `compression` capability listing the algorithms the plugin can support: zstd_stream and lz4_message. Both libraries (libzstd, liblz4) are already statically linked into libproxysql.a / pulled into the unit-test link line, so this does not introduce any new runtime dependency. - handler_capabilities_set() detects when a client sets the `compression` capability and parses the OBJECT value's sub-keys: - `algorithm` (required string) — must match an advertised value; anything else is rejected with X-Protocol error 5052 (the capability-prepare-failed code per the spec). - `server_combine_mixed_messages` / `combine_mixed_messages` (bool) - `server_max_combine_messages` / `max_combine_messages` (uint) Both spelling variants are accepted because mysql-connector-python emits the short form while libmysqlclient emits the spec form, and the upstream MySQL server tolerates both. - Negotiation outcome is stored on MysqlxSession via three new members (compression_algo_, compression_combine_mixed_messages_, compression_max_combine_messages_). Phase 2 / Phase 3 will read these to drive (de)compression. They are reset by both init() and reset() so a session reuse does not inherit stale negotiation. - Unsupported algorithms (e.g. deflate_stream) and structurally malformed values (wrong protobuf type, missing algorithm field) are rejected with a non-fatal X-Protocol Error frame — the session remains healthy so the client can either retry CapabilitiesSet or proceed without compression. Why 5052 (non-fatal) and not 5008 --------------------------------- 5008 is the runtime "compression message arrived but compression is disabled" error and stays in dispatch_client_message() for now. 5052 is the spec-defined "capability prepare failed" status used during the CapabilitiesSet handshake; treating an unknown algorithm as a capability error matches the upstream MySQL X plugin's behavior and lets compliant clients downgrade to no-compression on the same connection. Tests ===== New unit test: test/tap/tests/unit/mysqlx_compression_unit-t.cpp (22 sub-tests, all passing). Covers: - CapGet response advertises `compression` with both algorithms - CapSet zstd_stream + combine hints accepted, stored on session - CapSet lz4_message accepted, stored on session - CapSet deflate_stream rejected with 5052 (non-fatal), session remains healthy with no algorithm set - CapSet with wrong-shape compression value (scalar instead of object) rejected with 5052 Existing mysqlx_session_unit-t / plugin_lifecycle_unit-t still pass (the two pre-existing failures in mysqlx_session_unit-t at sub-tests 33-34, "auth succeeded for correct password" and "session in WAITING_CLIENT_XMSG after auth", are present on this branch before this commit and unrelated to compression — verified by stashing only the mysqlx_session.{cpp,h} edits and re-running). Build infra fix piggy-backed on this commit ------------------------------------------- test/tap/tests/unit/Makefile referenced $(SQLITE3_LDIR)/../libsqlite_rembed.a in the GenAI-tier link line, but that .a is no longer produced on this branch (it was the artifact of the now-removed sqlite-rembed Rust extension). Without removing this stale reference, no plugin/mysqlx unit test can link, which blocks verifying the new compression test alongside the existing mysqlx tests required by the task. Mirrors the upstream fix that already landed on sibling branches as commit 44a6c9d58 ("build(test): drop libsqlite_rembed.a from TAP and rag test link order"). Phase 2 (decompression) and Phase 3 (compression on output) will follow as separate commits; the COMPRESSION dispatch handler still emits 5008 until Phase 2 lands.	4 weeks ago
Rene Cannao	79cac4c976	chore(mysqlx): retire MysqlxFrontendSession, MysqlxBackendSession, X_FAST_FORWARD ... Three pieces of dead code that survived the MysqlxWorker retirement (commit 98aee7db2) and the v2 event-driven rewrite: * `MysqlxFrontendSession` and `MysqlxBackendSession` (plugins/mysqlx/ include/* + src/*): an alternative session implementation from the pre-event-driven days. Compiled into the .so but not instantiated anywhere — `Mysqlx_Thread::accept_new_connection` creates a `MysqlxSession` directly. The Phase-1 placeholder in `MysqlxFrontendSession::handler_capabilities_set` (returning "TLS capability not supported by mysqlx plugin in Phase 1") was particularly misleading on review since the real, TLS-aware `MysqlxSession::handler_capabilities_set` in mysqlx_session.cpp:202 has full TLS negotiation wired up. * `X_FAST_FORWARD` enum value, `handler_fast_forward()` declaration, dispatch case, and empty body (plugins/mysqlx/src/mysqlx_session.cpp): state was declared but no code path ever set `status_ = X_FAST_FORWARD`, so the dispatch case was unreachable and the handler body was empty. Drop the four files from `plugins/mysqlx/Makefile`'s SRCS list and delete them. Plugin .so size drops from ~9.3 MB to ~8.5 MB (approximately 800 KB of dead code eliminated). Build clean under PROXYSQLGENAI=1.	4 weeks ago
Rene Cannao	c723ede0cf	Merge remote-tracking branch 'origin/plugin-chassis' into ProtocolX-rebased ... # Conflicts: # lib/ProxySQL_PluginManager.cpp # plugins/mysqlx/Makefile # plugins/mysqlx/include/mysqlx_plugin.h # plugins/mysqlx/include/mysqlx_session.h # test/tap/groups/groups.json # test/tap/tests/test_mysqlx_listener_smoke-t.cpp # test/tap/tests/unit/Makefile	1 month ago
Rene Cannao	e462bb0cb4	fix: address PR review feedback ... Resolves in-scope review items on PR #5651. Items deemed out of scope for this PR (O(N) admin dispatch lookup, per-plugin mutex, worker backpressure, runtime-reload DDL semantics, E2E tests going through ProxySQL rather than directly to MySQL) are tracked elsewhere; items that were already fixed in earlier commits on this branch (Admin_Handler MYSQLX alias vectors, Admin_Bootstrap PROXYSQL40 gating) are acknowledged as stale. Changes: * .github/workflows/CI-mysqlx.yml: add `include/` and `src/proxysql_global.cpp` to both `sparse-checkout` blocks. The plugin Makefile discovers the repo root via `src/proxysql_global.cpp` and includes headers from `include/`; without these, `make` inside `plugins/mysqlx` climbs up to `/` or fails on missing includes. (coderabbit: "Checkout the files required by the plugin Makefile.") * .github/workflows/CI-mysqlx.yml: `fail-on-cache-miss` now depends on `github.event_name == 'workflow_run'`. Strict behavior is retained for the cache-driven pipeline that CI-trigger populates; manual `workflow_dispatch` runs no longer fail when the SHA-specific cache doesn't exist yet. (coderabbit: "Don't make manual runs depend on a pre-existing build cache.") * plugins/mysqlx/Makefile: replace the unbounded `while [ ! -f ./src/proxysql_global.cpp ]; do cd ..; done` with a bounded upward walk (up to 12 levels) that fails fast with a clear error message when the marker file is missing — e.g. on a sparse checkout that omits `src/proxysql_global.cpp`. Build output no longer appears to hang indefinitely. (coderabbit: "Prevent root discovery from hanging on sparse or invalid checkouts.") * doc/PLUGIN_API.md: update startup sequence from the old three-phase (load → init → start) description to the four-phase chassis lifecycle (load → register_schemas → admin materialize → init → start). Document the `register_schemas` descriptor field and the `abi_version` / `PROXYSQL_PLUGIN_ABI_VERSION` contract. Clarify that `stop()` pairs with `init()` (not `start()`) for teardown symmetry. (coderabbit: "Update plugin lifecycle documentation to reflect four-phase model and ABI versioning.") * plugins/mysqlx/include/mysqlx_session.h: default-initialize every field in `MysqlxCredentials`. `bool x_enabled` was indeterminate for a default-constructed instance, which would let a lookup miss produce a value that silently authenticates. In-class initializers give every field a deterministic default. (coderabbit: "Default- initialize credential fields used by auth decisions.") * plugins/mysqlx/src/mysqlx_backend_session.cpp: close `backend_fd_` when `authenticate_backend()` returns false. Previously `connect()` stored the newly-opened fd in `backend_fd_` and returned the auth result; a retry on the same `MysqlxBackendSession` would then overwrite `backend_fd_` and leak the previous socket. (coderabbit: "Close/reset `backend_fd_` when authentication fails.") * lib/Admin_Bootstrap.cpp: `materialize_plugin_tables()` now runs DDL only for the subset of plugin-declared tables that were freshly inserted into `tables_defs_*` (the rest are already materialized). DDL failures are startup-fatal: on `execute()` returning false, log the plugin kind + table name + SQL and `exit(EXIT_FAILURE)`. A partially materialized schema produces opaque runtime errors from the plugin later on, so aborting startup is the right default. (gemini + coderabbit: "Execute DDL only for newly materialized plugin tables" / "add error checking".) Not changed in this commit (deferred / out of scope / stale): * CI-mysqlx E2E tests going through ProxySQL (requires plumbing listener + admin config in the CI workflow). * O(N) admin dispatch scan → hash map. * Descriptor build-id hash for libstdc++ compat detection. * Per-plugin mutex on mutable plugin context. * Worker client-fd enqueue backpressure. * Stale plan-doc feedback on docs/superpowers/plans/*.md. Verification (clean tree): * `unset PROXYSQL*; make cleanall && make debug -j$(nproc) && make build_tap_test_debug -j$(nproc)` -- 0 errors. * `PROXYSQLGENAI=1 make cleanall && PROXYSQLGENAI=1 make debug -j$(nproc) && PROXYSQLGENAI=1 make build_tap_test_debug -j$(nproc)` -- 0 errors. * Plugin unit tests (PROXYSQL40): 48 + 52 + 26 + 46 + 10 = 182 pass.	1 month ago
Rene Cannao	3ba92815f8	fix(plugin-chassis): make chassis fully invisible in v3.x builds ... Review feedback identified that the chassis was leaking into the v3.0 stable build via unfenced references in core code: * Admin_Bootstrap.cpp had `if (auto* m = proxysql_get_plugin_manager())` and a full materialize_plugin_tables() implementation outside any PROXYSQL40 fence * proxysql_admin.h declared materialize_plugin_tables() * proxysql_glovars.hpp held GloVars.plugin_modules and proxysql_load_plugin_modules_from_config() * ProxySQL_GloVars.cpp had three unfenced .clear()/use sites * ProxySQL_Admin.cpp::dispatch_plugin_admin_command (template + two instantiations) was unfenced * src/main.cpp's GloPluginManager, LoadConfiguredPlugins, InitConfiguredPlugins, StartConfiguredPlugins, StopConfiguredPlugins, UnloadPlugins teardown, the four-phase orchestration block, and the proxysql_load_plugin_modules_from_config call were all unfenced Separately, Admin_Handler.cpp held 16 hardcoded MYSQLX alias vectors (LOAD_MYSQLX_USERS_FROM_MEMORY, etc.) and a 16-deep if-ladder that called resolve_admin_alias_to_canonical(). This embedded plugin-specific knowledge in core code -- core should know nothing about any specific plugin's commands. The agreed architecture: v3.0/v3.1 builds have NO plugin loader at all; the entire plugin chassis (including its public manager surface, its loader, and its dispatch helpers) only exists under PROXYSQL40. v3.0 core code therefore holds zero plugin-related code paths. Changes: * include/ProxySQL_Plugin.h, include/ProxySQL_PluginManager.h, lib/ProxySQL_PluginManager.cpp: wrap entire file body in `#ifdef PROXYSQL40 ... #endif`. Including the header from a v3.x translation unit yields nothing; the .cpp compiles to an empty object file. Removed redundant inner PROXYSQL40 fences inside the now-empty v3.x body of Plugin.h (kept the few that are inside the file-wide outer for clarity). PROXYSQL_PLUGIN_ABI_VERSION is now unconditionally 2 (the macro is no longer reachable from a v3.x compile, so the PROXYSQL40-vs-1 split is moot). * include/proxysql_glovars.hpp, lib/ProxySQL_GloVars.cpp: gate GloVars.plugin_modules field, all .clear() sites, the proxysql_load_plugin_modules_from_config declaration + definition. * include/proxysql_admin.h: gate materialize_plugin_tables and dispatch_plugin_admin_command<S> declarations. * lib/Admin_Bootstrap.cpp: gate `#include "ProxySQL_PluginManager.h"`, the `if (auto* m = proxysql_get_plugin_manager())` merge block, and the materialize_plugin_tables definition. * lib/Admin_Handler.cpp: delete the 16 LOAD_/SAVE_MYSQLX_* const std::vector<std::string> declarations entirely. Delete the helper resolve_admin_alias_to_canonical(). Delete the 16-deep if-ladder + the !PROXYSQL40 fallback dispatch branch. The remaining plugin dispatch is one generic call to proxysql_resolve_configured_plugin_admin_alias() inside an #ifdef PROXYSQL40 block; v3.x compiles it out entirely. Core now holds no plugin-specific knowledge of any kind. * lib/ProxySQL_Admin.cpp: gate dispatch_plugin_admin_command template body + both explicit instantiations. * src/main.cpp: gate `#include "ProxySQL_PluginManager.h"`, GloPluginManager declaration, all four wrapper functions (LoadConfiguredPlugins / InitConfiguredPlugins / StartConfiguredPlugins / StopConfiguredPlugins), the StopConfiguredPlugins call inside UnloadPlugins, the four-phase orchestration in ProxySQL_Main_init_phase2___not_started, the GloVars.plugin_modules.clear() and the proxysql_load_plugin_modules_from_config call in ProxySQL_Main_process_global_variables. v3.x builds reduce that whole block to a single ProxySQL_Main_init_Admin_module call. * plugins/mysqlx/src/mysqlx_plugin.cpp, plugins/mysqlx/src/mysqlx_admin_schema.cpp: drop the !PROXYSQL40 fallback paths. The plugin only loads under PROXYSQL40 now (no loader exists in v3.x), so the legacy combined-init code path and the 16 fallback register_command() calls were dead code. The descriptor unconditionally wires register_schemas; mysqlx_init holds only context setup. * Makefile (top-level): the four build_src_* recipes that recurse into plugins/mysqlx are wrapped in `$(if $(filter 1,$(PROXYSQL40)),...)`. v3.x builds emit "[skip] mysqlx plugin (PROXYSQL40 not set)" instead of building the .so. Clean targets stay unconditional. * test/tap/tests/unit/mysqlx_admin_alias_resolution_unit-t.cpp: delete the test (it covered resolve_admin_alias_to_canonical, which is itself deleted). Remove from unit Makefile UNIT_TESTS list. Verification: * v3.0 stable build (`make`, no flags): - Produces 147 MB src/proxysql binary - `nm src/proxysql | grep -cE 'invoke_register_schemas_phase|GloPluginManager|materialize_plugin_tables'` -> 0 - mysqlx plugin .so is correctly skipped: "[skip] mysqlx plugin (PROXYSQL40 not set)" * PROXYSQLGENAI=1 build (cascades PROXYSQL40/31/FFTO/TSDB): - Produces 200 MB src/proxysql binary with chassis symbols - mysqlx plugin .so built (9.2 MB), contains mysqlx_register_schemas * All 5 plugin unit tests pass under PROXYSQL40: plugin_config_unit-t 48/48 plugin_dispatch_unit-t 52/52 plugin_lifecycle_unit-t 26/26 plugin_query_hook_unit-t 46/46 plugin_prometheus_unit-t 10/10	1 month ago
Rene Cannao	ab9d5a1036	fix(plugin-chassis): address deep-review findings ... Follow-up to cd48c5613 (PROXYSQL40 gating commit). Multiple reviewers reported issues ranging from descriptor-layout UB to test false-positives; this commit resolves all of them. Showstoppers fixed: * plugins/mysqlx/Makefile: PROXYSQL40 (and all feature-tier flags) are now propagated from the top-level Makefile and converted to -DPROXYSQL40 in CXXFLAGS. Before this, PROXYSQL40=1 built a v4 core that dlopen'd a 6-field v3 mysqlx descriptor, and the loader then read past the end of the plugin's static struct. Top-level Makefile's four plugins/mysqlx recursive-make invocations now pass PROXYSQL40=$(PROXYSQL40); the test/tap/tests/unit mysqlx_plugin_build rule does the same. * ABI version gating (S2/S3): add PROXYSQL_PLUGIN_ABI_VERSION macro to ProxySQL_Plugin.h (1 without PROXYSQL40, 2 with). Plugins set descriptor.abi_version from this macro. Loader rejects abi_version < 1 or > PROXYSQL_PLUGIN_ABI_VERSION_MAX. The register_schemas field is only read when abi_version >= 2 -- a v1 plugin's 6-field descriptor is no longer dereferenced past its own layout even when loaded by a v4 core. ScopedRegistryTarget RAII guard replaces the inline g_registry_target = this / = nullptr pairs so exceptions from plugin callbacks can't leave the globals dirty. * Admin_Handler TOCTOU: proxysql_resolve_configured_plugin_admin_alias now returns std::string by value instead of const char*. The previous signature borrowed a c_str() from inside commands_ under the plugin manager lock; callers then released the lock before dispatch, and a concurrent reload between the two calls could dangle the pointer. Returning by value copies the canonical SQL out while the lock is still held. * test_phase_b_db_handles_are_null was a false-positive trap: it logged "null" whenever the getter pointer was null OR returned null, so a regression to get_admindb = nullptr would have silently passed. The fake plugin now emits three distinct markers (phase_b_getter_null vs phase_b_handles_null vs phase_b_handles_live); the test asserts the exact contract (non-null function pointer that returns nullptr). * Snapshot stubs contract: the header claimed Phase D services are "every field LIVE", but get_mysql_users_snapshot et al. point to a nullptr-returning stub in every phase. Header now explicitly documents that snapshot getters are not yet implemented and plugins MUST treat null return as "not available" across every phase. * stop() teardown symmetry: stop_all() previously skipped plugins that succeeded init() but failed start(). Resources allocated in init would leak. Now stop_all() runs stop() for every plugin where initialized=true, irrespective of started. proxysql_plugin_stop_cb's header docstring documents this: stop pairs with init, not start. * register_schemas partial-failure rollback: if a plugin's register_schemas registered some tables/commands and then returned false, the registry kept the partial registrations. A retry (same plugin, bug fixed) would then duplicate-fail. Both invoke_register_schemas_phase and init_all now snapshot the registration counts before invoking the plugin callback and truncate back on failure. Tightened / documented: * alias normalization delta (PROXYSQL40 collapses whitespace and trims trailing ';', !PROXYSQL40 is length-strict) is documented in canonicalize_plugin_command's docstring as an intentional UX improvement for the chassis tier, not a regression. * init-after-publish ordering invariant documented at both the publish site (proxysql_load_configured_plugins Phase-B branch) and at proxysql_init_configured_plugins. Phase D writes to commands_ / mysql_query_hook_ / pgsql_query_hook_ on the already-published manager; safe today because ProxySQL_Main_init_phase3___start_all runs strictly after Phase D, so no thread ever observes the manager during its Phase-D mutations. * Phase-D failure path documented: treated as fatal startup error (exit(EXIT_FAILURE) in main.cpp). Plugins that succeeded init() are still torn down via stop_all() during process teardown, now that stop pairs with init. Runtime reload is out of scope for this iteration. * Admin_Bootstrap.cpp materialize_plugin_tables(): removed duplicate ATTACH DATABASE calls for "disk" and "stats". init_sqlite3db() in the same startup flow attaches them first; a duplicate ATTACH errors out in SQLite. * test/tap/tests/unit/Makefile PROXYSQL31 autodetection: previously used the same probe symbol (MySQLFFTO) as PSQLFFTO, which collapsed PROXYSQL31 onto PROXYSQLFFTO. Now PROXYSQL31 is set when either PROXYSQLFFTO or PROXYSQLTSDB is detected (matching the top-level cascade's PROXYSQL31 => FFTO && TSDB). Tests added (all under PROXYSQL40 only): * plugin_lifecycle_unit-t: - test_phase_b_partial_failure_rolls_back: plugin registers a table then returns false; rollback verified by retrying the same plugin with the failure toggle cleared -- succeeds. - test_stop_runs_when_start_fails: exercises the init-success / start-failure path and asserts stop() marker in log. - test_bogus_abi_version_rejected: loads a descriptor with abi_version=99; loader rejects with "unsupported plugin ABI version" and init is not called. - plan(26), up from plan(14). * plugin_dispatch_unit-t: - test_cross_plugin_command_collision: loads both fake_plugin and fake_plugin2 as real .so's, both registering the same admin-command SQL; init fails with a message naming the collision. Complements the same-manager API-level collision test already present. - plan(52), up from plan(50). * plugin_query_hook_unit-t: - test_embedded_nul_in_query_text: dispatch with an embedded NUL in query_text asserts the full length_len byte range reaches the hook intact (not strlen-truncated). - test_reentrant_dispatch_across_protocols: mysql-hook callback re-enters dispatch_query_hook(pgsql). Asserts no deadlock and correct inner-hook invocation count. - plan(46), up from plan(41). Total plugin unit tests: 48 (config) + 52 (dispatch) + 26 (lifecycle) + 41 (prometheus) + 46 (query_hook) = 213 under PROXYSQL40; 47 + 32 = 79 under v3.1 non-chassis.	1 month ago
Rene Cannao	cd48c5613e	feat(plugin-chassis): gate chassis ABI additions behind PROXYSQL40 ... The four-phase plugin lifecycle (Step 2.2), pre-execution query-hook ABI (Step 2.1), shared Prometheus registry access (Step 2.3), and generic admin-command alias dispatch (Step 2.5) introduce new plugin ABI surface area — a new descriptor field (register_schemas), new services-struct entries (register_query_hook, get_prometheus_registry, register_command_alias), new public manager methods (invoke_register_schemas_phase, dispatch_query_hook, etc.), and a new startup ordering in main(). These changes should not appear in the v3.x stable/innovative tiers. Wrap every chassis-only declaration, definition, and dispatch path in `#ifdef PROXYSQL40` so the v3.0/v3.1 builds retain the pre-chassis behavior: a single init()-only descriptor layout, two-phase load+init lifecycle, and the hardcoded MYSQLX alias ladder in Admin_Handler.cpp. Changes: * Top-level Makefile: introduce PROXYSQL40 as a distinct tier above PROXYSQL31. Cascade rule: PROXYSQLGENAI=1 implies PROXYSQL40=1 implies PROXYSQL31=1. Bump major version number on PROXYSQL40=1 (previously tied to PROXYSQLGENAI). Export PROXYSQL40 for recursive submakes. * lib/Makefile, src/Makefile: add -DPROXYSQL40 via $(PSQL40) so the compile commands match the top-level tier selection. * include/ProxySQL_Plugin.h: gate the register_schemas descriptor field, query-hook types/enums/typedefs, the alias-registration callback, and the trailing services-struct fields (register_query_hook, get_prometheus_registry, register_command_alias). The pre-chassis descriptor remains six fields (name, abi_version, init, start, stop, status_json) so plugins built against v3.x headers still link and load. * include/ProxySQL_PluginManager.h: gate the chassis-exclusive public methods and private members (schemas_registered, aliases, services_phase_b_, query hook pointers). Provide two declarations of proxysql_load_configured_plugins — the four-phase variant (paired with proxysql_init_configured_plugins) under PROXYSQL40, the legacy single-call variant otherwise. * lib/ProxySQL_PluginManager.cpp: gate every method that belongs to the chassis (invoke_register_schemas_phase, register_command_alias, resolve_alias_to_canonical, register_query_hook, has_query_hook, dispatch_query_hook), plus the services_phase_b_ initialization (nullptr DB handle stubs), the proxysql_dispatch_configured_plugin_ query_hook entry point, and the alternate four-phase body of proxysql_load_configured_plugins / proxysql_init_configured_plugins. The legacy two-phase loader is restored under !PROXYSQL40. get_admindb_service / get_configdb_service / get_statsdb_service stay ungated because they back the unconditional services_ member. * lib/Admin_Handler.cpp: gate the `#include \"ProxySQL_PluginManager.h\"` and restore under !PROXYSQL40 the 16 hardcoded MYSQLX alias vectors plus the 16-deep if-ladder in admin_handler_command_*(). Under PROXYSQL40 the ladder is replaced by a single call to proxysql_resolve_configured_plugin_admin_alias(), which is plugin-agnostic. * src/main.cpp: gate InitConfiguredPlugins and branch the startup sequence in ProxySQL_Main_init_phase2___not_started: - PROXYSQL40: LoadConfiguredPlugins (Phase A+B) -> admin init (C) -> materialize_plugin_tables -> InitConfiguredPlugins (D) -> StartConfiguredPlugins (E). - !PROXYSQL40: admin init -> LoadConfiguredPlugins (combined load+init) -> materialize_plugin_tables -> StartConfiguredPlugins, matching the pre-chassis ordering. * plugins/mysqlx/src/mysqlx_plugin.cpp: split init under PROXYSQL40 into mysqlx_register_schemas (schema + commands) and a minimal mysqlx_init (context only). Under !PROXYSQL40 the original single mysqlx_init remains. Descriptor conditionally wires register_schemas. * plugins/mysqlx/src/mysqlx_admin_schema.cpp: under PROXYSQL40 use the reg() helper to register canonical+aliases via the new ABI; under !PROXYSQL40 fall back to 16 direct register_command() calls (aliases handled by Admin_Handler's hardcoded ladder). * test/tap/test_helpers/fake_plugin.cpp: gate fake_query_hook, fake_register_schemas, fake_descriptor_with_phase_b, the REGISTER_QUERY_HOOK block in fake_init, and the ENABLE_PHASE_B descriptor selection. The pre-chassis descriptor-returning stub is the sole return path under !PROXYSQL40. * test/tap/tests/unit/Makefile: autodetect PROXYSQL40 via invoke_register_schemas_phase (the chassis-exclusive symbol) in the library archive; add -DPROXYSQL40 to fake_plugin .so compilation so the descriptor layout matches the loader's expectations; append the three chassis-only tests (plugin_lifecycle, plugin_prometheus, plugin_query_hook) to UNIT_TESTS only when PROXYSQL40=1. * test/tap/tests/unit/plugin_config_unit-t.cpp, test/tap/tests/unit/plugin_dispatch_unit-t.cpp: compile-time-gate the chassis-only subtests, adjust plan() counts per mode (48/50 with PROXYSQL40, 47/32 without), and provide a proxysql_init_configured_plugins_compat() helper that is a no-op returning true under !PROXYSQL40 so shared test helpers work in both modes. Verification — all plugin unit tests pass in both modes: PROXYSQL40=1 build (chassis enabled): plugin_config_unit-t: 48/48 plugin_dispatch_unit-t: 50/50 plugin_lifecycle_unit-t: 14/14 plugin_query_hook_unit-t: 41/41 PROXYSQL40=0 build (v3.x behavior): plugin_config_unit-t: 47/47 plugin_dispatch_unit-t: 32/32 (plugin_lifecycle, plugin_query_hook not built)	1 month ago
Rene Cannao	51535aaeca	feat(plugin-abi): Step 2.5 — generic admin-command alias dispatch ... Replaces the hardcoded 16-deep MYSQLX alias ladder in lib/Admin_Handler.cpp with a table-driven lookup against ProxySQL_PluginManager::registered_commands. New plugins register their own canonical + alias spellings and admin dispatches them without any core change. ABI (include/ProxySQL_Plugin.h): * New callback typedef proxysql_plugin_register_command_alias_cb. * Appended register_command_alias field to ProxySQL_PluginServices. Additive, backward-compatible; plugins built against the previous layout don't read past the older tail (same tail-extension rule used in Step 2.2 for register_schemas). Manager (lib/ProxySQL_PluginManager.cpp): * registered_command_t grows an `aliases` vector of normalized alternate spellings. * ProxySQL_PluginManager::register_command_alias(canonical, alias) — rejects collision with any other command's canonical or alias; idempotent for duplicate (canonical, alias) pairs. * ProxySQL_PluginManager::resolve_alias_to_canonical(sql) — returns the canonical c_str() if the normalized `sql` matches a registered command's canonical or any of its aliases; nullptr otherwise. * dispatch_admin_command extended to match aliases too, and now passes the canonical form (not the incoming spelling) to the plugin callback — plugins match their own canonical strings only. * register_command_alias_service trampoline wires the callback into the active manager during init. Phase-B services expose this as well, consistent with register_command availability. Core dispatch (lib/Admin_Handler.cpp): * Deleted the 16 hardcoded MYSQLX alias vectors (LOAD/SAVE MYSQLX {USERS,ROUTES,BACKEND ENDPOINTS,VARIABLES} {FROM MEMORY,TO DISK}, ...) — ~65 lines. * Replaced the 16-deep if-ladder of resolve_admin_alias_to_canonical calls with a single proxysql_resolve_configured_plugin_admin_alias call. No MYSQLX-specific code left in Admin_Handler; the dispatcher is now plugin-agnostic. mysqlx plugin (plugins/mysqlx/src/mysqlx_admin_schema.cpp): * mysqlx_register_admin_schema now calls register_command_alias for every user-friendly alternate spelling that previously lived in Admin_Handler's hardcoded vectors: LOAD MYSQLX * TO RUNTIME: "FROM MEMORY", "FROM MEM", "TO RUN" SAVE MYSQLX * TO MEMORY: "TO MEM", "FROM RUNTIME", "FROM RUN" (disk commands: no aliases, as before) Tests: * plugin_dispatch_unit-t gets one new case covering the spec's requirement — two plugins registering non-conflicting commands, each with its own alias set. Asserts: canonical dispatch routes correctly; alias dispatch routes to the owning plugin's callback; resolve_alias_to_canonical returns the canonical regardless of which spelling was used to probe it; whitespace/case normalization; cross-plugin alias shadowing is rejected; duplicate (canonical, alias) registration is idempotent. plan(32) -> plan(50). No behavior change for users: the exact same set of LOAD/SAVE MYSQLX spellings continues to work; the routing now goes through the plugin manager rather than through Admin_Handler-embedded tables. A future plugin that wants to add admin commands can do so entirely from its own source — no core touch required.	1 month ago
Rene Cannao	f13463d334	feat(plugin-abi): Step 2.2 — four-phase plugin lifecycle ... Adds the register_schemas() callback to the plugin descriptor and wires the loader + main.cpp so that a plugin's schema is declared BEFORE the admin module bootstraps, and the plugin's init() runs AFTER the schema has been materialized. Phase sequence (previously two-phase: load+init): A. load() — dlopen each .so, read the descriptor. B. register_schemas() — NEW, optional; plugin calls services->register_table with its admin/config/stats table defs. Services hand has register_table live but all DB-handle getters stubbed to nullptr. C. ProxySQL_Main_init_Admin_module + GloAdmin->materialize_plugin_tables — admin bootstrap drains the pending-tables list via merge_plugin_tables (first-boot == reload). D. init() — plugin's startup logic with full services (live DB handles pointing at a schema that already contains its own tables). E. start() — plugin launches threads / accept loops. ABI (include/ProxySQL_Plugin.h): * New callback typedef proxysql_plugin_register_schemas_cb. * New descriptor field register_schemas — appended at the end of ProxySQL_PluginDescriptor so existing 6-field initializers leave it null (aggregate init value-initializes the trailing slot). Plugins that left register_schemas null keep the pre-existing two-phase behavior: Phase B is skipped and their init() is responsible for both schema registration and startup work. * Phase-by-phase services availability matrix documented next to ProxySQL_PluginServices. Manager (lib/ProxySQL_PluginManager.cpp): * invoke_register_schemas_phase(err) — public method that runs Phase B for every plugin whose descriptor sets the callback. Uses a "services_phase_b_" variant where DB-handle getters are nullptr stubs; register_query_hook returns false with a warning (hooks cannot register this early). * proxysql_load_configured_plugins() — now stops at Phase B. On success, installs the built manager as the active one so that ProxySQL_Admin::materialize_plugin_tables (which reads from the active manager's pending-tables list) sees the registered tables. * proxysql_init_configured_plugins() — new entry point, calls init_all() for Phase D. Separated from load so admin bootstrap can run between the two. Main (src/main.cpp): * Reorder in ProxySQL_Main_init_phase2___not_started: LoadConfiguredPlugins(); // Phase A + B ProxySQL_Main_init_Admin_module(...); // Phase C GloAdmin->materialize_plugin_tables(); // Phase C (materialize) InitConfiguredPlugins(); // Phase D — NEW StartConfiguredPlugins(); // Phase E * Error messages updated to name the phase that failed. mysqlx plugin: * mysqlx_init split into: - mysqlx_register_schemas() — calls mysqlx_register_admin_schema (pure services->register_table use, no DB access); - mysqlx_init() — wires the plugin context (services, config store, started flag) and returns true. * Descriptor wires the new callback, exercising the full four- phase path end-to-end. Tests: * plugin_lifecycle_unit-t (new) — four cases: both callbacks fire in order; register_schemas-null skips Phase B; Phase-B DB handles return nullptr; Phase-B failure aborts init. * plugin_config_unit-t, plugin_dispatch_unit-t, plugin_query_hook_unit-t — existing tests updated to also call proxysql_init_configured_plugins after proxysql_load_configured_plugins where init side-effects are asserted. test_init_registration_failure_aborts_load reworked: the fake plugin's invalid-table registration happens in init (not register_schemas), so load now succeeds and the failure surfaces at init time. * test_mysqlx_admin_tables / test_mysqlx_listener_smoke — add an explicit invoke_register_schemas_phase() call before init_all() so the mysqlx plugin's schema gets registered under its new split. No behavior change for plugins that didn't opt into Phase B: they still see the same services_ layout and the same {load, init, start, stop} ordering. Opt-in plugins gain DB access from init() against a schema that already contains their tables, eliminating the chicken- and-egg that required ProtocolX's fd5f02947 workaround.	1 month ago
Rene Cannao	6fe50376d5	Merge remote-tracking branch 'origin/feature/mysqlx-route-identity' into HEAD ... # Conflicts: # plugins/mysqlx/src/mysqlx_session.cpp # test/tap/tests/unit/Makefile # test/tap/tests/unit/mysqlx_robustness_unit-t.cpp	1 month ago
Rene Cannao	a2e99eed50	perf(mysqlx): only invoke handler() for sessions with real work ... process_all_sessions previously forced sess->to_process=true on every tick and unconditionally called sess->handler(), burning CPU at large idle session counts (one full state-machine traversal per session per loop iteration, regardless of whether anything had changed). Now only call handler() when at least one of these is true: - a poll event landed on the client or server data stream - the session self-flagged to_process (handler wants to re-run) - a complete frame is already buffered on either stream Also make Mysqlx_Thread::sessions_mutex_ mutable and take it in get_session_count() const. Previously const accessors that needed to lock the mutex couldn't — and the session count was read without the lock at all, racing the writer that appends/removes sessions.	1 month ago
Rene Cannao	bbe8122511	fix(mysqlx): reject auth without credential_lookup and release pooled fd ... Session auth previously accepted any client when credential_lookup_ was unset — an open-proxy fallback that turned a wiring bug into a silent security hole. Both PLAIN (handle_auth_plain) and MYSQL41 (handler_auth_challenge_response) now hard-reject with 1045 when no lookup is installed. A credential_lookup is always wired from mysqlx_thread.cpp via the config store, so there is no legitimate code path that reaches auth without one. Also tightened the credential check: password_hash must be exactly 20 bytes (MYSQL41 hash length) rather than simply non-empty, which could previously let a short or garbage hash pass CRYPTO_memcmp's min()-clamped comparison. In return_backend_to_pool(), replace the fresh-construct move-assign over server_ds_ with server_ds_.close_and_reset(). The old code constructed a new MysqlxDataStream (fd=-1, no SSL) and moved it in, whose destructor then close()d the fd that had just been returned to the pooled MysqlxConnection — silently corrupting the pool entry. close_and_reset() releases SSL/buffers without touching the fd. Unit test mysqlx_robustness_unit-t.cpp: setup_authenticated_session now installs a real credential_lookup with a known password, drives the server's AuthenticateContinue frame to extract the challenge, and computes the MYSQL41 scramble reply. This matches the new reality that auth requires a valid lookup. The previously-misnamed test_mysql41_no_credential_lookup_ accepts_any is renamed _rejects and asserts that without a lookup the session goes unhealthy and does not reach WAITING_CLIENT_XMSG.	1 month ago
Rene Cannao	6a921514cc	fix(mysqlx): protocol, data-stream and stats robustness fixes ... mysqlx_connection.cpp: Drain leading NOTICE frames in read_auth_frame() instead of returning nullopt on the first NOTICE. MySQL backends commonly emit a session-state-change notice before AuthenticateContinue or Ok, and returning nullopt caused the auth state machine to spin on try-read for the full 10s handshake timeout before completing. The two callers (step_auth_capabilities_get_sent and step_auth_capabilities_set_sent) now use the shared helper and drop their duplicated NOTICE checks. Also added a frame-size guard before reading the message-type byte. mysqlx_data_stream.{h,cpp}: Add close_and_reset() which tears down SSL/BIO state and clears every read/write buffer and parse flag without close()ing the fd. Required by mysqlx_session.cpp's return_backend_to_pool(), where the fd is owned by the pooled MysqlxConnection and must stay open after the data stream is wiped. Fix SSL_read return handling: a 0-return is a clean TLS shutdown (close_notify) and must surface as a connection close, not as a WANT_IO/retry. The previous code treated 0 and <0 identically and would loop forever on a cleanly-closed TLS peer. mysqlx_protocol.cpp: mysqlx_build_frame now rejects serialized payloads at the uint32 boundary so the +1 for the message-type byte cannot wrap to 0. This mirrors the X_MAX_PAYLOAD_SIZE clamp already applied by the inbound parser in MysqlxDataStream. mysqlx_stats.cpp: Rewrite the stats_mysqlx_routes INSERT builder to use std::string concatenation instead of a fixed 1024-byte snprintf buffer. Long route names plus escaping could overflow the buffer and the row was silently dropped without reaching the statsdb.	1 month ago
Rene Cannao	84e90e5e3c	build(mysqlx): pin protobuf ABI and harden plugin .so flags ... Detect the installed libprotobuf via pkg-config at configure time and fail fast unless it is 3.x. The vendored .pb.cc/.pb.h were generated with protoc 3.21.12, and the plugin links dynamically against the system libprotobuf — which is ABI-compatible only within the 3.x major version (4.x released with an incompatible ABI and a changed SONAME). Without this check, a .so that linked cleanly would crash the first time a virtual dispatched into the proto runtime. Add -fvisibility=hidden and -fvisibility-inlines-hidden so only the explicitly extern "C"-declared proxysql_plugin_descriptor_v1 entry point is exported. Prevents ODR collisions with the proxysql core when the .so is dlopen'd, and stops template instantiations from leaking across the boundary. Add -fstack-protector-strong unconditionally, and -D_FORTIFY_SOURCE=2 when not building under ASAN and when OPTZ is not -O0 (both conditions are incompatible with FORTIFY_SOURCE). Install the built ProxySQL_MySQLX_Plugin.so to /usr/lib/proxysql/plugins/ from the top-level Makefile install target, with matching cleanup in uninstall. Previously the plugin was built but never staged into a system location, so `make install` produced a proxysql binary that couldn't find it.	1 month ago
Rene Cannao	d0f6d8e4a8	Merge remote-tracking branch 'origin/fix/mysqlx-listener-lifecycle' into HEAD ... # Conflicts: # plugins/mysqlx/Makefile # plugins/mysqlx/src/mysqlx_plugin.cpp # test/tap/tests/unit/Makefile # test/tap/tests/unit/mysqlx_robustness_unit-t.cpp	1 month ago
Rene Cannao	fb1a0cd706	Merge remote-tracking branch 'origin/fix/mysqlx-backend-tls-post-auth' into HEAD ... # Conflicts: # test/tap/tests/unit/mysqlx_robustness_unit-t.cpp	1 month ago
René Cannaò	5ed92a7594	Merge pull request #5646 from sysown/fix/mysqlx-check-connect-poll ... fix(mysqlx): harden check_connect() poll and getsockopt handling	1 month ago
René Cannaò	9bbc0d71a4	Merge pull request #5643 from sysown/fix/mysqlx-stale-row-sync ... fix(mysqlx): sync empty source tables to overwrite stale rows	1 month ago
Rene Cannao	3d107c3bed	chore: commit pre-existing plugin manager improvements ... Improvements left uncommitted at the end of Step 0: - atomic g_active_plugin_manager pointer - reject plugin descriptors with null/empty name - assert single-threaded init phase - debug-log dispatched plugin commands - propagate mysqlx_register_admin_schema failure into init - additional plugin manager unit tests (init/start/stop fail, double load) - updated ABI doc/comments for table copy semantics, command context lifetime, services availability, and status_json storage	1 month ago
Rene Cannao	76aafdac67	fix(mysqlx): harden check_connect() poll and getsockopt handling ... Addresses the coderabbitai outside-diff finding originally surfaced on PR #5641: the previous check_connect() conflated poll() errors with timeouts, never retried on EINTR, did not check getsockopt()'s return, and accepted POLLNVAL / POLLERR / POLLHUP as though they were normal "not ready" states. The practical failure mode was slow and misleading: a fatal poll error or a bad descriptor was masked as "not ready yet" until the outer connect_timeout_ms_ (default 10s) elapsed, at which point the session reported a generic "connect failed" reason rather than the actual cause. A concurrent signal (EINTR) would extend that masked window further. Meanwhile, a failing getsockopt() with an uninspected return would leave `err` at its initializer value of 0 and optimistically transition the state machine to AUTHENTICATING, so the very next read or write on the invalid fd would fail in a less diagnostic path. What changed: - poll() is now retried on EINTR. - poll() returning -1 sets ERROR_STATE and returns -1 immediately. - POLLNVAL / POLLERR / POLLHUP in revents are treated as hard errors even if POLLOUT is also set, since the underlying socket is already in a terminal async condition. - getsockopt() return is checked; a nonzero return is now a hard error rather than a silent state advance. Testing: three new assertions in mysqlx_robustness_unit-t.cpp exercise the bad-fd error path, the happy-path connect, and the not-ready timeout path. Plan count 33 → 42. Out of scope: other poll() sites in the mysqlx plugin (the unit test helpers at the top of mysqlx_robustness_unit-t.cpp have their own poll() loops but they aren't on the backend-connect critical path); the connect_timeout_ms_ check itself, which is unchanged. Pre-existing ripple: same Makefile link-gap fix as earlier PRs — adds mysqlx_config_store.cpp to the mysqlx_robustness_unit-t link line so MysqlxConfigStore::resolve_identity() resolves when the binary links.	1 month ago
Rene Cannao	7e2f7828ae	fix(mysqlx): make sync transactions atomic on execute() failure ... sync_disk_to_memory() and copy_to_runtime() in mysqlx_plugin.cpp now check every admindb.execute() return value. On BEGIN / DELETE / INSERT / COMMIT failure the code issues a best-effort ROLLBACK, logs via services->log_message, and skips the current table. Both functions return false if any table's transaction failed, true if every replace succeeded. coderabbitai flagged this on #5642, #5643, and #5644 (outside-diff because the file appears in each PR's surrounding context). Without the return checks, a failed INSERT landing between a successful DELETE and the unconditional COMMIT silently wiped routes / users / endpoints / backend_endpoints — the transaction wrap advertised atomicity it did not deliver. The fix restores that guarantee. The shared BEGIN/DELETE/INSERT/COMMIT body is now a local helper replace_table_atomically() that takes a ProxySQL_PluginServices* so it can log via the existing plugin log_message callback; the two callers reduce to four-line loops over their table lists. mysqlx_start passes ctx.services through; its return values remain advisory (log-and-continue on a false return), intentionally — plugins shouldn't block proxysql startup on one table's sync failure. Upgrading to fail-fast is a future decision. Testing: added test_insert_failure_rolls_back in test/tap/tests/unit/mysqlx_robustness_unit-t.cpp. It seeds a dst table with a CHECK (id %% 2 = 0) constraint plus pre-existing rows, points src at odd ids, runs the atomic replace, and asserts (a) the function returns false, (b) dst still holds its original rows, (c) the row sum matches the pre-transaction state. plan(35) -> plan(39). All 39 assertions pass. Out of scope: the Makefile link-line fix from 82fe27f4b still applies; no further ripple.	1 month ago
Rene Cannao	2ffa38bc6c	fix(mysqlx): use resolved backend_username when setting up the backend connection ... The session's backend-connection setup site in handler_connecting_server now falls back to identity_->backend_username when that field is non-empty, and only reuses the frontend username_ when it is empty (the mapped-mode / default case). coderabbitai flagged on PR #5645 (outside-diff; the code originates in PR #5641): the pre-fix setup call always passed the frontend username_ to set_backend_user() while pairing it with the resolved password from identity_->backend_password. For mysqlx_users rows whose backend_auth_mode is `service_account` and whose backend_username differs from the frontend username, this mismatch paired userA's password with userB's name — backend auth failed with access-denied even though both columns were internally consistent. The route-identity refactor (earlier commits on this branch) widened the user lookup to return the full MysqlxResolvedIdentity struct, which already carries backend_username. This commit wires that existing field through at the backend-setup site; no store-side changes are required. See MysqlxBackendAuthMode in mysqlx_config_store.h for the full set of modes and their semantics. Testing: two new unit assertions covering both branches of the selector — mapped mode (empty backend_username falls back to frontend username_) and service_account mode (distinct backend_username is used verbatim). Assertion count moves from 46 to 48. A test-only getter MysqlxConnection::get_backend_user_for_test() was added so the test can observe the string that was ultimately passed to set_backend_user. Out of scope: enforcement of MysqlxBackendAuthMode semantics (for example, rejecting service_account rows that omit backend_username at load time). That validation belongs with the config-store loader and is tracked separately.	1 month ago
Rene Cannao	c78d7b859c	fix(mysqlx): reconcile bind-address changes, document single-admin-thread assumption ... The listener reconciler built its desired snapshot keyed only by route name, so editing a route's `bind` column (e.g. from `:33061` to `:33062`) and running LOAD MYSQLX ROUTES TO RUNTIME left the old listener running and never opened the new port — the removal pass saw the name still in the desired set and skipped the route entirely. The removal pass now also compares the currently-bound `host:port` on the owning thread against the desired bind and treats any mismatch as a removal; the subsequent addition pass rebinds under the new spec. To support that comparison, `Mysqlx_Thread` now stores bind ports in a parallel `listener_ports_` vector alongside the existing `listener_addrs_` and exposes `get_listener_addr_for_route()` returning the canonical `"host:port"` form. Also added a comment documenting the pre-existing single-admin-thread invariant that justifies snapshotting the DB outside of `route_to_thread_mutex` (noted in code review, not a behavioral change). Testing: new unit-test assertions exercise a full bind-change reconcile end-to-end — pick two free ports, reconcile at port1, update the runtime row to port2, reconcile again, and assert the listener is now bound at port2 with total listener count still 1. 42 assertions pass (was 38). Out of scope: changing the concurrency model. The single-admin-thread assumption is pre-existing throughout ProxySQL admin execution.	1 month ago
Rene Cannao	dd131b0aa2	fix(mysqlx): reconcile listeners at startup and on LOAD ROUTES TO RUNTIME ... Startup previously capped listener creation at `pool_size` routes via `ti < ctx.threads.size()` in mysqlx_plugin.cpp's startup loop, so any routes beyond the thread-pool size had no listener. Runtime route changes via `LOAD MYSQLX ROUTES TO RUNTIME` never touched the listener topology at all — adding, removing, or toggling a route's `active` flag had no effect until a full restart. Flagged in the ProtocolX code review as item #4. This commit: - Drops the `ti < pool_size` cap in the startup loop. - Distributes routes across threads round-robin (the original intent) using a shared plugin-scope `route_to_thread` map guarded by a mutex, with a `next_rr_index` cursor. - Adds route-name tracking to Mysqlx_Thread: a new parallel `listener_route_names_` vector alongside `listener_fds_` / `listener_addrs_`, plus a new `remove_listener_for_route(name)` method that closes the fd and prunes all three vectors. Returns true/false so callers can use it idempotently. - Adds a `mysqlx_reconcile_listeners(admindb)` desired-state reconciler. Reads active routes from `runtime_mysqlx_routes`, binds missing listeners round-robin, and removes listeners for routes that are gone or deactivated. Startup and `LOAD MYSQLX ROUTES TO RUNTIME` both go through this single path, so both agree. The reconciler is idempotent: re-running with the same desired set is a no-op. - The reconciliation core lives in a new file `plugins/mysqlx/src/mysqlx_listener_reconcile.cpp` as a pure helper `mysqlx_reconcile_listeners_impl(...)` taking state by parameter, so unit tests can drive it against a minimal fake context. The convenience wrapper that reads `mysqlx_context()` lives in plugin.cpp and is declared weak so tests of admin_schema.cpp that don't link plugin.cpp still link cleanly (admin_schema null-checks the weak pointer). Tests: mysqlx_robustness_unit-t grows from 33 to 38 assertions — four thread-API tests (add_listener with route name on two threads, remove_listener_for_route removes and is idempotent) plus one integration test that builds an in-memory admin DB with one route and asserts `mysqlx_reconcile_listeners_impl` binds exactly one listener and records the mapping. Also fixes the pre-existing unit-test Makefile link gap: four tests that include `mysqlx_thread.cpp` (which references `MysqlxConfigStore::resolve_identity`) now correctly link `mysqlx_config_store.cpp`, and the robustness test link line picks up `mysqlx_config_store.cpp` + `mysqlx_listener_reconcile.cpp` for the new coverage. Out of scope: switching the distribution strategy to SO_REUSEPORT / all-threads-on-all-routes (future design iteration); treating an in-place `active` flag toggle as anything other than remove + add.	1 month ago
Rene Cannao	3e8c3da9de	fix(mysqlx): preserve backend TLS state past auth handshake ... What: Stop rewrapping the raw backend fd in a fresh session-owned MysqlxDataStream after backend auth completes. Route every data-plane read/write (forward_to_backend, handler_waiting_server_msg, handler_session_reset_waiting, and the cached-conn attach path) through MysqlxConnection::backend_ds() instead. Why: The optional backend TLS handshake runs against the connection's backend_ds_ (see mysqlx_connection.cpp init_ssl_connect) and the resulting SSL* lives there. The prior session code discarded that SSL* by init()-ing a new plain MysqlxDataStream around the same raw fd, then used it for cleartext I/O on a socket where the server expects TLS frames. Flagged in the ProtocolX code review (item #2). Testing: mysqlx_robustness_unit-t picks up five new assertions that pin the invariant: MysqlxSession::server_ds() aliases backend_conn_->backend_ds() whenever a backend is attached, and falls back to a fd == -1 placeholder otherwise. Baseline 33, post-fix 38, all pass. Out of scope: - Connection cache semantics: cached connections keep their SSL state; MysqlxConnection::reset() already leaves backend_ds_ untouched so the invariant extends to pooled reuse. - MysqlxDataStream internals (untouched). - The dormant worker path (different PR). Pre-existing ripple: test/tap/tests/unit/Makefile mysqlx_robustness_unit-t link line was missing mysqlx_config_store.cpp, same gap other mysqlx tests have been closing as they get touched. Added it here so the test links.	1 month ago
Rene Cannao	98aee7db21	chore(mysqlx): retire dormant MysqlxWorker path and its smoke test ... Deletes the mysqlx worker implementation and the TAP smoke test that was its only caller. Also removes the corresponding entries from the plugin Makefile, the unit-tests Makefile, and the CI groups manifest. Files removed: - plugins/mysqlx/src/mysqlx_worker.cpp - plugins/mysqlx/include/mysqlx_worker.h - test/tap/tests/test_mysqlx_listener_smoke-t.cpp References cleaned: - plugins/mysqlx/Makefile: dropped mysqlx_worker.cpp from SRCS - test/tap/tests/unit/Makefile: dropped test_mysqlx_listener_smoke-t from UNIT_TESTS and deleted its build rule - test/tap/groups/groups.json: removed the smoke-test entry from the unit-tests-g1 group Why: the worker was an earlier parallel implementation of the mysqlx session path (separate accept thread + worker queue + per-listener route tracking + identity.default_route -> pick_endpoint). It never reached any call site in the production proxysql binary. The only consumers of its public API were the smoke test and the worker code itself. The route-identity fix in PR #5641 used the worker's design as a reference and implemented the fix in the active session path (Mysqlx_Thread + MysqlxSession + MysqlxConfigStore::pick_endpoint). With that landed, the dormant code is pure tech debt — reviewers repeatedly mistook its logic for the active path. Why retire the smoke test at the same time: the test exercised mysqlx_start_listeners_from_runtime_routes() which was the worker's only entry function. With the worker gone, the smoke test cannot link; keeping it would produce a broken CI target. Out of scope: no changes to the active session path, no changes to the MysqlxConfigStore or admin-schema tables, no changes to any other tests. Pure deletion + Makefile/groups cleanup.	1 month ago
Rene Cannao	82fe27f4b0	fix(mysqlx): sync empty source tables to overwrite stale rows ... What: removed `if (cnt == 0) continue;` in `sync_disk_to_memory` and `copy_to_runtime` in `plugins/mysqlx/src/mysqlx_plugin.cpp`. Also removed the now-dead `SELECT COUNT(*)` + result unpacking that those skip conditions depended on. Why: the original skip was meant as an optimization for "nothing to copy" but it was a correctness bug. If a user emptied a mysqlx table in `main` and saved to disk, on next restart `sync_disk_to_memory` would see disk count == 0, skip the replace, and leave stale rows in `main.*`. Same gap for main -> runtime. The BEGIN/DELETE/INSERT/COMMIT sequence is already atomic and correctly no-ops when the source is empty (DELETE dest, INSERT zero rows). Testing: added a unit test at `test/tap/tests/unit/mysqlx_robustness_unit-t.cpp` exercising the "empty source overwrites stale dest" invariant. Also added `mysqlx_config_store.cpp` to the `mysqlx_robustness_unit-t` link line so the test binary resolves `MysqlxConfigStore::resolve_identity` (a latent build gap uncovered while validating the fix). Assertion count 33 -> 35. Out of scope: no refactor of the inline atomic-replace sequence (its atomicity is pre-existing). No changes to the admin-schema `copy_table` path, which was already unconditional.	1 month ago
Rene Cannao	e099081796	fix(mysqlx): record stats on unreachable guard, update stale comment ... What: - resolve_backend_target()'s !identity_ guard now calls mysqlx_stats().record_conn_err("", 0) so the missing-identity path does not silently drop observability. - Stale "Not yet wired" comment in mysqlx_session.h replaced with a current-state doc reflecting that resolve_backend_target() is called from the auth handlers before send_auth_ok(). Why: - Final review flagged that the spec's "all failure modes record stats" contract wasn't held on the programming-error guard. Tuple ("", 0) matches the empty-route 4000 case, which is acceptable aliasing since both legitimately lack route context. - Comment update prevents future confusion about whether resolve is integrated (Task 4 is merged). Out of scope: no behavior change, no test change. Assertion count remains 46.	1 month ago
Rene Cannao	74e678006b	feat(mysqlx): wire resolve_backend_target() into auth flow ... Both auth paths (PLAIN and MYSQL41) now call resolve_backend_target() after credential verification and BEFORE send_auth_ok(). On a routing failure (empty default_route -> 4000, unknown route -> 4001, no backend -> 4002) the session emits an X-Protocol Error frame, transitions to X_SESSION_CLOSING, and never puts an Ok frame on the wire. The MYSQL41 branch also keeps to_process=true on failure so the state machine drives the session to closed on the next handler tick. Fixes the reported bug where sessions would complete auth with target_address_ == "" and target_port_ == 0 on cache miss, then attempt to connect to "":0 on the first client query. Testing: - New integration test test_plain_auth_empty_default_route_closes_session drives PLAIN auth (TLS-marked client_ds) with an identity whose default_route is empty; asserts session reaches X_SESSION_CLOSING, is_healthy() is false, and the client sees a Mysqlx ERROR frame (not an Ok). - setup_authenticated_session now stands up a process-global MysqlxConfigStore + Mysqlx_Thread with a "default_test_route" fixture and pre-seeds the session's identity_ via the existing inject_identity_for_test hook. Pre-seeding identity_ directly (instead of set_identity_lookup) preserves the helper's existing all-zero-scramble contract by keeping identity_lookup_ unset, so the auth handler's credential verification is still skipped -- matching test_mysql41_no_credential_lookup_accepts_any's open-proxy invariant while still supplying a resolvable route. - test_mysql41_no_credential_lookup_accepts_any is updated the same way: attached to the default test thread + pre-seeded identity. Its "without identity_lookup, any scramble accepted" assertion is unchanged. - Plan bumped 43 -> 46; all 46 assertions pass. Out of scope (future tasks): the dormant MysqlxWorker path, cache key/resolver-key semantics, and listener wiring.	1 month ago
Rene Cannao	14c5d68260	feat(mysqlx): add resolve_backend_target() method on session ... Introduces MysqlxSession::resolve_backend_target(), a private method that translates the authenticated user's identity_->default_route into the concrete (target_hostgroup_, target_address_, target_port_) triple that handler_connecting_server needs to reach the backend. The method reads identity_->default_route, looks up the hostgroup + endpoint via the thread's MysqlxConfigStore, and returns 0 on success; on failure it returns 4000/4001/4002 (empty default_route / unknown route / no-backend respectively), emits an X-Protocol Error frame, records a stats miss via mysqlx_stats().record_conn_err(), and marks the session unhealthy. Why: fixes the design gap where sessions populated target_* fields only from a pool cache lookup and otherwise connected to "" on port 0 (see docs/superpowers/specs/2026-04-17-mysqlx-route-identity-design.md). The pre-Ok timing is deliberate — once the X-Protocol Ok frame is on the wire, a routing failure cannot be cleanly reported, so resolution must run before send_auth_ok(). Invariants preserved: resolve_backend_target() is not yet wired into the auth flow. Task 4 owns that step. This commit is therefore a pure addition — no production call site touches the new method; only the new unit tests exercise it, via test-only public accessors (inject_identity_for_test / resolve_backend_target_for_test / target_*_for_test). Mysqlx_Thread gains get_config_store() so the session can reach the store; MysqlxStatsStore gains reset_for_test() and get_last_conn_err_for_test() so the stats-recording contract can be asserted without a live SQLite stats DB. Out of scope: wiring resolve_backend_target() into handle_auth_plain and handler_auth_challenge_response (Task 4); removing the dormant worker path (Task 5); reconciling error codes 4000/4001/4002 with the project-wide ProxySQL error-code policy (follow-up commit). Test coverage (plan bumped +7, now 43 assertions): - test_routing_happy_path — rc=0, target fields populated - test_routing_no_default_route — rc=4000 - test_routing_unknown_route — rc=4001 - test_routing_no_backend — rc=4002 (route exists, 0 endpoints) - test_routing_stats_on_failure — (route,hg) tuples for all 3 modes - test_routing_unknown_user — identity_lookup nullopt => unhealthy Also updates mysqlx_session_unit-t / mysqlx_thread_unit-t / mysqlx_message_dispatch_unit-t / mysqlx_concurrent_unit-t Makefile rules to link mysqlx_config_store.cpp + mysqlx_stats.cpp, which were already required transitively by mysqlx_thread.cpp::resolve_identity() since Task 2 but had not been wired into the unit-test link set.	1 month ago
Rene Cannao	f0a8655be3	refactor(mysqlx): replace MysqlxCredentials with MysqlxResolvedIdentity ... The session's identity-lookup callback now returns the full MysqlxResolvedIdentity struct from the config store rather than a stripped-down MysqlxCredentials. Password-hash derivation moves into the session via a private derive_stored_hash helper (handles both "*HEX40" and cleartext forms). No behavior change; the existing credential-lookup guard semantics are preserved so sessions without a lookup still follow the open-proxy path exercised by the robustness tests. mysqlx_robustness_unit-t passes unchanged (36/36).	1 month ago
Rene Cannao	ca6ddb860c	feat(mysqlx): add route_exists() predicate to MysqlxConfigStore ... Needed to distinguish unknown-route from route-with-no-backend when resolving a session's backend target. route_hostgroup() returns 0 in both cases and can't be used for disambiguation.	1 month ago
Rene Cannao	4b353ba0b1	test(mysqlx): add MysqlxConfigStore::install_for_test helper ... Adds a test-only seam on MysqlxConfigStore so unit tests can populate the store's private routes_ and hostgroup_endpoints_ maps directly, without building a runtime SQLite3DB fixture. This unblocks the upcoming route-identity unit tests that need to exercise route lookups in isolation. Also wires plugins/mysqlx/src/mysqlx_config_store.cpp into the mysqlx_robustness_unit-t link line. The file is already listed in the plugin's own Makefile but was missing from the unit-test target, so since MysqlxConfigStore::resolve_identity became a runtime dependency of Mysqlx_Thread the link had been broken.	1 month ago