2282 Commits (6cdd4e697eb86982e3dfd2ceab009ead5da0efcb)

Author	SHA1	Message	Date
René Cannaò	54844df0bb	Merge branch 'v3.0' into v3.0-genai-plugin	2 months ago
Rene Cannao	6f2b69d2eb	test(mysqlx): align unit tests with new variable + response-state contracts ... Two tests that the rebase onto v3.0 surfaced as failing under ASAN-coverage CI. Both are real test/code drift introduced by earlier commits in this stack, not pre-existing baseline issues. mysqlx_admin_commands_unit-t (test 24) Commit f051c89f7 added mysqlx_tls_backend_mode as the fifth variable persisted by save_variables_to_admin_table, but the test still asserted COUNT(*) == 4. Bump to 5 and update the comment listing the well-known variables. mysqlx_robustness_unit-t (3 tests, 7 sub-asserts) The response-state-machine refactor (commits 76850980 + 44df2d8e + b99f9cc5) split is_terminal_for_state into is_frame_allowed + is_terminal_frame, with both predicates now switching on a more granular response_state_ matrix (RESP_WAITING_STMT_EXECUTE, _CRUD, _PREPARE_*, _CURSOR_*, …). Default RESP_IDLE rejects every backend frame, so the three tests below no longer observed terminal/non-terminal/pipeline behaviour — the handler dropped the frame as unsolicited before any of the post-handler asserts ran. Fix: position each test in RESP_WAITING_STMT_EXECUTE via the set_response_state_for_test() backdoor exposed exactly for this purpose. The state choice is principled — it's the only one that admits all three frame types each test sends (SQL_STMT_EXECUTE_OK as terminator, ColumnMetaData as non- terminal resultset frame, and the ColumnMetaData → Row → STMT_EXECUTE_OK pipeline). Affected tests: test_server_response_terminal_frame test_server_response_non_terminal_keeps_waiting test_server_response_multi_frame_pipeline Verified locally inside ubuntu24_dbg_build with WITHTSAN=1 + PROXYSQL40=1: both *-t binaries pass cleanly (27/27 and 74/74).	2 months ago
Rene Cannao	960eeab8d2	feat(mysqlx): wire per-route tls_mode through handler_capabilities_set entry path ... PR #5709 landed the splice mechanics for tls_mode='passthrough' plus the per-route tls_mode column on mysqlx_routes, but no code consulted the route's tls_mode to actually drive the session into X_PASSTHROUGH_FORWARD from real client traffic — a route configured as `tls_mode='passthrough'` behaved identically to `'inherit'`. This commit closes the gap (issue #5710): (1) Advertise gating in send_capabilities() ----------------------------------------- A new helper, MysqlxSession::effective_route_tls_mode(), reads listener_route_name_ + the thread's MysqlxConfigStore and returns the per-route tls_mode (or `inherit` when listener route is empty / config store is unwired — preserving historical behaviour for tests and unconfigured deployments). send_capabilities() now suppresses the `tls` capability from the advertised set when route_tls_mode == disabled. Other modes (inherit / preferred / required / passthrough) advertise TLS the same way as before — passthrough MUST advertise so the client even thinks to upgrade. (2) Symmetric refusal in handler_capabilities_set() ------------------------------------------------ A client that sends CapabilitiesSet(tls=true) on a disabled-mode route is refused with X-Protocol error 5052 ("TLS is not enabled on this route"). Symmetric with the advertise gate; we never told the client TLS was available, so accepting it would be a silent policy bypass. (3) Passthrough entry from handler_capabilities_set() -------------------------------------------------- When CapabilitiesSet(tls=true) lands on a passthrough-mode route: a. resolve_passthrough_backend_target() picks the endpoint from the route's destination_hostgroup (no identity_ — passthrough authenticates end-to-end); b. the original CapabilitiesSet frame bytes are buffered in passthrough_pending_capset_frame_ for verbatim forwarding; c. tls_mode_ flips to TLS_PASSTHROUGH and status_ to the new X_PASSTHROUGH_BACKEND_CONNECTING state. (4) X_PASSTHROUGH_BACKEND_CONNECTING handler ----------------------------------------- handler_passthrough_backend_connecting() drives the four-step sequence: 1. allocate / pick up backend_conn_, start non-blocking TCP connect; 2. poll check_connect() to completion; 3. write the buffered CapabilitiesSet bytes verbatim to the backend (with EAGAIN/partial-write retries); 4. read exactly one X-Protocol frame from the backend (the LAST X-Protocol parse this session does); on Ok forward to the client and transition to X_PASSTHROUGH_FORWARD; on Error propagate to the client and close. The handler is registered both in the early fast-path branch of MysqlxSession::handler() (so the X-Protocol read on client_ds_ is skipped — the client's next bytes are the TLS ClientHello which is not parseable as an X frame) and in the main dispatch switch. Tests (test/tap/tests/unit/mysqlx_session_unit-t.cpp, +11 ok()s): * test_route_tls_mode_disabled_no_tls_in_advertise: drives CapabilitiesGet on a tls_mode='disabled' route, parses the response, asserts the advertised set has no `tls` capability. * test_capabilities_set_tls_refused_on_disabled_route: drives CapabilitiesSet(tls=true) on the same route, asserts session becomes unhealthy and the client receives a Mysqlx::Error. * test_route_passthrough_full_entry_path: end-to-end exercise. Stands up a loopback TCP listener as the "backend"; configures a passthrough route pointing at the listener; drives CapabilitiesGet -> CapabilitiesSet(tls=true); asserts the proxy forwards the CapabilitiesSet bytes verbatim, transitions through X_PASSTHROUGH_BACKEND_CONNECTING into X_PASSTHROUGH_FORWARD on the backend's Ok response, and stays healthy. Plan count: 87 -> 98 (after Fix B) -> 109. Closes #5710.	2 months ago
Rene Cannao	b7c5a08b96	feat(mysqlx): backlog and resume EAGAIN'd writes in passthrough splice ... Previously, the passthrough splice loop's pump_one_direction helper treated EAGAIN/EWOULDBLOCK on write() as fatal — the session transitioned to X_SESSION_CLOSING and the connection was dropped. In practice this turned any transient destination back-pressure (slow client, slow backend, kernel send-buffer drained more slowly than the proxy reads from the source side) into a connection drop. End-to-end encrypted sessions (the whole point of tls_mode='passthrough') would RST mid-stream under bursty load. Fix: per-direction write backlog, drained before the next read. * Add MysqlxSession::passthrough_c2b_backlog_ and passthrough_b2c_backlog_ — std::vector<uint8_t> buffers for bytes that read() from the source side could not be immediately flushed to the destination side. Cleared on init() / reset(). * handler_passthrough_forward() now runs a drain pass over the backlog BEFORE the read pass. drain_backlog() loops on write(), EAGAIN means "not now, leave it for the next tick"; any other write error fails the session. * On a partial-write inside the read pass, the unwritten tail is appended to the backlog (append_to_backlog) and the read pass yields. A future pump tick (driven by either the libev EV_READ wakeup on the source or the buffered-work re-arm via `to_process = true`) drains the backlog and resumes reading. * Backlog cap: 1 MiB per direction (PASSTHROUGH_BACKLOG_CAP). Above the cap, append_to_backlog refuses and the session is killed (X_SESSION_CLOSING) — slow- consumer DoS protection. The drain-first contract limits the backlog organically to BURST_BYTES (64 KiB), so the cap is purely defensive against a future logic bug or a larger burst. * to_process = true on a non-empty backlog so the outer dispatch loop re-enters us on the next tick rather than waiting for a new fd-readable event on the source — without this, a long stall on the destination side could let the backlog grow on each new source byte before the previous backlog was drained. Tests (test/tap/tests/unit/mysqlx_session_unit-t.cpp): * test_passthrough_eagain_backlog_drains_across_ticks: drives 256 KiB through a backend leg with SO_SNDBUF/RCVBUF shrunk to 4 KiB so write() returns EAGAIN early and often. Asserts session stays healthy in X_PASSTHROUGH_FORWARD across the repeated EAGAIN events; all 256 KiB are eventually delivered; no byte-level corruption. * test_passthrough_backlog_cap_kills_session: uses the MYSQLX_TEST_BUILD-only seed_passthrough_c2b_backlog_for_test + try_append_to_passthrough_c2b_backlog_for_test helpers to drive the cap branch directly. Asserts within-cap append is accepted, over-cap append is refused, and the session transitions to X_SESSION_CLOSING with healthy=false. Plan count moves from 87 to 98.	2 months ago
Rene Cannao	e855ba37c8	fix(mysqlx): decode backend TLS error code before falling back to plaintext (preferred mode) ... Previously, when a session was in `preferred` TLS mode (try TLS, fall back to plaintext on Error) and the backend responded with ANY Mysqlx::Error to the client's CapabilitiesSet(tls=true), the code silently dropped TLS and proceeded plaintext. Non-TLS errors ("internal server error", "out of memory", "permission denied", etc.) were therefore swallowed and the auth flow proceeded against a backend that just told us it was unhealthy — masking real failures and encouraging brittle deployments. The upstream MySQL X client at plugin/x/client/xsession_impl.cc gates the same fallback exactly on `ER_X_CAPABILITIES_PREPARE_FAILED` (5001), which is the only code that the X plugin's `Capability_tls::set_impl` emits when the server has no SSL context (see plugin/x/src/capabilities/handler_tls.cc — code 5001 is what `ngs::Error(ER_X_CAPABILITIES_PREPARE_FAILED, ...)` produces). Mirror that policy here: * Parse the Mysqlx::Error body's `code` field. * Fall back to plaintext only when the code is exactly 5001. * Any other code (or a code-less / unparseable Error body) is fatal — transition to BACKEND_AUTH_ERROR and return -1. The existing `preferred-mode fallback` unit test (which already injects code 5001) still passes. Two new tests cover the new gate: * preferred-mode + non-TLS error code 1045 (Access denied): no fallback, auth_state=ER, rc=-1. * preferred-mode + Error frame with no decodable code (empty body): no fallback, auth_state=ER, rc=-1. The pre-existing required-mode test (no fallback regardless of code) is unchanged.	2 months ago
Rene Cannao	352455d79c	test(mysqlx): free fixture allocations leaked under LeakSanitizer ... Closes #5703. Phase 1 of the ASAN sanitizer pass on PR #5651 surfaced two test-side leaks: 1. mysqlx_admin_commands_unit-t.cpp:210 — the SQLite3_result* returned by execute_statement was wrapped in a unique_ptr with a custom no-op deleter, with the comment "leaked intentionally for test scope". The "intentionally" was test-author convenience, not a real reason — the default unique_ptr<SQLite3_result> deleter calls ~SQLite3_result(), which the type supports. Drop the no-op deleter. 2. mysqlx_message_dispatch_unit-t.cpp::test_connection_pool_matching — two MysqlxConnection* allocations (c1 at line ~551, c2 at line ~566) are inserted into the per-thread cache and then extracted via get_connection_from_cache(). Extraction transfers ownership to the caller (the cache erases the entry); the test never re- pools or deletes them, so they leak. Add explicit `delete found; delete f2;` at end of test with a comment explaining the ownership transfer. Verified locally under ASAN with detect_leaks=1: both tests now report 0 LeakSanitizer findings. The pre-existing failure of test 24 in mysqlx_admin_commands_unit-t ("mysqlx_variables has 4 rows after save (all known variables)") is unrelated to this PR — it predates the parity-cleanup branch and exists on parent.	2 months ago
Rene Cannao	b340da9ab6	test(mysqlx): add behavioural TAP scaffolding for SIGTERM mid-traffic ... Issue #5678 part (a) asks operators to demonstrate that an X-Protocol client connected to ProxySQL during SIGTERM receives a clean Mysqlx::Error frame with code 1053 ("Server is shutting down") rather than a TCP RST. The contract is: Mysqlx_Thread::run() -> for each session: MysqlxSession::shutdown_notify_client() -> send_error(1053, "Server is shutting down", fatal=true) -> client_ds_.write_to_net() -> if TLS: SSL_set_quiet_shutdown(1) + SSL_shutdown() (plugins/mysqlx/src/mysqlx_session.cpp:1875, plugins/mysqlx/src/mysqlx_thread.cpp:120-128). Why this is scaffolding, not an automated test: ProxySQL runs in its own Docker container under test/infra/control/run-tests-isolated.bash. Tests run in a separate test-runner container that does NOT have docker socket access, so it cannot kill -TERM the proxysql container. Even if it could, doing so mid-test would tear down the infrastructure that the rest of the TAP group expects to remain up. The harness has no "test that owns proxysql lifecycle" mode and adding one is out of scope. The equivalent automated path already exists out-of-band: test/scripts/mysqlx/behavioral_validation.py --scenario sigterm which (a) opens N X-Protocol sessions via mysql-connector-python, (b) os.kill(pid, SIGTERM)s the proxysql process directly, and (c) verifies each client's exception carries errno=1053 (Mysqlx shutdown frame), not a generic OperationalError-without-errno (TCP RST). It is invoked manually against staging or local-loop infrastructure where proxysql termination is acceptable. What this binary does: - Compiles + links, so groups.json registration exercises its existence on every CI run alongside the rest of the mysqlx-soak group; future drift in the link line or TAP harness shows up here, not at "we never built this". - At runtime, defaults to skip_all() with a pointer to the behavioral_validation.py script and the manual procedure documented in the file's header comment. - With MYSQLX_SIGTERM_INFLIGHT_OPT_IN=1 set, the binary refuses loudly (fails with a planned ok(false, ...)) rather than silently doing nothing, so an operator who flips the env var without first wiring up an automated body gets a clear signal. - Header comment includes the pseudocode for the future automated body so a follow-up PR can lift it directly. Wired into mysqlx-soak-g1 alongside the other mysqlx behavioural tests; the skip_all keeps CI green while the registration ensures the binary is built and the test name is well-known. Build: WITHASAN=1 NOJEMALLOC=1 PROXYSQLGENAI=1 make test_mysqlx_sigterm_inflight-t Verified that the binary skip_alls correctly when MYSQLX_SIGTERM_INFLIGHT_OPT_IN is unset: $ ./test_mysqlx_sigterm_inflight-t 1..0 # skip manual procedure -- see header comment. ...	2 months ago
Rene Cannao	9e6243026f	test(mysqlx): add behavioural TAP test for LOAD MYSQLX ROUTES TO RUNTIME mid-traffic ... Demonstrates issue #5678 part (b) end-to-end against a real ProxySQL instance with the mysqlx plugin loaded. The test: 1. Opens 5 X-Protocol clients to ProxySQL via route r1 and dispatches SELECT 1 on each so the session is past handshake/routing. 2. Connects to the admin port (classic protocol via libmariadbclient) and runs: DELETE FROM mysqlx_routes WHERE name='r1'; LOAD MYSQLX ROUTES TO RUNTIME; 3. Asserts each in-flight session can still execute SELECT 1 -- the contract from mysqlx_listener_reconcile.cpp::remove_listener_for_route ("in-flight sessions are NOT torn down on route removal; they continue against their already-resolved target_hostgroup_/target_address_/target_port_") matching MySQL's "DROP TABLE doesn't cancel in-flight queries" semantics. 4. Asserts a NEW TCP connect to the dropped route's port is refused (Mysqlx_Thread::remove_listener_for_route close()d the listener fd). 5. Restores the route via INSERT + LOAD MYSQLX ROUTES TO RUNTIME so downstream tests in the same group are not disrupted. Wired into the mysqlx-soak-g1 TAP group, which is the only group that provisions route r1 + user alice in ProxySQL (mysqlx-e2e-g1 uses SKIP_PROXYSQL=1 against a dbdeployer sandbox so the route doesn't exist there). Runs through the standard isolated docker harness (test/infra/control/run-tests-isolated.bash); requires no new infrastructure beyond what mysqlx-soak/setup-infras.bash already provisions. The test issues skip_all() if the X-Protocol listener is unreachable, which keeps it inert in groups that don't bring up the route-r1 fixture and on bare workstations without infrastructure. Build: WITHASAN=1 NOJEMALLOC=1 PROXYSQLGENAI=1 make test_mysqlx_route_drop_inflight-t Reuses the protocol helpers from plugins/mysqlx/src/mysqlx_protocol.cpp and the protobuf objects from plugins/mysqlx/proto/, identical to the existing test_mysqlx_e2e_*-t binaries.	2 months ago
Rene Cannao	bd906167fb	fix(mysqlx): place NOSONAR(cpp:S4423) on the same line as TLS_method() ... SonarCloud's NOSONAR annotation only suppresses when it appears on the same source line as the rule trigger; preceding comment blocks don't count. The previous attempt put the rationale in a 5-line block comment above the SSL_CTX_new call, which SonarCloud ignored — quality gate stayed red on PR #5711 / #5713 / #5714 because the S4423 vulnerability finding still landed. Move the rationale into the function-level doc comment and put a short `// NOSONAR(cpp:S4423)` trailer on the SSL_CTX_new line itself. That's the syntactic shape SonarCloud's parser actually recognizes.	2 months ago
Rene Cannao	8566d93df9	fix(mysqlx): suppress SonarCloud S4423 false positive on TLS_method() in classifier test ... The cpp:S4423 rule flags TLS_method() as a 'weak protocol' constructor. That was true for SSLv23_method() — TLS_method() is the OpenSSL 1.1+ recommended replacement and defaults to TLS 1.2+. Protocol-version floor is set via SSL_CTX_set_min_proto_version where the test runs real handshakes (e.g. create_test_ssl_ctx at line 64 sets TLS1_2_VERSION). The classifier test fixture in make_synthetic_ssl (line 292) never runs a handshake — it stages SSL_set_verify_result state and feeds the SSL* directly to the classifier under test. Version negotiation is not exercised, so a min_proto_version setter would be cargo-cult. Rather than introduce a no-op call, suppress the false positive with NOSONAR + a comment explaining why. This was the lone VULNERABILITY on PR #5711's SonarCloud report; with it suppressed, new_security_rating drops from 4 (D) to 1 (A) and the quality gate flips green.	2 months ago
Rene Cannao	0ccf3615f1	fix(mysqlx): classify backend TLS handshake failures into specific error codes ... Closes #5698 (P3 — meaningful TLS error messages on backend failures). Background: prior to this commit, backend TLS handshake failures all emitted a single generic "Backend TLS handshake failed" with code 3152; frontend TLS failures all emitted "TLS handshake failed" code 3151. Operators debugging TLS issues had to attach with `openssl s_client` or run the proxy under SSL_TRACE / SSLKEYLOGFILE to figure out whether the underlying cause was an expired cert, a hostname mismatch, an untrusted CA, or a protocol-version mismatch. MySQL Router translates each of these into a distinct X-Protocol error code and shortcuts the debug session. Implementation: 1. New MysqlxTlsErrorClass enum (HANDSHAKE_FAILED, CERT_VERIFY_FAILED, CERT_EXPIRED, HOSTNAME_MISMATCH, PROTOCOL_MISMATCH, UNKNOWN_CA, NO_SSL_CTX, UNKNOWN). Lives in mysqlx_protocol.h alongside the per-class X-Protocol error code constants. 2. New mysqlx_classify_tls_error(SSL*, peek_err_queue) helper. Classification logic in order: a. SSL == nullptr -> NO_SSL_CTX b. SSL_get_verify_result() != X509_V_OK -> chain class (CERT_EXPIRED for expired/not-yet-valid; HOSTNAME_MISMATCH; UNKNOWN_CA for self-signed-in-chain or unable-to-get-issuer; CERT_VERIFY_FAILED for everything else cert-chain) c. ERR_get_error() walked for SSL_R_UNSUPPORTED_PROTOCOL / TLSV1_ALERT_PROTOCOL_VERSION / WRONG_VERSION_NUMBER / UNKNOWN_PROTOCOL -> PROTOCOL_MISMATCH d. Fallback -> HANDSHAKE_FAILED Cert-chain reasons take precedence because the OpenSSL queue sometimes carries both, and the chain reason is more actionable. 3. Backend codes (5+ distinct, 3152-3157): 3152 BACKEND_TLS_ERR_HANDSHAKE_FAILED (was: only existing code) 3153 BACKEND_TLS_ERR_CERT_VERIFY_FAILED (NEW) 3154 BACKEND_TLS_ERR_CERT_EXPIRED (NEW) 3155 BACKEND_TLS_ERR_HOSTNAME_MISMATCH (NEW) 3156 BACKEND_TLS_ERR_PROTOCOL_MISMATCH (NEW) 3157 BACKEND_TLS_ERR_UNKNOWN_CA (NEW) Backend messages name the OpenSSL reason in operationally-useful terms ("certificate hostname mismatch") but do NOT include the raw OpenSSL queue string in the wire frame — that is logged to stderr separately for the operator's benefit. 4. Frontend codes (3 distinct): 3150 FRONTEND_TLS_ERR_NOT_CONFIGURED (existing — NO_SSL_CTX) 3151 FRONTEND_TLS_ERR_HANDSHAKE_FAILED (existing — every other class) 3158 FRONTEND_TLS_ERR_PROTOCOL_MISMATCH (NEW — operationally useful for legitimate clients hitting a too-old/too-new TLS version) Frontend collapses CERT_*/HOSTNAME_MISMATCH/UNKNOWN_CA onto HANDSHAKE_FAILED specifically to avoid leaking attacker-supplied cert info into the response (asymmetric threat model — the frontend client may be the attacker). 5. New MysqlxConnection::tls_error_class_ member + accessors. Set by step_auth_tls_handshake() at the failure site (the OpenSSL error queue is thread-local FIFO and must be drained while fresh; we can't defer the classification to the session). Read by the session's BACKEND_AUTH_ERROR branch in handler_connecting_server when emitting send_error(). 6. Frontend handler_tls_accept_init() also runs the classifier and emits via the frontend code/message helpers. Logs the OpenSSL queue to stderr (operator-side only). Tests (in mysqlx_tls_unit-t): - test_classify_null_ssl: nullptr -> NO_SSL_CTX (1 ok) - test_classify_cert_expired_via_verify_result (1 ok) - test_classify_hostname_mismatch_via_verify_result (1 ok) - test_classify_unknown_ca_via_verify_result: 2 sub-cases (UNABLE_TO_GET_ISSUER + SELF_SIGNED_IN_CHAIN) (2 ok) - test_classify_generic_cert_verify_failed: SIGNATURE_FAILURE -> generic (1 ok) - test_classify_handshake_failed_default (1 ok) - test_classify_code_message_round_trip_backend: 7 classes * 2 (msg + code-range) + 1 distinct-codes assertion (15 ok) - test_classify_code_message_round_trip_frontend: collapse + 3 distinct-code asserts (4 ok) The cert-fixture-driven tests (real expired/hostname-mismatched/ self-signed cert chains) are explicitly punted per the issue's "don't generate cert fixtures in this PR" guidance. The classifier accepts SSL_set_verify_result-staged synthetic state, which exercises the same code paths as a real cert failure (since the only thing the classifier reads off SSL is the verify_result long); a follow-up issue covers wiring an end-to-end TAP test against a fixture-driven backend. Build: NOJEMALLOC=1 WITHASAN=1 PROXYSQLGENAI=1 plugin .so + build_tap_test_debug all green. Pre-existing RSA_new deprecation warnings unchanged. Tests: mysqlx_tls_unit-t (44/44, was 18 — +26 from 8 new tests). Other suites unchanged. ASAN-clean.	2 months ago
Rene Cannao	767f808309	fix(mysqlx): mark post-Session::Reset connections non-cacheable ... Closes #5697 (P3 — Session Reset pool invalidation). Background: X-Protocol's Session::Reset frame asks the server to wipe session state (current schema, isolation level, charset, prepared statements, session vars) without reauthenticating. ProxySQL forwards the frame and dispatches the backend Ok response correctly — but until this commit, the connection was returned to the per-thread cache as a regular IDLE entry afterwards. The cache key (hostgroup, user, schema, tls_active) doesn't capture session-state-version, so a subsequent reuse from the same identity could pick up the post-reset connection and silently inherit blank state instead of the per-identity defaults. The pool-leak scenario in practice: - Client A issues Session::Reset on backend B. - Backend B is returned to the per-thread cache as IDLE. - Client A2 (same user/schema/hostgroup/tls posture) next request pulls B from the cache. - Client A2 sees blank session state, even though it expected the per-identity defaults a fresh-auth connection would provide. MySQL Router treats Session::Reset as pool-terminating; this commit brings ProxySQL in line. Implementation: - New MysqlxConnection::needs_post_reset_rehandshake_ bool member + accessor pair (set/get). Default false. - is_reusable() returns false when the flag is set, so Mysqlx_Thread::return_connection_to_cache deletes the connection instead of pooling it. - reset() defensively clears the flag too. Production path doesn't rely on this (the cache deletes non-reusable conns instead of resetting them), but the invariant matters for any future code path that calls reset() directly (e.g. retry-on-error). - handler_session_reset_waiting() sets the flag on the Mysqlx.Ok branch — the only path where Session::Reset succeeded. The Error branch already marks the connection non-reusable for unrelated reasons (the Reset itself failed), so no change needed there. We do NOT implement the rehandshake path itself in this commit; the simpler "drop on reset" semantic matches Router's behaviour and is strictly safer than the pre-fix "silently pool blank state". Tests: - mysqlx_connection_unit-t::test_connection_post_reset_rehandshake_flag asserts the flag's defaults, set/get round-trip, is_reusable() interaction, and reset() defensive-clear behaviour (6 ok()). - mysqlx_message_dispatch_unit-t::test_dispatch_sess_reset_marks_non_cacheable drives the full SESS_RESET → backend-Ok → return-to-pool flow with a fake backend and asserts the connection is non-cacheable afterwards. Strengthens the existing test_dispatch_sess_reset which only proved the dispatch reached the backend (5 ok() — pre-flag/state, post-dispatch status, post-condition observed via backend_conn being deleted on the non-reusable path). - mysqlx_connection_unit-t plan switched to plan(0) so future additions don't require re-counting. Build: NOJEMALLOC=1 WITHASAN=1 PROXYSQLGENAI=1 plugin .so + build_tap_test_debug all green, no compile warnings. Tests: mysqlx_connection_unit-t (16/16, was 10), mysqlx_message_dispatch_unit-t (114/114, was 109). Other suites unchanged. ASAN-clean.	2 months ago
Rene Cannao	31a3934014	fix(mysqlx): explicit auth-phase notice forwarding policy in step_auth ... Closes #5695 part 2 (P2 — explicit per-state policy for backend NOTICE frames during the backend auth handshake; data-plane validation landed in the previous commit). Background: prior to this commit, MysqlxConnection::read_auth_frame() silently drained ALL backend NOTICE frames during the BACKEND_AUTH_* state machine. The drain was needed (MySQL backends commonly emit a SESSION_STATE_CHANGED notice before AuthenticateContinue / Ok, and returning nullopt on a NOTICE caused auth to spin until the 10s handshake timeout) but it was indiscriminate — a malformed/unknown-type NOTICE would be drained as silently as a legitimate one. This commit adds an explicit per-state policy: - New helper auth_phase_notice_is_drainable(body, body_len). Parses the NOTICE as a Mysqlx::Notice::Frame and applies a per-type decision: WARNING / SESSION_VARIABLE_CHANGED / SESSION_STATE_CHANGED / GROUP_REPLICATION_STATE_CHANGED / SERVER_HELLO -> drain malformed / empty / unknown enum value -> fail auth WARNING and GROUP_REPLICATION_STATE_CHANGED are also logged on the drain path: WARNING during auth typically signals a server-side misconfig (deprecated auth method, etc.); GROUP_REPLICATION_* during auth is operationally out-of-place (cluster-membership notices belong on data-plane connections, not handshakes). Both are drained to preserve compatibility with over-eager backends but surfaced for operator triage. - Wired into read_auth_frame() between the MAX_LEADING_NOTICES cap and the type==NOTICE branch. On a fail-auth result, the helper has already set auth_state_=BACKEND_AUTH_ERROR and logged; the caller propagates std::nullopt and step_auth fails the connection. - Frontend forwarding during auth: explicitly NEVER allowed. The frontend client has no context to interpret a backend NOTICE before it sees AuthenticateOk; surfacing them would also leak server-side state mid-handshake (the load-bearing concern in issue #5695). The proxy terminates ALL backend NOTICEs on the backend leg during the backend auth phase — even legitimate ones. - Test-only hooks under MYSQLX_TEST_BUILD: a public auth_phase_notice_is_drainable_for_test() and a set_auth_state_for_test() so unit tests can drive the per-type decision matrix directly without running the full step_auth state machine. Tests (in mysqlx_backend_auth_unit-t): - test_auth_phase_notice_known_types_drained: all five spec types are drainable and don't perturb auth_state_ (5*2=10 ok()). - test_auth_phase_notice_unknown_type_fails_auth: type=99 fails auth and sets BACKEND_AUTH_ERROR (2 ok()). - test_auth_phase_notice_malformed_fails_auth: garbage protobuf bytes fail auth (2 ok()). - test_auth_phase_notice_empty_fails_auth: empty body fails auth (2 ok()). - Switched the existing plan(42) to plan(0) so future additions don't require re-counting; existing test_backend_auth_notice_skip (which uses Frame_Type_WARNING) continues to pass under the new policy because WARNING is still on the drainable list. Build: NOJEMALLOC=1 WITHASAN=1 PROXYSQLGENAI=1 plugin .so + build_tap_test_debug all green, no compile warnings. Tests: mysqlx_backend_auth_unit-t (58/58, was 42 — +16 from the 4 new tests), other suites unchanged. ASAN-clean.	2 months ago
Rene Cannao	6ef8cf3f1f	fix(mysqlx): validate Mysqlx::Notice::Frame::type before forwarding ... Closes #5695 part 1 (P2 — explicit notice forwarding awareness; auth-phase per-state policy is the next commit; per-notice metric remains under #5691 follow-up scope). Background: X-Protocol NOTICE frames are non-terminal informational messages a backend can emit at essentially any state transition (warnings, session-state changes, generated-doc-id notifications, group-replication membership changes, etc.). Prior to this commit the proxy forwarded NOTICE frames uncritically; a buggy or hostile backend (or an MITM that bypassed TLS) could ship a NOTICE frame with an unknown `type` enum value and confuse a strict client. The risk is bounded — info disclosure, no auth bypass — but it's unhardened territory MySQL Router handles explicitly. This commit adds a per-frame validation hook on the data plane: - New MysqlxSession::is_notice_frame_valid(body, body_len). Pure query — no session-state mutation. Parses the body as a Mysqlx::Notice::Frame and verifies the outer `type` field is in the spec range (1..5: WARNING, SESSION_VARIABLE_CHANGED, SESSION_STATE_CHANGED, GROUP_REPLICATION_STATE_CHANGED, SERVER_HELLO) using the protoc-generated Frame_Type_IsValid(). Empty body, malformed protobuf, missing required `type` field, and out-of-range enum values all return false. - Wired into handler_waiting_server_msg() and handler_session_reset_waiting() before forwarding NOTICE to the client. Invalid notices are dropped with a stderr log line tagged with route+hostgroup+body_len for operator triage; the response sequence continues (NOTICE is non-terminal in every state). - Test-only hook is_notice_frame_valid_for_test() exposed under MYSQLX_TEST_BUILD so unit tests can drive the predicate directly with synthetic bodies without setting up a fake backend. Tests (in mysqlx_message_dispatch_unit-t): - test_notice_validation_known_types_accepted: all five spec types pass (5 ok()). - test_notice_validation_unknown_type_rejected: types 0, 99, and 100000 are rejected (3 ok()). - test_notice_validation_empty_body_rejected: nullptr body and zero-length body both rejected (2 ok()). - test_notice_validation_malformed_protobuf_rejected: garbage bytes that fail ParseFromArray are rejected (1 ok()). We deliberately do NOT validate the inner `payload` bytes — that's a type-specific protobuf the client parses, and the proxy has no business reaching into it. The outer `type` is the load-bearing field for client-side branching, so that's where we focus. Build: NOJEMALLOC=1 WITHASAN=1 PROXYSQLGENAI=1 make build_lib + plugin .so + build_tap_test_debug all green. Tests: mysqlx_session_unit-t (87/87), mysqlx_message_dispatch_unit-t (109/109, was 98 — +11 from the 4 new tests), mysqlx_compression_unit-t (64/64), mysqlx_tls_unit-t (18/18), mysqlx_thread_unit-t (25/25), mysqlx_concurrent_unit-t (6/6). All ASAN-clean.	2 months ago
Rene Cannao	9ea2b31bf5	fix(mysqlx): match upstream X-Protocol error code for compression-without-algorithm ... Closes #5696 (P2 — MySQL Router parity gap, compression error code). Before this change the mysqlx plugin emitted error code 5008 for every compression-related failure ("Compression frame received without negotiated algorithm", parse failures, decompression failures, etc.). That code does not match upstream MySQL X plugin: 5008 in plugin/x/src/xpl_error.h is ER_X_BAD_CONNECTION_SESSION_ATTRIBUTE_TYPE — a totally unrelated session-attribute error. A client written specifically against the upstream X-Protocol error contract would see the wrong error class entirely. Authoritative source for the canonical codes: plugin/x/src/xpl_error.h in the MySQL 8.4 source tree (deps/mysql-connector-c-8.4.0/.../xpl_error.h in this repo). Mapping: - 5170 ER_X_FRAME_COMPRESSION_DISABLED — compression frame received when the client did not negotiate compression. Upstream message: "Client didn't enable the compression." Reproduced verbatim. - 5174 ER_X_BAD_COMPRESSED_FRAME — structural problem with the Compression envelope (empty body, malformed protobuf, bogus uncompressed_size hint, decompressed payload that doesn't reframe into valid X messages). - 5171 ER_X_DECOMPRESSION_FAILED — algorithm-side failure (lz4/zstd error, OOM, stall, decompressed payload exceeds cap). - 5000 ER_X_BAD_MESSAGE — server-direction compression on the client→server path; treated as a wrong-direction message rather than a compression-specific failure. The pre-auth capability replay-cap error (also previously emitting 5008) is a ProxySQL-specific guardrail with no upstream analogue and is left untouched in this commit — the parity fix is scoped to the genuine compression error sites. Test updates: - mysqlx_message_dispatch_unit-t::test_dispatch_compression_rejected: asserts 5170 (was 5008). - mysqlx_compression_unit-t: oversize/garbage rejections assert 5171, no-negotiation rejection asserts 5170, "no error frame" tests accept any of 5170/5171/5174 (renamed got_5008 → got_compression_err). - test_compression_without_negotiation_still_5008 → test_compression_without_negotiation_still_5170. Build: NOJEMALLOC=1 WITHASAN=1 PROXYSQLGENAI=1 make build_lib + plugin .so + build_tap_test_debug all green. Tests: mysqlx_session_unit-t (87/87), mysqlx_message_dispatch_unit-t (98/98), mysqlx_compression_unit-t (64/64), mysqlx_tls_unit-t (18/18), mysqlx_thread_unit-t (25/25), mysqlx_concurrent_unit-t (6/6) — all ASAN-clean.	2 months ago
Rene Cannao	2e7e532050	feat(mysqlx): X_PASSTHROUGH_FORWARD session state — raw byte splice ... Implements the data-plane half of the per-route TLS passthrough mode (issue #5692). Once a session enters this state the proxy stops parsing X-Protocol frames; bytes are read from one side and written to the other verbatim using read(2)/write(2). Used cases: end-to-end TLS where operator policy forbids proxy MITM (compliance, original cert/SNI/ALPN preservation). Session machine additions: * `MysqlxSession::Status::X_PASSTHROUGH_FORWARD` — terminal state for passthrough sessions. Once entered, the session never returns to any X-Protocol parsing state. handler() short-circuits past client_ds_.read_from_net()/parse so the bytes are not interpreted as frames. * `MysqlxTlsMode::TLS_PASSTHROUGH` — added back to the per-session TLS posture enum. The previous prototype carried the value without a real implementation; that was removed in the asymmetric-TLS series and is now reintroduced as a real feature. * `handler_passthrough_forward()` — pumps both directions per call up to a 64 KiB burst, then yields so other sessions on the thread are not starved. EAGAIN/EWOULDBLOCK is the normal "no more bytes right now" exit. EOF on either side or any other I/O error transitions to X_SESSION_CLOSING. Listener-route propagation: * New `MysqlxSession::init(fd, thread, listener_route)` overload. The base init(fd, thread) overload calls it with an empty route name, preserving existing test harnesses that construct sessions directly. * `Mysqlx_Thread::accept_new_connection()` looks up the listener fd in the parallel `listener_route_names_` vector and propagates the route name to the new session at accept time. This is the hook that lets per-route policies fire before any X-Protocol message is received — most importantly, before any local handshake decision so a passthrough route can splice from the very first byte. Test-only entry path: * `enter_passthrough_for_test(int backend_fd)` (gated behind MYSQLX_TEST_BUILD) drops a session straight into the splice state with a caller-supplied backend fd. Bypasses CapabilitiesSet / auth / resolve_backend_target so the splice mechanics can be asserted in isolation. set_reusable(false) is set on the stub backend connection to mirror the production-side invariant that a passthrough connection never returns to the pool. Tests (mysqlx_session_unit-t, +22 assertions, plan 65 -> 87): * listener_route_propagation: init() captures, reset() clears * forward_client_to_backend / forward_backend_to_client: bytes survive the splice unchanged in both directions * close_on_client_eof / close_on_backend_eof: peer half-close transitions to X_SESSION_CLOSING and marks the session unhealthy * disables_backend_reuse: backend_conn() exists but is_reusable()=false, and tls_mode_ reports TLS_PASSTHROUGH * handler_dispatch_skips_xprotocol: bytes that would parse as a fake X frame in a non-passthrough state survive the splice unconsumed — proves handler() takes the fast path before the X frame parser sees the buffer Production wiring of the entry path (CapabilitiesSet on a passthrough route -> connect to backend, splice the TLS handshake) lands in a follow-up. This commit is intentionally limited to the splice mechanics plus the listener-route plumbing the entry path will need.	2 months ago
Rene Cannao	09c42d4fba	feat(mysqlx): add per-route tls_mode column with passthrough enum value ... Adds the schema and config-store plumbing needed to express MySQL Router's TLS-Passthrough mode as a per-route attribute on `mysqlx_routes`. The actual data-plane forwarding is implemented in a follow-up commit; this commit only wires the config so the operator can set / round-trip / project the value. Schema additions: * `mysqlx_routes.tls_mode VARCHAR NOT NULL DEFAULT 'inherit'` plus a CHECK constraint on the canonical lowercase spellings ('inherit', 'disabled', 'preferred', 'required', 'passthrough'). Same column added to `runtime_mysqlx_routes`. * Default 'inherit' means "use the deployment-wide `mysqlx_tls_mode`", preserving existing behaviour for every row that was loaded before this commit. Operators wanting compliance-pinned passthrough can override one route at a time without flipping the global mode. Config store additions: * `MysqlxRouteTlsMode` enum class (inherit / disabled / preferred / required / passthrough) plus matching `mysqlx_route_tls_mode_from_string` (case- insensitive, empty/NULL -> inherit, unknown -> nullopt) and `_to_string` helpers — same shape as the existing `mysqlx_backend_tls_mode_*` family. * `MysqlxRoute::tls_mode` field with a default of `inherit`. * `MysqlxConfigStore::route_tls_mode(name)` accessor; unknown routes report `inherit` (matching how `route_hostgroup` returns 0 for unknown routes). LOAD / SAVE / project paths: * `install_routes_from_admin` first PRAGMA-probes for the `tls_mode` column so a pre-upgrade admin DB without it loads cleanly (column treated as NULL -> inherit). A malformed value (e.g. operator typo) fails the install with a descriptive `err` rather than silently coercing. * `save_routes_to_admin_table` and `project_routes_to_runtime_view` write the column using `mysqlx_route_tls_mode_to_string`. Tests: * mysqlx_config_store_unit-t gains nine assertions covering parser case-insensitivity, the empty-string / unknown-value contract, the canonical render, the legacy-schema fallback path, and a full install / parse-failure round-trip against a schema that has the column. * All previously-passing unit tests remain green. This commit is intentionally behaviour-preserving: passthrough is parsed and stored but not yet enforced. Resolving a route with tls_mode=passthrough still goes through the proxy-terminated path until the follow-up commit adds the X_PASSTHROUGH_FORWARD session state.	2 months ago
Rene Cannao	744fb0bf36	feat(mysqlx): preferred-mode backend TLS fallback to plaintext + docs ... Closes the loop on issue #5693 (P1: asymmetric TLS / AsClient mode). The previous two commits added the runtime variable and the per- session decision; this commit wires the actual `preferred`-mode fallback path in the backend auth state machine and updates the operator-facing documentation to describe the full four-mode model. What changes: * MysqlxConnection::step_auth_capabilities_set_sent now treats a Mysqlx::Error response from CapabilitiesSet(tls=true) as a fall-back trigger when backend_tls_fallback_allowed_ is set (set by the session for mode=preferred). The connection silently downgrades on the same TCP socket — backend_tls_required_ is cleared so subsequent steps don't re-attempt TLS, tls_active_ stays false (which keeps the connection out of the encrypted half of the per-thread pool), and step_auth proceeds straight into AuthenticateStart on plaintext. Under mode=required (and AsClient + frontend-TLS), the same Error is still fatal — the operator's policy demands encryption. The pre-existing `Frame is not OK` rejection branch is preserved for malformed / unexpected response types. * doc/mysqlx/README.md §8 rewritten: * §8.1 frontend TLS modes (mysqlx_tls_mode, unchanged behaviour). * §8.2 backend TLS modes (mysqlx_tls_backend_mode, full table of the four documented values, migration notes from MySQL Router AsClient, and a connection-pool partitioning note explaining that the cache key now includes tls_active. * §8.3 configuration example showing the asymmetric pattern (frontend REQUIRED + backend preferred). The §3.1 variables table gains the new variable. * doc/mysqlx/MYSQL_ROUTER_PARITY.md updated: * Feature comparison row 4 (TLS modes) now lists 4-frontend x 4-backend with explicit mention of mysqlx_tls_backend_mode and issue #5693. * Asymmetric TLS row marked implemented. * Architecture section §TLS rewritten to describe both the frontend mysqlx_tls_mode and backend mysqlx_tls_backend_mode models; identifies passthrough as the remaining gap. * Feature gaps table marks "Asymmetric TLS / AsClient" and "Per-message response state machines" as implemented. * Summary updated. Tests: test/tap/tests/unit/mysqlx_backend_auth_unit-t (was 34): adds 7 new ok lines covering both branches of the fallback decision: * test_backend_auth_preferred_mode_fallback_to_plaintext: 5 assertions across the wire-level transition (state on entry, rc=1 on Error, AUTHENTICATE_START_SENT after fallback, backend_tls_required_ cleared, tls_active false, AuthStart frame on the wire). * test_backend_auth_required_mode_no_fallback_on_error: 2 assertions confirming the same Error is fatal when fallback_allowed=false. Total 42. All 8 (mode x frontend_tls) acceptance combinations from the issue are now covered: 6 by the decision-function tests in mysqlx_message_dispatch_unit-t (commit 2), and the remaining 2 mode=preferred + backend Error cases by the wire-level fallback tests added here. Tested under NOJEMALLOC=1 WITHASAN=1 PROXYSQLGENAI=1. Closes #5693. Stacks on #5706 / #5704.	2 months ago
Rene Cannao	167a896c16	feat(mysqlx): mode-driven backend TLS decision + tls_active conn-cache key ... Implements the per-session decision side of issue #5693 (P1: asymmetric TLS / AsClient mode parity gap with MySQL Router 8.0). The previous commit added the runtime variable; this commit wires it through to the actual backend-connect path and partitions the connection pool by encryption posture so AsClient/required-TLS sessions never reuse plaintext-pooled backends (and vice versa). What changes: * Lifts the per-session decision out of an inline switch in MysqlxSession::handler_connecting_server() into a pure helper `mysqlx_resolve_backend_tls_decision(mode, endpoint_override, frontend_is_encrypted) -> {require_tls, fallback_allowed}`. The helper lives at file scope (not in the anonymous namespace) so the unit test can exercise the 8 (mode x frontend_tls) combinations called out in the issue acceptance criteria directly, without driving the full session state machine. The decision itself replaces the legacy `target_use_ssl_ || client_ds_.is_encrypted()` expression. * Adds two fields on MysqlxConnection: * tls_active_ -- set true by step_auth_tls_handshake() once the OpenSSL handshake completes successfully on the backend leg. Read by the connection cache to partition encrypted-pooled connections from plaintext-pooled ones. * backend_tls_fallback_allowed_ -- carries the `preferred` mode's "downgrade-to-plaintext-on-error" intent through to the auth state machine. Read-only metadata in this commit; the actual fallback path lands in the next commit. * Mysqlx_Thread::get_connection_from_cache now takes a `bool tls_active` parameter and matches on it. Without this, an AsClient TLS session pulling a pooled connection could land on a plaintext backend (or vice versa), corrupting the wire protocol when the next dispatched frame goes out over a socket in the wrong encryption posture. * Endpoint-override semantics (mysqlx_backend_endpoints.use_ssl=1) preserved exactly: it can promote plaintext to TLS regardless of mode, but cannot demote a TLS-required mode to plaintext. Under mode=preferred, the override leaves fallback_allowed=true so the operator's best-effort "preferred" intent isn't silently upgraded to "required". Tests: test/tap/tests/unit/mysqlx_message_dispatch_unit-t (was 86): adds 8 combinations (mode x frontend_tls) + 4 endpoint-override cases = 12 new ok lines, total 98. test/tap/tests/unit/mysqlx_thread_unit-t (was 22): adds test_connection_cache_tls_partition with 3 ok lines exercising the (plaintext, encrypted) hard partition, total 25. Known limitation (TODO addressed in next commit): mode=preferred currently fails the backend connect on Mysqlx::Error from CapabilitiesSet(tls=true) because the fallback-to-plaintext branch isn't wired in step_auth_capabilities_set_sent yet. The metadata (`backend_tls_fallback_allowed_` on the connection) is in place; only the state machine branch is missing. Two of the 8 documented test combinations (mode=preferred + Error from backend) are validated against the metadata only in this commit; the end-to-end fallback behaviour follows. Tested under NOJEMALLOC=1 WITHASAN=1 PROXYSQLGENAI=1. Refs #5693. Stacks on #5706 / #5704.	2 months ago
Rene Cannao	0246237426	feat(mysqlx): add MysqlxBackendTlsMode enum + mysqlx_tls_backend_mode runtime variable ... Wires up the configuration plumbing for issue #5693 (P1: asymmetric TLS / AsClient mode parity gap with MySQL Router 8.0). This commit is behaviour-neutral: the new variable is parsed, validated, persisted and exposed via MysqlxConfigStore::get_backend_tls_mode(), but the per- session backend-TLS decision still uses the legacy target_use_ssl_ || client_ds_.is_encrypted() expression. The decision site is rewritten in the next commit. What this adds: * MysqlxBackendTlsMode enum with four values matching MySQL Router's client_ssl_mode / server_ssl_mode taxonomy: disabled, preferred, required, as_client. Default is as_client because that most closely matches the legacy implicit behaviour where the backend leg encryption was tied to the frontend leg's encryption. * mysqlx_backend_tls_mode_from_string() / mysqlx_backend_tls_mode_to_string() for case-insensitive parsing and canonical lower-case rendering. The parser returns std::optional so the install path can surface a useful error to the operator on a typo instead of silently coercing to a default. * MysqlxConfigStore now reads the mysqlx_tls_backend_mode key from mysqlx_variables in install_variables_from_admin(), persists it via save_variables_to_admin_table(), and projects it in project_variables_to_runtime_view(). install fails atomically with a descriptive error when the value is unrecognised; an absent row leaves the cached mode untouched (matches how the other tunables already behave). * MysqlxBackendEndpoint.use_ssl=1 remains an operator-controlled override that forces TLS regardless of the mode (per existing comment at handler_connecting_server). The mode interacts with that flag in the next commit. Tests: test/tap/tests/unit/mysqlx_config_store_unit-t: 16 -> 24 assertions. New coverage: parser accepts all four documented values case-insensitively, parser rejects unknown values, default mode is as_client, LOAD round-trip caches the parsed mode, invalid value fails install with descriptive error, store retains last-good mode after rejected install, absent row leaves cached mode untouched. Tested under NOJEMALLOC=1 WITHASAN=1 PROXYSQLGENAI=1. Stacks on PR #5706 (mysqlx response state machines), which itself stacks on PR #5704 (mysqlx observability P0). Refs #5693.	2 months ago
Rene Cannao	170ead7ad6	fix(mysqlx): reject backend frames disallowed in current response state ... Tightens MysqlxSession::is_frame_allowed: a backend frame outside the per-state allowed set now closes the session with X-Protocol Error 4006 ("Backend sent an unexpected message in the current response state") instead of being forwarded blindly to the client. This guards against a buggy or hostile backend pushing a frame whose shape the client cannot parse, which would silently desync the wire and (in the worst case) amplify a backend protocol bug into a client-side fault. Adds a new private sub-state field MysqlxSession::seen_column_metadata_ that gates RESULTSET_ROW frames in the four response states where the X-Protocol requires ColumnMetaData to precede any Row: - RESP_WAITING_STMT_EXECUTE - RESP_WAITING_CRUD - RESP_WAITING_PREPARE_EXECUTE - RESP_WAITING_CURSOR_OPEN The flag is set when ColumnMetaData is forwarded in the validation loop, cleared at every transition into a state that begins a new column- metadata sequence (in dispatch_client_message), at terminal-frame flush in handler_waiting_server_msg, and at init() / reset(). It is deliberately NOT cleared on transition into RESP_WAITING_CURSOR_FETCH — per the X-Protocol spec, ColumnMetaData is sent at Cursor::Open and not re-sent at Cursor::Fetch, so CURSOR_FETCH's allowed-set unconditionally accepts RESULTSET_ROW. The dispatch handler skips the clear at CURSOR_FETCH entry to make this carry-across explicit. The rejection action in handler_waiting_server_msg pops the offending frame from the backend queue (does not forward), emits a fatal error frame to the client, marks backend_conn_ non-reusable so return_backend_to_pool deletes it instead of caching a poisoned connection, sets healthy=false and status_=X_SESSION_CLOSING, and short-circuits before the bytes_recv accounting (so the disallowed frame is not double-counted as forwarded traffic). Tests in mysqlx_message_dispatch_unit-t (20 new assertions across 8 cases) cover: 1. StmtExecute → Row-without-metadata (canonical hostile-backend case the issue called out): expect Error frame, !healthy, X_SESSION_CLOSING. 2. StmtExecute → ColumnMetaData → Row → SQL_STMT_EXECUTE_OK: happy path, terminal flushes seen_column_metadata_. 3. CursorOpen → ColumnMetaData → Row → FETCH_SUSPENDED: terminal, response_state_ resets, flag cleared at boundary (Cursor::Fetch does not consult it). 4. CursorOpen → ColumnMetaData → FETCH_DONE: terminal, clean exit. 5. PreparePrepare → SQL_STMT_EXECUTE_OK (allowed-set is OK-only): expect rejection, X_SESSION_CLOSING. 6. PreparePrepare → OK: happy path. 7. STMT_EXECUTE → ColumnMetaData → NOTICE → Row → SQL_STMT_EXECUTE_OK: NOTICE is universal-allowed, non-terminal, doesn't consume the response. 8. CURSOR_FETCH → Row (with seen_column_metadata_=false): forwarded, NOT rejected (the per-state-pair carve-out for Cursor::Fetch). Tests use a new setup_session_for_validation helper that drives the auth flow, attaches a fake backend over a socketpair, parks the session in WAITING_SERVER_XMSG with the desired response_state_ via the test-only set_response_state_for_test (gated behind MYSQLX_TEST_BUILD), and lets the test write synthetic server frames to the backend half of the pair. Drains pending client-side bytes via a non-blocking helper so the validation hook's output is unambiguous. Closes the rejection-side acceptance criterion of #5694. Refs: #5694.	2 months ago
Rene Cannao	796f336cb0	feat(mysqlx): expose stats_mysqlx_routes via runtime-view refresh ... stats_mysqlx_routes was previously declared (DDL registered in mysqlx_admin_schema.cpp:466) but had no writer — the MysqlxStatsStore::flush_to_sqlite() projection method was implemented months ago and has never been called. Issue #5691 caught that "no caller" gap; admin SELECTs against the table return whatever's left from previous test fixture data, which is empty in normal operation. Wires the projection through the chassis ABI 3 register_runtime_view hook landed in PR #5688. Identical shape to the four runtime_mysqlx_<X> projections already in place for users / routes / endpoints / variables: chassis fires the refresh callback before any admin SELECT references the registered table name, callback rebuilds the projected rows from the canonical in-memory state. The wrinkle vs the existing four: stats data lives in statsdb, not admindb. The chassis-supplied admindb argument is ignored; refresh_stats_routes_view fetches the statsdb handle through the plugin's cached services pointer (mysqlx_context().services-> get_statsdb()) and hands that to flush_to_sqlite. flush_to_sqlite writes a bare table name, so reaching stats_mysqlx_routes via the admindb-attached `stats.` schema would work but adds an unnecessary detour and forces a write through the wrong handle. Mirrors how core's stats___mysql_processlist() writes statsdb directly (lib/ProxySQL_Admin_Stats.cpp:885). Known limitation, flagged in the issue: the chassis only fires plugin runtime views on admin-session SELECTs (gated on `if (admin)` in lib/ProxySQL_Admin.cpp). SELECTs against the stats port (port 6032 with stats credentials) bypass the dispatcher entirely → return the table contents at last refresh. Filing as a separate chassis- side follow-up. Plumbing additions: - mysqlx_admin_schema.cpp pulls in mysqlx_stats.h and defines refresh_stats_routes_view (~10 lines). - The single-line register_runtime_view call goes inside the existing tail-extension guard (`if (services.register_runtime_view != nullptr)`) — same pattern as the other four. - test/tap/tests/unit/Makefile: three test build rules (mysqlx_admin_schema_unit-t, mysqlx_admin_commands_unit-t, mysqlx_admin_disk_commands_unit-t) now also need to compile mysqlx_stats.cpp into the test binary, since mysqlx_admin_schema.cpp now depends on it. Single-line addition per rule. Refs: #5691.	2 months ago
Rene Cannao	04f771d2a3	feat(mysqlx): account per-route bytes_sent / bytes_recv on the data plane ... The MysqlxRouteStats struct already had bytes_sent / bytes_recv atomic fields, but no setter and no caller — issue #5691 caught the third and fourth dead counters on the same dead-code list as conn_ok / conn_used. Adds two setters to MysqlxStatsStore: - record_bytes_sent(route, hg, n) - record_bytes_recv(route, hg, n) Both fetch_add(n, relaxed) under the existing mutex, with a 0-arg fast path that skips the lock entirely (frame.size() == 5 means a header- only frame and is the natural early-out — no underflow guard needed on `size - 5` because the call sites only fire when size > 5). Wires them at the two natural data-plane sites: - MysqlxSession::forward_to_backend — bytes_sent is "X-Protocol payload bytes the proxy forwarded from client to backend", measured as `frame.size() - 5` (strip the 5-byte X-Protocol frame header so the counter tracks operator-meaningful query/CRUD bytes rather than wire-level total). - MysqlxSession::handler_waiting_server_msg — bytes_recv is the same payload-only measure for the backend → proxy → client leg. NOTICE frames are counted (they're part of the data plane the operator paid for forwarding); the OK / EXECUTE_OK / FETCH_DONE family contributes 0 because their payloads are typically empty. Both counters need the route name available at the I/O site without re-reading identity_->default_route per frame. Caches that string into MysqlxSession::route_name_ at the resolve_backend_target() success tail and clears it in init() and reset(). The dataplane sites read route_name_ directly. (identity_ would not be null at those sites in practice — resolve_backend_target() is gated on a populated identity — but route_name_ avoids the optional-deref entirely and side-steps any future code path that nulls identity_ in-flight without a corresponding state transition.) Tests: extends mysqlx_stats_unit-t.cpp to plan(26), adding 4 assertions covering the new bytes_sent / bytes_recv accumulation through flush_to_sqlite, including the 0-arg no-op invariant and that destination_hostgroup is carried through. Refs: #5691.	2 months ago
Rene Cannao	b501fa17dc	genai: harden real persistence and connection bugs ... Fix the remaining concrete defects from the GenAI review pass: - serialize MCP variable access with the handler rwlock instead of relying on external callers - make GenAI/MCP save-to-admin operations transactional and fail closed on SQLite errors - clamp genai_log() newline handling after truncation - use PQconnectdbParams() for libpq connections so credentials and schema values are passed safely - install and uninstall the GenAI plugin alongside mysqlx in the top-level Makefile - add a regression test for stale output handling in get_variable_string() These changes keep the runtime and persistence paths consistent with the branch's plugin lifecycle fixes.	2 months ago
Rene Cannao	967ab401ec	genai: fix lifecycle reloads and drop dead MCP stats registration ... Load GenAI and MCP variables before runtime initialization, reinitialize the AI stack on reload, and make the anomaly hook obey the configured enable flags. Restore the missing MCP command aliases and the disk/config reload verbs, fix the MySQLX packaging filename mismatch, and update the unit harness for the plugin ABI. Also remove the stale stats_mcp_* registrations from the plugin surface. There is no plugin-side writer for those tables in this branch, so advertising them as live schema was misleading. Update the unit test and bootstrap comments to match the actual exposed surface.	2 months ago
Rene Cannao	dffa89de7a	fix(ci): repair non-PROXYSQL40 dbg builds + genai unit-test linking ... CI surfaced two separate build breakages that had been latent since the carve-out steps but only became visible once they ran on the right matrix variants. # lib/MySQL_Session.cpp + lib/PgSQL_Session.cpp: chassis dispatch # block needs PROXYSQL40 guard The query-hook dispatch I added in Step 2.2 references chassis types (`ProxySQL_PluginProtocol`, `ProxySQL_PluginQueryHookPayload`, `ProxySQL_PluginQueryHookResult`, etc.) without an `#ifdef PROXYSQL40` guard. Those types are only declared when ProxySQL_Plugin.h is compiled with `-DPROXYSQL40` — v3.0/v3.1 dbg builds (debian12,-dbg and ubuntu22,-tap on the CI matrix) define neither, so the lib step fails: MySQL_Session.cpp:4477:127: error: 'ProxySQL_PluginProtocol' has not been declared MySQL_Session.cpp:4477:85: error: 'proxysql_has_configured_plugin _query_hook' was not declared in this scope [+ 7 more identical errors in PgSQL_Session.cpp] Wrap both dispatch blocks (MySQL_Session.cpp:~4477-4509, PgSQL_Session.cpp:~2414-2446) in `#ifdef PROXYSQL40`. The block was only meaningful when the chassis is present anyway — the call to `proxysql_has_configured_plugin_query_hook` short-circuits to false when no plugin is loaded, but the SYMBOL doesn't exist at all in non-chassis builds. # test/tap/tests/unit/Makefile: PROXYSQLGENAI autodetect probe was # defining the macro without a way to satisfy the resulting linker # requirements Pre-Step-7 the autodetect probed for `MCP_Tool_Handler` in libproxysql.a; this symbol is plugin-side now, so the probe never matches and PROXYSQLGENAI stays empty. My previous attempt to "fix" the autodetect (probing `invoke_register_schemas_phase`) made it match again — but that defines PROXYSQLGENAI for the WHOLE unit- test build, including tests like `genai_llm_clients_unit-t` and `genai_mcp_endpoint_unit-t` that pull in plugin headers AND instantiate plugin classes. Those classes' definitions live in plugin .cpp files that the unit-test build does NOT link, so the result was: genai_llm_clients_unit-t.cpp:201: undefined reference to `LLM_WriteCallback(void*, unsigned long, ...)' [+5 more] Solution: never auto-define PROXYSQLGENAI for the unit-test build. Tests that legitimately exercise plugin internals (`genai_plugin_anomaly_unit-t`, `genai_plugin_backend_client_unit-t`, `genai_plugin_load_unit-t`, `genai_fts_string_unit-t`) declare a new GENAI_PLUGIN_DEFINES variable on their dedicated rule and add the specific plugin .cpp files they need to the link line. Every other genai_*_unit-t test guards its body in `#ifdef PROXYSQLGENAI ... #else SKIP #endif`, so falling through to the SKIP path keeps CI green. Coverage for those tests in non-genai builds was already zero — the autodetect was never activating PROXYSQLGENAI for them in practice (the probed symbol moved out in Step 6). This commit documents that reality instead of pretending to maintain coverage that didn't exist. # test/tap/tests/unit/genai_fts_string_unit-t.cpp: stub for # MySQL_Tool_Handler::execute_query The new dedicated rule for genai_fts_string_unit-t compiles plugins/genai/src/MySQL_FTS.cpp directly into the test binary. MySQL_FTS::index_table calls MySQL_Tool_Handler::execute_query — but this test only exercises the pure string helpers (sanitize_name / escape_*) and never reaches index_table, so the call site is unreachable at runtime. The linker still needs a body, so we provide an empty stub directly in the test .cpp. The real implementation lives in MySQL_Tool_Handler.cpp which we can't link without dragging in the full plugin runtime stack.	2 months ago
Rene Cannao	978a6f8d2f	fix(plugins/genai): blocker fixes from PR-#5701 review ... Five concrete fixes uncovered by the four-agent review (architecture, code-safety, build/packaging, test coverage): # Blocker 1: per-row INSERT failure leaves transaction half-done In the six MCP_Threads_Handler methods that follow the BEGIN -> DELETE -> prepare -> loop INSERT -> COMMIT pattern, the SAFE_SQLITE3_STEP2 macro inside the loop didn't check `rc`. If a single INSERT failed (constraint violation, disk full, ...), the loop would continue binding subsequent rows and the COMMIT below would either succeed with partial data or be left dangling. Hoist the per-row binding into three insert_*_row(stmt, row) -> bool helpers in an anonymous namespace, then have every caller check the return value and `ROLLBACK; return false/return;` on failure. Affects save_auth_profiles_to_admin_table, save_target_profiles_to _admin_table, save_query_rules_to_admin_table, project_auth_profiles _to_runtime_view, project_target_profiles_to_runtime_view, project_query_rules_to_runtime_view, plus the new save_profiles_to _admin_table below. # Blocker 2: cross-table install/save atomicity for profiles target.auth_profile_id is an FK to auth.auth_profile_id. The old mcp_load_target_auth_map_from_admindb / mcp_save_target_auth_map_to_ admindb wrappers ran the two installs / two saves as separate operations under separate locks / transactions. A failure of the second step (or a concurrent LOAD/SAVE racing in between) left target_auth_map rebuilt from a (auth_v2, target_v1) mismatch, or left main.mcp_target_profiles referencing auth_profile_ids that didn't exist in main.mcp_auth_profiles. Add atomic combined methods: - install_profiles_from_admin: reads both source tables BEFORE taking the wrlock (so a slow admindb doesn't gate the listener), then under one wrlock swaps both vectors and rebuilds the joined target_auth_map exactly once. - save_profiles_to_admin_table: snapshots both vectors under one rdlock, then deletes target before auth (FK direction-aware) and inserts auth before target inside a single BEGIN ... COMMIT. The plugin_main.cpp wrappers now route through these atomic methods. The per-table install_*_from_admin / save_*_to_admin_table public methods stay (used by future granular admin verbs and unit tests), but each rebuilds target_auth_map after its own swap as before. # Blocker 3: unit-test PROXYSQLGENAI autodetect probe test/tap/tests/unit/Makefile had been probing libproxysql.a for the `MCP_Tool_Handler` symbol to autodetect PROXYSQLGENAI. Step 7 of the carve-out moved that symbol to plugins/genai/, so the probe silently found 0 matches and stopped defining -DPROXYSQLGENAI for unit-test builds. Switch the probe to invoke_register_schemas_phase (the chassis-exclusive symbol that's already used to probe PROXYSQL40) so the macro auto-defines whenever the chassis is available — which matches when genai-plugin unit tests can actually be built and exercised. # Blocker 4: triplet coverage through dispatch surface Extend genai_plugin_load_unit-t to drive the full install/save/project flow through the production dispatch path: - seed mcp_<X> tables in admindb - dispatch "LOAD MCP PROFILES TO RUNTIME" via mgr.dispatch_admin_ command (same path admin SQL takes) - manually invoke refresh_runtime_views_for_query (the chassis hook that fires on every admin SELECT) and assert runtime_mcp_<X> rows match the source - mutate main.<X> directly to simulate an operator stomp, then dispatch "SAVE MCP PROFILES TO MEMORY" and assert the in-memory snapshot was written back over the stomp This is the end-to-end coverage that plugin_runtime_views_unit-t can't provide (it uses synthetic callbacks) and that a standalone genai_mcp_config_store unit test would have provided at the cost of stubbing 9 plugin classes. Plan bumped from 25 -> 36. # Should-fix: atomic fence + GloGATH lifetime genai_anomaly_embed_fn was std::atomic with relaxed ordering; the comment claimed nothing else needed synchronising. Wrong: the embed function dereferences GloGATH, a non-atomic global mutated by genai_init / genai_stop. Pair the load with acquire and the stores with release so a reader that observes the non-null pointer also observes the prior `GloGATH = new ...`, and a reader that observes the post-stop nullptr short-circuits before GloGATH's deletion is visible. Acquire/release is free on x86_64 / arm64 — no perf regression. # Should-fix: harden DEB awk substitution + RPM glob DEB entrypoint: replace the inline awk -v repl="$..." substitution (which mangled paths containing awk metacharacters via gsub) with a file-backed awk getline that reads the replacement block verbatim. Add a fail-fast grep for the placeholder after substitution so a silently-broken substitution can't ship a malformed package. RPM (rhel + suse): the spec gates `/usr/lib/proxysql/*.so` on %if 0%{?with_plugins}, but rpmbuild aborts on "File not found by glob" if the directory is empty (silent plugin build failure + with_plugins=1). Have the entrypoint compgen-check whether any .so actually got staged and only pass --define "with_plugins 1" when true.	2 months ago
Rene Cannao	b840468cd5	test(unit): align genai_plugin_load + register plugin_runtime_views ... Two unit-test-side fixes uncovered by running unit-tests-g1 against the new ABI-3 / Phase-B plugin shape: # genai_plugin_load_unit-t * setup_admindb_schema was using a pre-Step-4.G shape for the mcp_auth_profiles / mcp_target_profiles tables — the new install_*_from_admin reads the full canonical column set (use_ssl, ssl_mode, comment for auth; description, comment for targets) and was failing with `no such column: use_ssl`. Fixture now mirrors the DDL in include/ProxySQL_Admin_Tables_Definitions.h and also creates mcp_query_rules + runtime_mcp_query_rules so the newly-added install_query_rules_from_admin path also has tables to read. * As of Step 4.G the genai plugin publishes its admin/config/stats table set via genai_register_schemas (Phase B), not init. The test was calling load → init_all → start_all and the table-count assertions saw zero rows because Phase B never fired. Add an explicit invoke_register_schemas_phase between load and init_all (matches the order ProxySQL itself uses at startup) and bump the plan from 25 to 26 to cover the new ok(). # Makefile UNIT_TESTS list plugin_runtime_views_unit-t.cpp lives in test/tap/tests/unit/ and exercises the chassis ABI-3 register_runtime_view dispatch surface that landed in PR #5688. It had a build rule but wasn't in UNIT_TESTS, so the runner reported "(not-found)" for it. Wire it in alongside the other plugin-chassis unit tests under PROXYSQL40.	2 months ago
Rene Cannao	16253a1c99	feat(plugins/genai): adopt ABI-3 separation-of-duties for runtime_mcp_* ... Wire the genai plugin into the plugin-chassis ABI-3 contract added in PR #5688: Admin owns the editable mcp_<X> tables, the module owns the in-memory snapshot under its own mutex, and runtime_mcp_<X> is a chassis-projected view repopulated lazily by a refresh callback before each admin SELECT. No more triple-storage of profile / query-rule data. # MCP_Threads_Handler: per-table install / save / project triplets Replaces the monolithic load_target_auth_map(joined-resultset) with three independent triplets, mirroring the mysqlx config_store rework: install_<X>_from_admin reads main.mcp_<X>, replaces the in-memory snapshot vector under wrlock, rebuilds the joined target_auth_map (for profiles) save_<X>_to_admin_table REPLACEs main.mcp_<X> from the snapshot project_<X>_to_runtime_view DELETE+INSERT runtime_mcp_<X> from the snapshot — invoked only by the chassis runtime-view refresh dispatcher Two new vectors (auth_profiles_, target_profiles_, query_rules_) hold the authoritative per-table rows; target_auth_map remains as the joined view consumed by the listener and is rebuilt whenever either profile vector changes. # Plugin descriptor + table registration genai_register_admin_tables now also calls services->register_runtime_view for runtime_mcp_auth_profiles, runtime_mcp_target_profiles, and runtime_mcp_query_rules, pointing each at a project_*_to_runtime_view trampoline. The runtime_mcp_<X> tables stay registered for their CREATE TABLE schema only — their rows are entirely owned by the per-SELECT projection callbacks from this point on. # Admin SQL surface LOAD MCP PROFILES TO RUNTIME / LOAD MCP QUERY RULES TO RUNTIME now go through install_<X>_from_admin and never touch runtime_mcp_<X>. SAVE MCP PROFILES TO MEMORY is added (was missing pre-ABI-3) and joins SAVE MCP QUERY RULES TO MEMORY in pulling from the module snapshot instead of the runtime view. Query_Tool_Handler::refresh_target_registry no longer re-reads runtime_mcp_<X> via SQLite on every refresh; it consumes the in-memory snapshot directly through GloMCPH->get_all_target_auth_contexts(). # Drop stale GENAI:-prefix tests test/tap/tests/genai_{async,embedding_rerank,live_validation}-t.cpp were left over from before Step 4.A removed the GENAI: query-prefix escape hatch — they would all have failed under ai-g1. Delete them and their groups.json / ai/README.md references; equivalent coverage now flows through the MCP /mcp/ai endpoint and the genai_module-t suite that already drives the plugin surface.	2 months ago
Rene Cannao	25e4388ae6	feat(plugins/genai): Step 7 follow-up — Phase B schema registration + ai-g1 plugin wiring ... Three small but related fixes that unblock running the ai-g1 integration suite against the post-Step-7 build. 1. Plugin: register MCP admin tables in Phase B (register_schemas), not Phase C (init). The chassis ABI guarantees that tables registered via `services->register_table` from a plugin's `register_schemas` callback are merged into the admin SQLite schema BEFORE admin bootstrap creates the DB. Registering at init() instead is too late: by then the DB is already created without the plugin-declared tables. The genai plugin was registering at init(). Symptom: at plugin start (Phase D) the call into `mcp_load_target_auth_map_from_admindb` errored out with "no such table: runtime_mcp_auth_profiles" because the table never got created. plugins/genai/src/plugin_main.cpp: - New `genai_register_schemas` callback that invokes `genai_register_admin_tables(services)`. - Plugin descriptor gains the `register_schemas` field (mirrors plugins/mysqlx pattern; Step 0's PROXYSQL_PLUGIN_ABI_VERSION = 2 supports it). - Removed the redundant call in genai_init(). 2. Core: drop PROXYSQLGENAI gates that Step 7 left around the SET mcp-X / SET genai-X validation paths. Step 7 stopped passing -DPROXYSQLGENAI to core compiles, so the `#ifdef PROXYSQLGENAI` blocks I'd left in lib/Admin_Handler.cpp for the loose-prefix `mcp-*` / `genai-*` validators (Step 4.F / Step 5) silently expanded to nothing. Symptom: `SET mcp-port=6071` admin SQL got "Unknown global variable". lib/Admin_Handler.cpp: - Loose mcp-* / genai-* prefix checks in is_valid_global_variable are now unconditional (no #ifdef). - The MCP VARIABLES disk-side handler (LOAD MCP VARIABLES FROM DISK / SAVE MCP VARIABLES TO DISK) is also un-gated; it's pure SQL DML against global_variables LIKE 'mcp-%' with no plugin runtime dependency. The plugin-side runtime push remains the LOAD MCP VARIABLES TO RUNTIME verb registered in plugin_commands.cpp. 3. Test infra: bind-mount the genai plugin .so + per-group cnf for ai-g1, mirroring the mysqlx-soak / mysqlx-e2e pattern. Without these, the ai-g1 setup hook fails at SET mcp-port because the proxysql container's cnf doesn't list `plugins=("...")` and the .so isn't visible inside the container. test/infra/control/start-proxysql-isolated.bash: - PROXYSQL_LOAD_GENAI_PLUGIN=1 (set by the group's env.sh) now triggers a `-v ${WORKSPACE}/plugins/genai/...so:/usr/lib/proxysql/...so:ro` bind into the proxysql container. test/tap/groups/ai/proxysql-ci.cnf (new): - Identical to test/infra/control/proxysql-ci.cnf except for the trailing `plugins=("/usr/lib/proxysql/ProxySQL_GenAI_Plugin.so")` line that asks the chassis to load the genai plugin at Phase A. test/tap/groups/ai-g1/env.sh: - PROXYSQL_LOAD_GENAI_PLUGIN=1 - PROXYSQL_CONFIG_OVERRIDE=…/test/tap/groups/ai/proxysql-ci.cnf After this commit the ai post-infras hook (`MCP configuration completed` … `MySQL test data seeding completed` … `PostgreSQL test data seeding completed` … `AI post-infras hook completed`) runs to completion against the post-carve-out proxysql + genai plugin .so.	2 months ago
Rene Cannao	4552236700	feat(plugins/genai): Step 6 — move RAG/Vector/discovery surface into the plugin ... Final file-move step in the carve-out: relocates the leaf modules that didn't depend on (and weren't depended on by) anything still in core. After this commit, the only PROXYSQLGENAI-guarded code remaining in libproxysql.a is `MCP_Tool_Handler` — the base class the plugin's AI_Tool_Handler / RAG_Tool_Handler inherit from — and that lives in core only because removing it would force pulling those derived plugin classes into core just for the inheritance edge. Step 7 deletes the PROXYSQLGENAI macro entirely. Files moved (git mv preserves history) Headers (include/ -> plugins/genai/include/) AI_Vector_Storage.h MySQL_Catalog.h Discovery_Schema.h Static_Harvester.h PgSQL_Static_Harvester.h Sources (lib/ -> plugins/genai/src/) AI_Vector_Storage.cpp MySQL_Catalog.cpp Discovery_Schema.cpp Static_Harvester.cpp PgSQL_Static_Harvester.cpp Core changes were minimal because these are leaf modules: include/proxysql.h: drop the AI_Vector_Storage.h include. include/cpp.h: drop AI_Vector_Storage / Discovery_Schema / MySQL_Catalog / Static_Harvester / PgSQL_Static_Harvester includes (replaced with a single carve-out history comment that Step 7 will delete entirely). lib/ProxySQL_Admin.cpp: drop the Discovery_Schema.h include — its only live use was via the now-#if-0'd save/load_mcp_query_rules blocks 4.C left behind for follow-up reference. lib/Makefile: drop AI_Vector_Storage.oo, MySQL_Catalog.oo, Discovery_Schema.oo, Static_Harvester.oo, PgSQL_Static_Harvester.oo from the GenAI _OBJ_CXX list. Test fixture probe update test/tap/tests/unit/Makefile: PROXYSQLGENAI auto-detect probe switched from `AI_Vector_Storage` (now plugin-only) to `MCP_Tool_Handler` (the last GenAI-tier symbol still in core). Step 7 deletes the auto-detect entirely. All three plugin unit tests pass: genai_plugin_load_unit-t : 25/25 genai_plugin_anomaly_unit-t : 6/6 genai_plugin_backend_client : 27/27 Binary size deltas: src/proxysql 159 MB -> 152 MB (-7 MB) plugins/genai/...Plugin.so 47 MB -> 55 MB (+8 MB) Net process RSS roughly neutral, as the design predicted. Carve-out scoreboard: Step 0-2 : infrastructure Step 3 : ~1.1 K LOC (Anomaly_Detector) Step 4.B : ~0.5 K LOC (backend_client) Step 4.C : ~13 K LOC (MCP subsystem + tool handlers + MySQL_FTS) Step 4.F : runtime-reconfig admin SQL routed through plugin commands Step 4.G : MCP admin / config / stats tables registered via plugin ABI Step 5 : ~7 K LOC (GenAI/LLM/AI/RAG) Step 6 : ~5 K LOC (AI_Vector_Storage + Discovery_Schema + MySQL_Catalog + Static_Harvester pair) Pending: Step 7 — delete the PROXYSQLGENAI macro from build flags and any remaining `#ifdef PROXYSQLGENAI` blocks.	2 months ago
Rene Cannao	28cf4ceab8	feat(plugins/genai): Step 5 — move GenAI/LLM/AI surface into the plugin ... Largest carve-out commit since 4.C: moves the GenAI / LLM / AI / RAG surface (about 7K LOC) out of core into plugins/genai/. After this commit, libproxysql.a no longer publishes any of GenAI_Threads_Handler, LLM_Bridge, LLM_Clients, AI_Features_Manager, AI_Tool_Handler, RAG_Tool_Handler, or the legacy GloGATH/GloAI globals. Files moved (git mv preserves history) Headers (include/ -> plugins/genai/include/) GenAI_Thread.h LLM_Bridge.h AI_Features_Manager.h AI_Tool_Handler.h RAG_Tool_Handler.h (promoted from Step 6 — bidirectional dep with AI_Features_Manager forced the early move, same way MySQL_FTS.h got promoted in 4.C) Sources (lib/ -> plugins/genai/src/ or …/tool_handlers/) GenAI_Thread.cpp LLM_Bridge.cpp LLM_Clients.cpp (.cpp without a header; class decls live in LLM_Bridge.h) AI_Features_Manager.cpp AI_Tool_Handler.cpp -> tool_handlers/ RAG_Tool_Handler.cpp -> tool_handlers/ Plugin lifecycle takeover plugins/genai/src/plugin_main.cpp: - Defines plugin-local GloGATH and GloAI alongside the existing GloMCPH (all three are .so-internal aliases for the contexts genai_init constructs). - genai_init now also constructs GenAI_Threads_Handler (calls ->init()) and AI_Features_Manager (calls ->init()), mirroring the pre-Step-5 ProxySQL_Main_init_GenAI_module sequence. - genai_stop tears them down in reverse order. plugins/genai/src/ProxySQL_MCP_Server.cpp: - AI / RAG endpoint registration re-enabled (was #if 0'd in 4.C because the constructors weren't reachable from the plugin then). /mcp/ai and /mcp/rag are once again live when the operator has GloAI configured. plugins/genai/src/Anomaly_Detector.cpp + plugin_main.cpp: - get_query_embedding() now calls through a function-pointer hook (genai_anomaly_embed_fn) that plugin_main installs in genai_init and clears in genai_stop. This indirection lets the test fixture (which compiles Anomaly_Detector.cpp directly into a test binary, without GenAI_Thread.cpp) link cleanly — the pointer stays null, the detector short-circuits to empty embedding, and the existing nullptr-guard in check_embedding_similarity does the rest. Core neutralization src/main.cpp: - Drops the GloGATH / GloAI globals, ProxySQL_Main_init_GenAI_module definition + call, the shutdown branch, and the `if (GloGATH) init_genai_variables()` call. lib/Admin_Bootstrap.cpp / Admin_FlushVariables.cpp / Admin_Handler.cpp / ProxySQL_Admin.cpp / MySQL_Session.cpp / ProxySQL_Admin_Stats.cpp: - Drop the corresponding `#include` and `extern` declarations. - flush_genai_variables___* stubbed (mirrors the 4.C MCP pattern; original bodies preserved in `#if 0` for the upcoming LOAD/SAVE GENAI verbs port). lib/Admin_Handler.cpp: - Loose-prefix accept for `genai-*` in is_valid_global_variable (mirrors the 4.F mcp-* fix), so SET genai-X=Y still reaches main.global_variables; runtime push happens via the plugin's LOAD GENAI VARIABLES TO RUNTIME (registered in a follow-up). include/cpp.h, include/proxysql.h: - Removed the AI_Features_Manager.h / LLM_Bridge.h / GenAI_Thread.h / AI_Tool_Handler.h / RAG_Tool_Handler.h includes that no longer resolve from core's -I path. test/tap/test_helpers/test_globals.cpp: - Drops the matching `#include`s and stub globals. Test fixture test/tap/tests/unit/Makefile: - PROXYSQLGENAI auto-detect probe switched from `GenAI_Thread` (now plugin-only) to `AI_Vector_Storage` (still in core for Step 6). After Step 6 the probe will switch again, and Step 7 deletes the auto-detect entirely. All three plugin unit tests pass: - genai_plugin_load_unit-t : 25/25 - genai_plugin_anomaly_unit-t : 6/6 - genai_plugin_backend_client : 27/27 Carve-out scoreboard: Step 0-2 : infrastructure Step 3 : ~1.1 K LOC (Anomaly_Detector) Step 4.B : ~0.5 K LOC (backend_client) Step 4.C : ~13 K LOC (MCP subsystem) Step 4.F : runtime-reconfig admin SQL routed through plugin commands Step 4.G : MCP admin / config / stats tables registered via plugin ABI Step 5 : ~7 K LOC (GenAI/LLM/AI/RAG) Pending: Step 6 (AI_Vector_Storage, MySQL_Catalog, Discovery_Schema, Static_Harvester, PgSQL_Static_Harvester) and Step 7 (delete the PROXYSQLGENAI macro entirely).	2 months ago
Rene Cannao	9078fd3150	feat(plugins/genai): Step 4.G — register MCP admin tables via plugin ABI ... Moves MCP admin / config / stats SQLite table registration from lib/Admin_Bootstrap.cpp into the plugin via the chassis register_table service. Pure refactor: the same DDL strings produce the same SQLite tables in the same SQLite DBs; only the registration path changes. New plugins/genai/src/plugin_tables.cpp registers 14 tables: admin DB (6) mcp_query_rules, runtime_mcp_query_rules mcp_auth_profiles, runtime_mcp_auth_profiles mcp_target_profiles, runtime_mcp_target_profiles config DB (3) mcp_query_rules, mcp_auth_profiles, mcp_target_profiles (no runtime_* in config; matches the pre-4.G layout) stats DB (5) stats_mcp_query_tools_counters, _reset stats_mcp_query_digest, _reset stats_mcp_query_rules The DDL strings still live in include/ProxySQL_Admin_Tables_Definitions.h for now — plugin_tables.cpp #includes the header. Migrating the macros into a plugin-private header is a follow-up gated on no remaining core caller of the MCP table macros (Step 7 territory). Core changes (lib/Admin_Bootstrap.cpp): - Deleted the 9 admin/config insert_into_tables_defs() entries inside `#ifdef PROXYSQLGENAI` for mcp_*. - Deleted the 5 stats insert_into_tables_defs() entries similarly. - Replaced both blocks with a one-line comment pointing at the new plugin owner. Plugin lifecycle: - genai_init() now calls genai_register_admin_tables() right after genai_register_admin_commands(). Both run during chassis Phase C (init), before admin module bootstrap so the chassis dispatcher merges plugin tables into the admin DB schema before any DML. Test extension (test/tap/tests/unit/genai_plugin_load_unit-t.cpp): - Updated the "skeleton registers no tables (yet)" assertions to expect 6 admin / 3 config / 5 stats — matching plugin_tables.cpp. - 25/25 tests passing. What this changes for operators (no-op): - Same SQL: SHOW TABLES still shows mcp_*, runtime_mcp_*, stats_mcp_*. - Same DDL: column lists, primary keys, defaults — all preserved. - Same admindb / configdb / statsdb attachment.	2 months ago
Rene Cannao	f90307ad75	feat(plugins/genai): Step 4.F — register LOAD/SAVE MCP QUERY RULES TO/FROM RUNTIME ... Adds the last LOAD/SAVE pair the pre-4.C admin SQL surface had: the MCP query rules cache held by Query_Tool_Handler's Discovery_Schema catalog. New helpers in plugins/genai/src/plugin_main.cpp: mcp_load_query_rules_to_runtime(ctx) SELECT * FROM main.mcp_query_rules WHERE active=1 ORDER BY rule_id → catalog->load_mcp_query_rules(rs). Mirrors the pre-4.C ProxySQL_Admin::load_mcp_query_rules_to_runtime. mcp_save_query_rules_from_runtime(ctx, runtime) Pulls catalog->get_mcp_query_rules() and REPLACEs the rows in main.mcp_query_rules (or runtime_mcp_query_rules when runtime=true). Mirrors the pre-4.C ProxySQL_Admin::save_mcp_query_rules_from_runtime. Both helpers fail cleanly (return false, log to stderr) when the MCP listener isn't running — there's no in-memory Query_Tool_Handler cache to read or write. Pre-4.C behaviour was to attempt a load_mcp_server() self-recovery in that case; we no longer have that shortcut from outside the listener lifecycle, so callers get a clear error instead of an opaque retry. New verbs registered in plugin_commands.cpp: LOAD MCP QUERY RULES TO RUNTIME aliases: FROM MEMORY, FROM MEM, TO RUN SAVE MCP QUERY RULES TO MEMORY aliases: TO MEM, FROM RUNTIME, FROM RUN Required new core-side includes in plugin_main.cpp for the Discovery_Schema and Query_Tool_Handler types these helpers traffic in. The .so already linked against them via the chassis --whole-archive trick on libproxysql.a; this is just a header dependency. Test extension (test/tap/tests/unit/genai_plugin_load_unit-t.cpp): - 6 new assertions for the QUERY RULES canonical + 5 aliases. - 25/25 tests passing. Carve-out scoreboard, runtime reconfig surface restored: LOAD MCP VARIABLES TO RUNTIME / FROM MEMORY / FROM MEM / TO RUN LOAD MCP PROFILES TO RUNTIME / FROM MEMORY / FROM MEM / TO RUN LOAD MCP QUERY RULES TO RUNTIME / FROM MEMORY / FROM MEM / TO RUN SAVE MCP VARIABLES TO MEMORY / TO MEM / FROM RUNTIME / FROM RUN SAVE MCP QUERY RULES TO MEMORY / TO MEM / FROM RUNTIME / FROM RUN Still pending: SET mcp-X=Y (chassis ABI gap, separate enhancement).	2 months ago
Rene Cannao	7a4f08416e	feat(plugins/genai): Step 4.F — register SAVE MCP VARIABLES TO MEMORY ... Adds the SAVE direction (runtime → main.global_variables) to the plugin's admin SQL surface, mirroring the LOAD direction wired in the previous commit. New helper plugins/genai/src/plugin_main.cpp::mcp_save_variables_to_admindb Walks ctx.mcp->get_variables_list(), reads each runtime value via ctx.mcp->get_variable, and REPLACEs the corresponding "mcp-<name>" rows in main.global_variables. Owns its own MCP_Threads_Handler wrlock so callers don't have to coordinate. Mirrors the pre-4.C flush_mcp_variables___runtime_to_database with runtime=false (no runtime_global_variables view write). New verb registered in plugins/genai/src/plugin_commands.cpp: SAVE MCP VARIABLES TO MEMORY (canonical) aliases: SAVE MCP VARIABLES TO MEM SAVE MCP VARIABLES FROM RUNTIME SAVE MCP VARIABLES FROM RUN Same alias convention as the LOAD verbs and as plugins/mysqlx. The on-disk write ("SAVE MCP VARIABLES TO DISK") stays in core's admin_handler_command_load_or_save — it's a pure SQL DML that copies main.* to disk.* and doesn't need plugin runtime state. Test extension (test/tap/tests/unit/genai_plugin_load_unit-t.cpp): - 4 new TAP assertions for the SAVE canonical + 3 aliases. - 19/19 tests passing. Carve-out scoreboard, runtime reconfig surface restored: LOAD MCP VARIABLES TO RUNTIME / FROM MEMORY / FROM MEM / TO RUN LOAD MCP PROFILES TO RUNTIME / FROM MEMORY / FROM MEM / TO RUN SAVE MCP VARIABLES TO MEMORY / TO MEM / FROM RUNTIME / FROM RUN Still pending: - SET mcp-X=Y (chassis ABI lacks per-variable namespace registration; would need a new ABI surface). - LOAD/SAVE MCP QUERY RULES TO/FROM RUNTIME (Query_Tool_Handler in plugin owns the runtime cache; helpers come in next commit).	2 months ago
Rene Cannao	ae13958995	feat(plugins/genai): Step 4.F (continued) — register LOAD MCP {VARIABLES,PROFILES} TO RUNTIME ... Adds plugins/genai/src/plugin_commands.cpp registering two admin SQL verbs through the chassis command registry, restoring runtime reconfiguration that 4.C disabled. Verbs registered (each with FROM MEMORY / FROM MEM / TO RUN aliases): LOAD MCP VARIABLES TO RUNTIME -> mcp_load_variables_from_admindb() -> mcp_start_listener_if_enabled() Re-pushes mcp-* values from main.global_variables into the running MCP_Threads_Handler, then re-evaluates the listener state (start if newly enabled). Replaces the FIXME-stubbed ProxySQL_Admin::flush_mcp_variables___database_to_runtime path. LOAD MCP PROFILES TO RUNTIME -> mcp_load_target_auth_map_from_admindb() Refreshes runtime_mcp_auth_profiles / runtime_mcp_target_profiles from main.* and rebuilds the in-memory target_auth_map. Replaces the FIXME-stubbed init_mcp_variables auth-map block. The chassis dispatcher in lib/Admin_Handler.cpp:3987 runs BEFORE admin_handler_command_load_or_save, so any admin SQL matching a registered plugin verb routes to the plugin and the (now no-op) hardcoded core handlers never fire. No core changes needed. What's still stubbed (deferred to follow-up commits): - SAVE MCP VARIABLES TO MEMORY / FROM RUNTIME (needs runtime -> admindb push in plugin) - SAVE MCP PROFILES TO DISK (writes runtime view back to disk.*) - LOAD/SAVE MCP QUERY RULES (Query_Tool_Handler in plugin owns them, but the runtime->admindb push isn't wired through plugin yet) - SET mcp-X=Y (chassis ABI has no register_variable_namespace; would need either a new ABI surface or per-variable register_command — defer to a chassis enhancement) Refactor: moved mcp_load_variables_from_admindb, mcp_load_target_auth_map_from_admindb, mcp_start_listener_if_enabled out of the anonymous namespace in plugin_main.cpp so plugin_commands.cpp can call them. Forward declarations added to genai_plugin.h. Test extension (test/tap/tests/unit/genai_plugin_load_unit-t.cpp): - 7 new TAP assertions verifying canonical SQL and aliases resolve through the plugin manager's resolve_alias_to_canonical, which is the same path admin SQL dispatch uses. - 15/15 tests passing.	2 months ago
Rene Cannao	84216c6670	feat(plugins/genai): Step 4.F (MVP) — restore MCP listener auto-start ... Step 4.C left the MCP listener inert: the admin-SQL surface that used to drive load_mcp_server() was stubbed in core, so no path existed from "mcp_enabled=true in admin DB" to "ProxySQL_MCP_Server is up". This commit restores the startup path entirely from inside the plugin, so the listener auto-starts again whenever the operator has it enabled in admin tables — which is the 80% case. genai_start() (plugins/genai/src/plugin_main.cpp) now does, in order: 1. mcp_load_variables_from_admindb(ctx) SELECT variable_name, variable_value FROM main.global_variables WHERE variable_name LIKE 'mcp-%'. For each row, strip the 'mcp-' prefix and call ctx.mcp->set_variable(name, value). Mirrors the pre-4.C flush_mcp_variables___database_to_runtime in lib/Admin_FlushVariables.cpp. 2. mcp_load_target_auth_map_from_admindb(ctx) Refresh runtime_mcp_auth_profiles / runtime_mcp_target_profiles from main.*, JOIN them, and pass the result set to ctx.mcp->load_target_auth_map(rs). Mirrors the pre-4.C ProxySQL_Admin::init_mcp_variables target-auth-map block. 3. mcp_start_listener_if_enabled(ctx) If ctx.mcp->variables.mcp_enabled, check SSL cert availability (when use_ssl=true), check_port_availability(port), and new ProxySQL_MCP_Server(port, ctx.mcp) + start(). Mirrors the startup branch of the pre-4.C ProxySQL_Admin::load_mcp_server(). What this MVP does NOT yet cover (deferred to 4.F continued / 4.G): - Runtime reconfiguration via admin SQL: "LOAD MCP VARIABLES TO RUNTIME", "LOAD MCP PROFILES …", "SAVE MCP …", and "SET mcp-X=Y". Those still hit the FIXME stubs in core left by 4.C. Future work registers them through the plugin command registry (services->register_command). - The variable-listener-restart branch from the original load_mcp_server (port/SSL change → tear down + restart). Today, operators must restart proxysql to pick up mcp-port / mcp-use-ssl changes. Test fixture update (test/tap/tests/unit/genai_plugin_load_unit-t.cpp) Replaced the fake `char*`-cast-to-SQLite3DB stubs with real in-memory SQLite3DB instances pre-seeded with the table shapes genai_start now reads. Without this, the test segfaults the moment the plugin tries to query the fake "DB". The schema is empty — no rows in global_variables / mcp_*_profiles — which exercises the "fresh install, MCP disabled" path: all queries succeed and return empty result sets, so the listener correctly stays down. All three plugin unit tests pass: - genai_plugin_load_unit-t : 8/8 - genai_plugin_anomaly_unit-t : 6/6 - genai_plugin_backend_client : 27/27	2 months ago
Rene Cannao	5fe286e972	feat(plugins/genai): Step 4.C — move MCP subsystem into the plugin ... Per docs/superpowers/plans/2026-04-19-step4-mcp-subsystem-move.md "4.C / 4.D / 4.E merge" revision. This is the largest single carve-out commit so far: ~13 K LOC physically relocated, plus surgical edits across the core admin SQL surface to neutralize references to the moved code. What moved (git mv preserves history) Tool handlers (lib/<name>.cpp -> plugins/genai/src/tool_handlers/<name>.cpp; include/<name>.h -> plugins/genai/include/<name>.h) Admin_Tool_Handler, Cache_Tool_Handler, Config_Tool_Handler, MySQL_Tool_Handler, Observe_Tool_Handler, Query_Tool_Handler, Stats_Tool_Handler. MCP listener stack (lib/<name>.cpp -> plugins/genai/src/<name>.cpp; include/<name>.h(pp) -> plugins/genai/include/...) MCP_Endpoint, MCP_Thread, ProxySQL_MCP_Server. Promoted from Step 6 (bidirectional dep with MySQL_Tool_Handler): MySQL_FTS. MCP_Tool_Handler base stays in core because AI_Tool_Handler and RAG_Tool_Handler (Steps 5/6 scope) inherit from it; moving the base would force pulling those in too. Plugin lifecycle takeover plugins/genai/include/genai_plugin.h: GenAIPluginContext now holds the MCP_Threads_Handler*. plugins/genai/src/plugin_main.cpp: genai_init constructs the MCP_Threads_Handler and assigns to a plugin-local GloMCPH symbol (the legacy global name, kept for the moved tool handlers that still reference it; the symbol no longer exists in core). genai_stop tears down on shutdown. genai_start is a no-op for now; the MCP listener auto-start is gated on the admin-SQL surface that 4.F restores. plugins/genai/src/ProxySQL_MCP_Server.cpp: AI_Tool_Handler and RAG_Tool_Handler endpoint registration is disabled via stub assignments (the headers stay included so member-pointer types resolve). Steps 5/6 re-enable when those classes move into the plugin. Core neutralization (FIXME stubs awaiting 4.F/4.G) src/main.cpp: GloMCPH global removed; ProxySQL_Main_init_MCP_module removed; shutdown block deleted. lib/ProxySQL_Admin.cpp: init_mcp_variables, load_mcp_server, load_mcp_query_rules_to_runtime, save_mcp_query_rules_from_runtime stubbed to no-ops; original bodies preserved inside `#if 0` for 4.F reference. lib/Admin_FlushVariables.cpp: flush_mcp_variables___database_to_runtime and flush_mcp_variables___runtime_to_database stubbed similarly. lib/ProxySQL_Admin_Stats.cpp: stats___mcp_query_tools_counters, stats___mcp_query_digest, stats___mcp_query_rules stubbed. lib/Admin_Handler.cpp: has_variable("mcp-…") and the load_target_auth_map call site in LOAD MCP PROFILES … stubbed. Result: the MCP listener does NOT start automatically between this commit and 4.F (mcp-* admin variables are not pushed into the running plugin). Documented in the plan's "4.C/4.D/4.E merge" section. src link change: --whole-archive on libproxysql.a src/Makefile: the proxysql binary now links libproxysql.a with -Wl,--whole-archive so all symbols are exported via -Wl,--export-dynamic, allowing the genai plugin .so to dlopen cleanly even when core code paths don't reference moved-but-still- in-core classes (LLM_Bridge, Discovery_Schema, MySQL_Catalog, Static_Harvester, …) directly. test/tap/tests/unit/Makefile gets the same change so unit tests can dlopen the plugin. Without this, ld dead-code-eliminates the unreferenced symbols and dlopen fails with "undefined symbol". Tests passing - genai_plugin_anomaly_unit-t : 6/6 - genai_plugin_load_unit-t : 8/8 (lifecycle init/start/stop + admin table registration) - genai_plugin_backend_client_unit-t : 27/27 Carve-out scoreboard Step 0-2 : 0 (infrastructure only) Step 3 : ~1.1 K (Anomaly_Detector) Step 4.B : ~0.5 K (backend_client helper, no callers yet) Step 4.C : ~13 K (this commit; MCP subsystem) Pending : Steps 4.F, 4.G, 5, 6, 7	2 months ago
Rene Cannao	4080606672	feat(plugins/genai): Step 4.B — add backend_client helper for local-proxy connections ... Per docs/superpowers/plans/2026-04-19-step4-mcp-subsystem-move.md "Sub-step 4.B", introduces the helper that 4.D will use to rewire Query_Tool_Handler and MySQL_Tool_Handler from direct-to-backend dialing to dialing 127.0.0.1:<local-proxy-port>. This commit is functionally a no-op: nothing in the plugin or core calls the new helper yet. The .so links it in for future use; only the unit test exercises it. New files: plugins/genai/include/backend_client.h BackendTarget, MySQL/PgSQLDialResult, dial_*, dial_*_local, LocalProxyEndpoint, parse_interfaces_first_tcp, resolve_*_endpoint. plugins/genai/src/backend_client.cpp dial_mysql / dial_pgsql: thin wrappers around mysql_real_connect / PQconnectdb mirroring the dial sequence currently inlined in Query_Tool_Handler.cpp. No CLIENT_MULTI_STATEMENTS change of semantics relative to existing code. dial_*_local: resolve local proxy endpoint via admindb, then dial. plugins/genai/src/local_proxy_endpoint.cpp Reads mysql-interfaces / pgsql-interfaces from global_variables, parses the semicolon-separated host:port format (mirrors MySQL_Thread::listener_add's parser at lib/MySQL_Thread.cpp ~L240), canonicalises 0.0.0.0 -> 127.0.0.1 and :: -> ::1, skips Unix-socket tokens. No caching today; trivial to add when 4.D wires real callers. test/tap/tests/unit/genai_plugin_backend_client_unit-t.cpp 27 TAP assertions: parser variants (IPv4/IPv6/wildcards/multi/Unix- skip/whitespace/port-bounds), null-admindb guards on every public entry point, invalid-host/zero-port guards on direct-dial. Full MySQL handshake left for the 4.D integration coverage. Plumbing: - plugins/genai/Makefile: compile both new .cpp into the .so. - test/tap/tests/unit/Makefile: build genai_plugin_backend_client_unit-t using the same compile-twice trick as genai_plugin_anomaly_unit-t. - test/tap/groups/groups.json: register the new test in unit-tests-g1 with @proxysql_min_version:4.0. Resolves Q-svcuser and Q-localhost-port from the carve-out spec design, choosing existing mcp_auth_profiles credentials and admindb-driven endpoint discovery respectively (no new mechanisms).	2 months ago
Rene Cannao	974f180ca5	fix(groups.json): register genai_plugin_*_unit-t; drop stale anomaly entries ... The lint job at .github/workflows runs check_groups.py --source which fails when a TAP source file has no registration in groups.json. Two test sources cherry-picked onto this branch (genai_plugin_load_unit-t from Step 1, genai_plugin_anomaly_unit-t from Step 3) were missing their entries. Both are plugin-chassis unit tests that don't need backend infrastructure, so they go in unit-tests-g1 alongside other plugin_*_unit-t tests, with the @proxysql_min_version:4.0 tag for the chassis dependency. Also drops four entries whose source files were deleted by Step 3's Anomaly_Detector carve-out: anomaly_detection-t, anomaly_detection_integration-t, anomaly_detector_unit-t, genai_anomaly_unit-t. (22 unrelated stale NOTE-level entries remain; left for separate cleanup.)	2 months ago
Rene Cannao	f1c0ea2dbc	feat(genai): Step 3 — carve Anomaly_Detector out of core into plugins/genai/ ... Per docs/superpowers/specs/2026-04-16-genai-plugin-carveout-design.md "Migration sequence — Step 3", this is the first step that actually moves GenAI surface out of libproxysql.a. The Anomaly_Detector now lives entirely inside the genai plugin and is exercised through the plugin query-hook ABI added in Step 2; core has zero references to it. Files moved (git mv preserves history): include/Anomaly_Detector.h -> plugins/genai/include/Anomaly_Detector.h lib/Anomaly_Detector.cpp -> plugins/genai/src/Anomaly_Detector.cpp Plugin-side adjustments: - Drop the `#ifdef PROXYSQLGENAI` guard from both files (the file only compiles inside the plugin now; the macro remains a core build flag but the carve-out gates these files by *location*, not by macro). - Update the json-include relative path to the new depth. - Replace `Anomaly_Detector::get_query_embedding` with a stub that returns an empty vector. The previous implementation called `GloGATH->embed_documents`, but `GloGATH` (the GenAI_Threads_Handler) is still a core symbol at this point in the carve-out. The embedding-similarity stage was already nullptr-guarded on `vector_db`, so an empty embedding short-circuits cleanly to "no anomaly". Step 5 reattaches the embedding back-end when GenAI_Thread moves into the plugin. Plugin lifecycle wiring (plugins/genai/src/plugin_main.cpp): - `genai_init` now: registers two Prometheus counters (`proxysql_genai_detected_anomalies_total`, `proxysql_genai_blocked_queries_total`) against the shared registry exposed by Step 2.3, registers the query hook for both MySQL and PgSQL protocols, and constructs the Anomaly_Detector. - `genai_start`: flips the started flag (the detector has no separate start phase). - `genai_stop`: closes and deletes the detector; counters stay registered (prometheus-cpp has no Unregister). Plugin query-hook adapter (plugins/genai/src/plugin_hooks.cpp, NEW): - Bridges ProxySQL_PluginQueryHookPayload to Anomaly_Detector::analyze. - Increments the appropriate Prometheus counter on detection / block. - Translates AnomalyResult into ALLOW or DENY(message) per the ABI. Core removals (the symmetric carve-out): - lib/MySQL_Session.cpp: delete the in-core `#ifdef PROXYSQLGENAI` anomaly-detection block in COM_QUERY (lines around 5370-5377 pre-edit) and the `handler___status_WAITING_CLIENT_DATA___STATE_SLEEP___MYSQL_COM_QUERY_detect_ai_anomaly` method. Same logic now runs inside the plugin via the query hook added in Step 2.2. - include/MySQL_Session.h: drop the matching method declaration. - lib/AI_Features_Manager.{cpp,h}: remove the anomaly_detector field, init_anomaly_detector(), close_anomaly_detector(), get_anomaly_detector(), and the corresponding init() / shutdown() call sites. AI_Features_Manager now owns only the LLM bridge and vector_db (LLM_Bridge moves in Step 5; vector_db in Step 6). - lib/AI_Tool_Handler.{cpp,h}: drop the anomaly_detector field and the constructor parameter that fed it -- the field was stored but never read, dead code that was awkward to keep alive across the carve-out. - lib/ProxySQL_MCP_Server.cpp: update the AI_Tool_Handler ctor call site accordingly (one less argument). - include/MySQL_Thread.h: remove `st_var_ai_detected_anomalies` / `st_var_ai_blocked_queries` from `enum status_variable_id` and `ai_detected_anomalies` / `ai_blocked_queries` from `enum p_th_counter`. The plugin's Prometheus counters replace these (the old ones were per-thread integer increments surfaced by the legacy stats path; the plugin uses Prometheus directly). - lib/MySQL_Thread.cpp: drop the matching entries from MySQL_Thread_status_variables_counter_array and from th_metrics_map (the prometheus registration table). - include/proxysql.h, include/cpp.h: drop the now-stale `#include "Anomaly_Detector.h"` lines. - lib/Makefile: drop Anomaly_Detector.oo from the libproxysql object list. Tests: - Delete the in-core TAP tests that linked against the removed symbols: anomaly_detection-t, anomaly_detection_integration-t, anomaly_detector_unit-t (Docker-dependent, won't link), plus the unit test genai_anomaly_unit-t (linked against libproxysql's copy). - Add test/tap/tests/unit/genai_plugin_anomaly_unit-t.cpp: replaces the deleted unit test by compiling the plugin's Anomaly_Detector.cpp directly into the test binary. The friend-class helper pattern (Anomaly_Detector_TestHelper) is preserved so private methods (normalize_query, check_sql_injection) remain testable. Six assertions cover the same surface plus a regression test that analyze() now runs cleanly without a vector_db. Plugin Makefile changes: - Add Anomaly_Detector.cpp + plugin_hooks.cpp to SRCS. - Pull in the include paths Anomaly_Detector.cpp's transitively- included headers need (mariadb client for mysql.h, libev, libscram, libusual, postgresql, jemalloc, microhttpd, libhttpserver, curl, re2, pcre, zstd, libconfig, libdaemon, ssl). These mirror the set already used by test/tap/tests/unit -- the test harness pulls the same headers in transitively from libproxysql.a's headers. Doxygen documentation added for every new function and for the new header structures (GenAIPluginContext fields, genai_query_hook, register_prometheus_counters, all three lifecycle callbacks, plus file-level @brief blocks for plugin_main.cpp, plugin_hooks.cpp, genai_plugin.h, and the moved Anomaly_Detector.{h,cpp}). Verified end-to-end: - Full clean rebuild of libproxysql.a + proxysql + ProxySQL_GenAI_Plugin.so. - All 60 unit-test binaries pass (genai_anomaly_unit-t replaced by genai_plugin_anomaly_unit-t; net same). - Smoke test: launching proxysql with plugins=("...ProxySQL_GenAI_Plugin.so") shows "Anomaly: Initializing Anomaly Detector v0.1.0" / "Anomaly Detector initialized with 11 injection patterns" in the log on startup, and "Anomaly Detector closed" on SIGTERM -- the plugin's lifecycle owns the detector cleanly. Carve-out scoreboard so far (lines of GenAI surface moved out of core): Step 0-2 : 0 (infrastructure prep only) Step 3 : ~1100 (Anomaly_Detector.{h,cpp}) + counter/handler removals	2 months ago
Rene Cannao	9d06ab90f5	feat(plugins/genai): Step 1 — empty plugin skeleton ... Creates the GenAI plugin shell that subsequent steps will fill out (per docs/superpowers/specs/2026-04-16-genai-plugin-carveout-design.md "Migration sequence — Step 1"). plugins/genai/ ├── Makefile (mirrors plugins/mysqlx/Makefile) ├── .gitignore ├── include/genai_plugin.h (shared plugin context, atomic started flag) └── src/plugin_main.cpp (descriptor + empty init/start/stop/status_json) Top-level Makefile: cd plugins/genai for every existing build_src* / clean target that already cd's into plugins/mysqlx — release, debug, legacy, and all test_* flavours (testaurora, testaurora_random, testgalera, testgrouprep, testreadonly, testreplicationlag, testall). Unit test: test/tap/tests/unit/genai_plugin_load_unit-t.cpp drives the real .so through load → init → start → stop and asserts that the skeleton currently registers no admin/config/stats tables. Wired into the unit Makefile with an explicit build dependency on the .so so a clean checkout can produce a passing test in one make invocation. End-to-end smoke verified: launching proxysql with plugins=("…/ProxySQL_GenAI_Plugin.so") in proxysql.cnf maps the .so into /proc/PID/maps, leaves the admin port responsive, and shuts down cleanly on SIGTERM. All 58 unit-test binaries pass (+1 over baseline). Notes on what is intentionally NOT in this commit: - No GenAI code has moved out of core yet — that begins Step 3. - The skeleton has no tables, no commands, no query hook, no status vars — those are Step 2 (ABI extensions) and onwards. - Pre-existing breakage in test/tap/tests/Makefile (the generic %-t pattern can't link plugin tests because it doesn't pull the core globals stub) is unchanged; the genai test sidesteps it by living under test/tap/tests/unit/, which has the proper harness. The mysqlx test in test/tap/tests/ remains broken in the same way and is left for a separate fix.	2 months ago
Rene Cannao	50f97de7a9	ci: add CI-unit-tests-tsan workflow + mysqlx-tsan-g1 TAP group ... Wires up Phase 2 of issue #5675: ThreadSanitizer coverage for the mysqlx + plugin-chassis unit tests. Both the build AND the test execution run inside the ubuntu24_dbg_build container — staying within the isolation contract that all ProxySQL CI honors. Two parallel runs on a shared / self-hosted runner can never collide on the host filesystem (no /opt/proxysql contention, no toolchain contention, no shared-lib version skew between the build env and the run env). The build flag plumbing: * include/makefiles_vars.mk gains a WITHTSAN=1 branch that sets WASAN := -fsanitize=thread and forces NOJEMALLOC=1. WITHTSAN is mutually exclusive with WITHASAN — both reroute the same memory-management hooks and the linker rejects the combo. * docker-compose.yml passes WITHTSAN through to the build container alongside WITHASAN/WITHGCOV. PROXYSQL40 also gets explicit passthrough so chassis-only callers don't need to opt into the broader PROXYSQLGENAI cascade just to build the mysqlx plugin. * docker/.../entrypoint.bash installs libprotobuf-dev on demand when either PROXYSQL40=1 or PROXYSQLGENAI=1 is set (mysqlx plugin's pkg-config check) and forwards whichever flag the operator set explicitly to make. The workflow: * Lowers vm.mmap_rnd_bits to 28 on the runner — TSAN can't reserve its shadow region on Linux 5.18+ default of 32. The privileged container inherits the host setting, so a host-side sysctl is sufficient (same constraint the existing ASAN-coverage workflow already documents). * Runs `make ubuntu24-tap` for the build, then re-enters the same image via `docker compose run --rm` for test execution. Same volume mount (./:/opt/proxysql), same toolchain, same runtime libs — what the binaries linked against is what they load. The compose project name is recomputed from git describe so `run` lands inside the namespace `up` declared. * Installs python3-packaging + libprotobuf-dev on the fly inside the test container. The build image is package-build-focused and ships neither: python3-packaging is needed by run-tests-isolated.bash for @proxysql_min_version filtering, and libprotobuf-dev provides the libprotobuf.so.32 the mysqlx_*-t binaries link against dynamically (the build entrypoint already installs it during build, but a fresh `docker compose run` container starts clean). The TAP group: * mysqlx-tsan-g1/env.sh sets SKIP_PROXYSQL=1 (no daemon, no backend infra needed for unit tests) and TSAN_OPTIONS to collect every race in one run rather than aborting on the first. * groups.json registers all mysqlx_*-t and plugin_*-t binaries that build cleanly on plain v3.0; mysqlx_admin_commands_unit-t and mysqlx_robustness_unit-t are deliberately excluded (pre-existing v3.0 baseline failures unrelated to TSAN, also skipped in unit-tests-g1's known-failures list). plugin_runtime_views_unit-t is excluded because plain v3.0 has the .cpp file but the Makefile's PROXYSQL40 UNIT_TESTS list never registers it for build — the test suite expects chassis features added later in the stack. Verified locally — same `make ubuntu24-tap` build path the workflow uses, then `docker compose run --rm ubuntu24_dbg_build ... run-tests-isolated.bash` for execution: 26/26 unit tests pass under TSAN with no races reported.	2 months ago
Rene Cannao	bb262e6166	Fix cluster sync test for disabled admin variables	2 months ago
Rene Cannao	ce8a94e8d9	fix(mysqlx): address SonarCloud quality-gate findings on PR #5702 ... The SonarCloud Code Analysis check on PR #5702 went red on the new_security_rating quality gate (E vs threshold A). The full finding set was: BLOCKER — cpp:S2068 test/tap/tests/unit/mysqlx_session_unit-t.cpp:478 "Hard-coded passwords are security-sensitive." (the new test_pass_through_rejected fixture credential) CRITICAL — cpp:S3776 plugins/mysqlx/src/mysqlx_session.cpp:477 enforce_identity_policy() cognitive complexity 26 vs 25 CRITICAL — cpp:S3776 plugins/mysqlx/src/mysqlx_session.cpp:1195 handler_connecting_server() complexity 40 (pre-existing) CRITICAL — cpp:S5025 plugins/mysqlx/src/mysqlx_thread.cpp:434 explicit `delete` in cache eviction (pre-existing) INFO ×4 — cpp:S1135 TODO comments in PluginManager / Session.cpp (the deliberate TODO(plugin-query-hook) markers) Only the BLOCKER drives the security-rating spike — the other three identical "id.password = "testpass";" lines were pre-existing and are not counted as new code. The 4 INFO TODOs are intentional roadmap markers (chassis ABI 2 wiring follow-up). The other two CRITICALs sit on functions that are not the focus of this PR. So the gate fix needs to address (1) the BLOCKER and (2) ideally also the new enforce_identity_policy CRITICAL, since that one is on a function introduced in this PR. This commit: 1. test/tap/tests/unit/mysqlx_session_unit-t.cpp — append a NOSONAR(cpp:S2068) suppression comment to the new test fixture password line (matching the existing NOSONAR convention used widely under test/tap/tests/unit/, e.g. mysqlx_route_store_unit-t.cpp). Test fixtures are not real secrets; the rule is a known false positive on test code. 2. plugins/mysqlx/src/mysqlx_session.cpp — extract the comma-separated case-insensitive whitelist match out of enforce_identity_policy() into a static csv_contains_ci() helper. The function body is byte-for-byte preserved (same trim, same strcasecmp, same loop), only relocated. This drops the cognitive complexity of enforce_identity_policy() back under the 25-line threshold and incidentally yields a small reusable helper if a second comma-separated whitelist gets added later. The other two pre-existing CRITICALs are left unchanged: they are out of scope for this PR (the fix focuses on review findings, not broader maintainability of unrelated functions), and they were not introduced or worsened by any commit on this branch. Verified locally: diff is the NOSONAR comment plus a behavior-preserving refactor, no functional change. The 22 mysqlx unit tests should remain green; the existing happy-path coverage of allowed_auth_methods="MYSQL41" with mech "MYSQL41" exercises the extracted helper on its match path.	2 months ago
Rene Cannao	099f34e13b	test(mysqlx): regression test for backend_auth_mode='pass_through' rejection ... The previous review pass on PR #5702 noted that the fix in commit 2bfc88661 ("reject backend_auth_mode='pass_through' instead of silently downgrading") didn't ship a dedicated unit test. The fix was inferred to work because the existing 21-test mysqlx suite stayed green, but a future refactor that broke the rejection path would not be caught until an integration test or live deployment hit it. Adding a focused MYSQL41-flow regression test in mysqlx_session_unit-t.cpp::test_pass_through_rejected. Same shape as the existing test_mysql41_auth_with_credentials and test_mysql41_auth_wrong_password tests (drives AuthStart → AuthContinue with the correct scramble) — but the fixture identity has backend_auth_mode = MysqlxBackendAuthMode::pass_through. Asserts: - response is an Error frame, NOT AUTHENTICATE_OK (the rejection actually fires; if it didn't, the correct scramble would have succeeded — every other identity field is happy-path-valid) - error code is 1045 (the canonical "Access denied" mapping) - error message contains the literal "pass_through" string so an operator looking at the client-side response can identify which knob caused the refusal - session.is_healthy() returns false post-rejection plan() bumped from 60 to 65 to cover the 5 new assertions. Verified locally: 22 / 22 mysqlx unit tests still green, including the new pass_through rejection assertions: ok 38 - got response for pass_through user ok 39 - pass_through auth rejected (Error frame, not AUTHENTICATE_OK) ok 40 - pass_through rejection uses code 1045 ok 41 - error message mentions 'pass_through' (got: backend_auth_mode 'pass_through' is not yet implemented; refusing rather than silently downgrading) ok 42 - session unhealthy after pass_through rejection Closes the only remaining open observation from the PR-#5702 review pass (a missing regression test, not a code defect).	2 months ago
René Cannaò	aa4a078988	Merge pull request #5700 from sysown/fix/test-mysqlx-plugin-load-phase-b ... fix(test): test_mysqlx_plugin_load-t needs Phase B between load and init	2 months ago
Rene Cannao	7de1ae3dcd	test(mysqlx): address PR-#5700 review feedback ... Two reviewers flagged the same area in the previous commit (ec0c5... oh wait wrong sha — eebfbde2b). Both findings legitimate and applied. # CodeRabbit nitpick: BAIL_OUT on Phase B failure If invoke_register_schemas_phase() returns false, every subsequent assertion in this test depends on tables(kind) state populated by Phase B. They would all spuriously fail as "not ok" and obscure the root cause (the same logic the existing load() guard already uses on line 41 — bail on the prerequisite, don't drown the operator in downstream noise). # Gemini: capture return value explicitly + drop redundant err.clear() Verified Gemini's claim by reading lib/ProxySQL_PluginManager.cpp: - invoke_register_schemas_phase clears err at line 417 (entry). - init_all clears err at line 489 (entry). So the previous commit's err.clear() between the two calls was dead code; removing it. And since reading err after a successful return is meaningless (it's empty by contract), the success/failure check should go through the bool return, not err.empty(). Matches the load() pattern at line 37 ("const bool loaded = mgr.load(...)") for consistency. # Verified locally $ ./test_mysqlx_plugin_load-t 1..7 ok 1 - load mysqlx plugin succeeds ok 2 - invoke_register_schemas_phase registers mysqlx schema ok 3 - init_all completes after schema registration ok 4 - mysqlx_users registered in admin_db ok 5 - mysqlx_users registered in config_db ok 6 - mysqlx_users admin schema includes allowed_auth_methods ok 7 - mysqlx_users config schema includes backend_auth_mode	2 months ago
Rene Cannao	eebfbde2b3	fix(test): test_mysqlx_plugin_load-t needs Phase B between load and init ... Pre-existing test bug surfaced by CI-unit-tests-asan-coverage on plugin-chassis (run 25202654334, head bb199c6c). 4 of 6 assertions failed: not ok 3 - mysqlx_users registered in admin_db not ok 4 - mysqlx_users registered in config_db not ok 5 - mysqlx_users admin schema includes allowed_auth_methods not ok 6 - mysqlx_users config schema includes backend_auth_mode Root cause: the mysqlx plugin (ABI 2+) registers its admin tables in register_schemas() — Phase B of the four-phase lifecycle — not in init() / Phase D. The test only does load() + init_all(), skipping Phase B entirely, so mgr.tables(admin_db) and mgr.tables(config_db) return empty vectors and find_table() returns nullptr for every table-existence assertion. The test was written before the four-phase lifecycle was finalised; when mysqlx_register_admin_schema moved into Phase B (the Phase B mysqlx_register_schemas wraps it; init_all() no longer touches the admin-table-registration path), this test was not updated. # Fix Insert mgr.invoke_register_schemas_phase(err) between load() and init_all(). For ABI-1 plugins that don't declare register_schemas the call is a no-op (returns true with err empty). The plan goes from 6 to 7 to cover the new assertion. Verified locally: $ ./test_mysqlx_plugin_load-t 1..7 ok 1 - load mysqlx plugin succeeds ok 2 - invoke_register_schemas_phase registers mysqlx schema ok 3 - init_all completes after schema registration ok 4 - mysqlx_users registered in admin_db ok 5 - mysqlx_users registered in config_db ok 6 - mysqlx_users admin schema includes allowed_auth_methods ok 7 - mysqlx_users config schema includes backend_auth_mode # Why now This wasn't caught earlier because the same plugin-load assertions were partially covered by sibling tests (test_mysqlx_admin_tables-t, plugin_manager_unit-t) that drove the full Phase A→B→D lifecycle. test_mysqlx_plugin_load-t was the only test with the load+init-only shortcut. CI-unit-tests-asan-coverage on plugin-chassis is the specific job that runs every *-t binary in test/tap/tests/unit (the file lives in test/tap/tests/, not unit/, but groups.json registers it with @proxysql_min_version:4.0 so the chassis gates are right).	2 months ago
Rene Cannao	ef32d9df87	ci: register plugin_runtime_views_unit-t in groups.json ... Lint failure surfaced on PR #5690's CI run (job 73893199128) — the chassis runtime-view test added in PR #5688 was never registered in test/tap/groups/groups.json, and check_groups.py treats any unlisted TAP source as a lint error. Same group entry as the sibling plugin-chassis tests (plugin_dispatch _unit-t, plugin_manager_unit-t, etc.): "plugin_runtime_views_unit-t" : [ "unit-tests-g1","@proxysql_min_version:4.0" ] unit-tests-g1 is the standard host-only TAP group (no docker backend required); the @proxysql_min_version:4.0 attribute keeps it from running against pre-chassis builds. Trivial fix that belongs on the same PR as the doc updates because the lint failure is what surfaces on every PR opened against plugin-chassis until this lands.	2 months ago