26 Commits (feature/pgsql-native-backend-protocol)

Author	SHA1	Message	Date
René Cannaò	08cc2dbaa5	Merge commit from fork ... fix(security): MCP run_sql_readonly multi-statement bypass (CVE-2026-48774 / GHSA-7wh6-2vcc-gcm4)	1 month ago
Rene Cannao	e32b7fd50c	fix(security): harden MySQL_Tool_Handler parallel class (GHSA-7wh6-2vcc-gcm4) ... MySQL_Tool_Handler is a sibling of Query_Tool_Handler with the same shape: it opens a backend MySQL connection pool and exposes run_sql_readonly() and explain_sql() methods. It was not touched by the original GHSA-7wh6-2vcc-gcm4 fix because no MCP endpoint registration currently dispatches through it (mysql_tool_handler is declared in MCP_Thread but never assigned), but the class is allocated and destructed by MCP_Thread and referenced by MySQL_FTS / catalog code. Any future wire-up — MCP endpoint, new tool, accidental rebind — would reintroduce CVE-2026-48774 verbatim, because: * the connection pool was opened with CLIENT_MULTI_STATEMENTS; * validate_readonly_query had no contains_multi_statement check; * is_dangerous_query missed RENAME, FLUSH, RESET, LOCK/UNLOCK, KILL, OPTIMIZE, REPAIR, HANDLER, INSTALL/UNINSTALL, CHANGE, PURGE, BEGIN/START/COMMIT/ROLLBACK, XA, SHUTDOWN, INTO DUMPFILE, ANALYZE; * explain_sql performed ZERO validation before prepending "EXPLAIN ". Mirror the Query_Tool_Handler hardening here: * Drop CLIENT_MULTI_STATEMENTS in init_connection_pool(). * Expand the dangerous-keyword list to match the sibling class, plus ANALYZE so that obfuscated EXPLAIN ANALYZE forms ("EXPLAIN/**/ ANALYZE", "EXPLAIN (ANALYZE)", etc.) and standalone "ANALYZE TABLE x" are caught. * Add a contains_multi_statement() lexer that ignores ';' inside string literals, identifier-quoted names, and comments; call it at the start of validate_readonly_query(). * Add the same read-only validation to explain_sql(). The contains_multi_statement implementation is duplicated from Query_Tool_Handler intentionally for the short-term security fix; a follow-up should deduplicate into a shared utility so the two validators cannot drift.	1 month ago
Rene Cannao	9d8c6b77b3	fix(security): close MCP run_sql_readonly multi-statement bypass (GHSA-7wh6-2vcc-gcm4) ... The MCP /mcp/query endpoint advertised a read-only contract for the run_sql_readonly tool, but enforcement was a substring-only validator on the first keyword while the backend MySQL connection was opened with CLIENT_MULTI_STATEMENTS. A payload like SELECT 1; RENAME TABLE testdb.x TO testdb.y passed validation (starts with SELECT) and executed both statements, renaming the backend table. The pre-fix substring blacklist also omitted several side-effecting MySQL keywords (RENAME, SET, RESET, LOCK, UNLOCK, KILL, FLUSH, OPTIMIZE, REPAIR, HANDLER, ...) so the bypass also worked as a single statement when those keywords appeared at the start. Fixes applied in defense-in-depth order: 1. Drop CLIENT_MULTI_STATEMENTS from the MCP backend mysql_real_connect calls in Query_Tool_Handler.cpp:init_connection_pool and backend_client.cpp:dial_mysql. This alone closes the demonstrated PoC end-to-end: the server-side parser will reject the trailing statement as a syntax error. 2. Add Query_Tool_Handler::contains_multi_statement() which tracks lexical state (single/double-quoted strings, backtick-quoted identifiers, '--' line comments, '/* */' block comments) and rejects any ';' followed by non-whitespace. Call it at the top of validate_readonly_query so multi-statement payloads are rejected before they ever reach the backend. 3. Expand the dangerous-keyword denylist to cover RENAME, RESET, LOCK, UNLOCK, KILL, FLUSH, GRANT, REVOKE, OPTIMIZE, REPAIR, HANDLER, INSTALL/UNINSTALL, CHANGE, PURGE, SAVEPOINT/RELEASE, transaction control (BEGIN, START, COMMIT, ROLLBACK, XA), SHUTDOWN, INTO OUTFILE/DUMPFILE. Reject 'SET ' and 'USE ' as standalone leading keywords (they appear legitimately as substrings inside SELECT, but never as the first token of a read-only query). 4. Reject 'EXPLAIN ANALYZE' explicitly: PostgreSQL EXPLAIN ANALYZE actually executes the wrapped statement, and the same applies to some MySQL configurations. 5. Apply the same validation in the explain_sql handler, which previously did no read-only validation at all. An MCP caller could send "SELECT 1; RENAME TABLE ..." through explain_sql, the handler would prepend "EXPLAIN " and execute the multi-statement payload. Operator note: this only closes the bypass. /mcp/query authentication defaults to mcp-query_endpoint_auth="" (no auth) when MCP is enabled, which makes any deployment exposing /mcp/query reachable to an unauthenticated network attacker. Hardening that default is tracked separately.	2 months ago
René Cannaò	0c798fabbb	Add config query tool	2 months ago
Rene Cannao	1fe9c3a805	feat(genai): implement LLM Bridge semantic cache with vector similarity (Feature A) ... Complete the 4 TODO stubs in LLM_Bridge to make semantic caching functional, reducing LLM API costs for repeated similar queries. Config changes: - Add cache_enabled bool to LLM_Bridge config struct (default: true) - Update update_config() signature to accept cache_enabled parameter - Wire genai_llm_cache_enabled through AI_Features_Manager::init_llm_bridge() check_cache(): - Generate embedding for the prompt via GloGATH->embed_documents() - KNN search against llm_cache_vec (sqlite-vec) joined with llm_cache - Compute cosine similarity = 1 - distance; compare against threshold - On hit: update hit_count/last_hit, increment GloAI cache_hits counter - On miss: increment GloAI cache_misses counter - Track lookup timing via GloAI->add_llm_cache_lookup_time_ms() store_in_cache(): - Generate embedding, INSERT into llm_cache, INSERT into llm_cache_vec - Track store timing via GloAI->add_llm_cache_store_time_ms() clear_cache(): - DELETE FROM llm_cache_vec, DELETE FROM llm_cache get_cache_stats(): - SELECT COUNT(*), SUM(hit_count) FROM llm_cache - Read misses/lookups/stores from AI_Features_Manager::collect_status_variables() - Return JSON: {entries, hits, misses, lookups, stores}	2 months ago
Rene Cannao	b4ff986f10	feat(genai): add collect_status_variables() to genai/mcp/ai handlers (Feature C) ... Expose internal counters from all three handler classes so operators can monitor plugin health from the admin interface. Each handler gets a collect_status_variables() method returning std::vector<std::pair<std::string, std::string>>: - GenAI_Threads_Handler: 4 counters (threads_initialized, active_requests, completed_requests, failed_requests) - MCP_Threads_Handler: 3 counters (total_requests, failed_requests, active_connections) - AI_Features_Manager: 14 counters (llm_total_requests, llm_cache_hits/misses/lookups/stores, llm_local/cloud_model_calls, llm_total_response_time_ms, llm_cache_total_lookup/store_time_ms, anomaly_total_checks/blocked_queries/flagged_queries, daily_cloud_spend_usd) Total: 21 status variables queryable via stats_genai_global table.	2 months ago
Rene Cannao	dca155aa97	feat(genai): persist MCP query digest stats to SQLite (Feature E) ... Add restart resilience for MCP query digest statistics by persisting the in-memory digest map to the catalog's own SQLite database. New methods on Discovery_Schema: - create_digest_persist_table(): creates mcp_query_digest_persist table in the catalog DB (idempotent, called during init_schema()) - flush_digest_to_sqlite(): serialises the in-memory digest map to SQLite under BEGIN IMMEDIATE / COMMIT, called every 100 updates via update_mcp_query_digest() - load_persisted_digests(): restores persisted digest stats from SQLite back into the in-memory map, called during init_schema() Reset handling: get_mcp_query_digest(reset=true) now also clears the persist table so reset stats don't reappear after restart. Thread safety: flush uses rdlock (read-only traversal), load uses wrlock (populates the map). Reset uses the existing wrlock path.	2 months ago
Rene Cannao	a9f2eaaa13	feat: remove PROXYSQLGENAI flag, genai now builds under PROXYSQL40 (issue #5722) ... PROXYSQLGENAI flag eliminated entirely. PROXYSQL40=1 now unconditionally builds the genai plugin alongside core. All #ifdef PROXYSQLGENAI replaced with #ifdef PROXYSQL40 in source, Makefiles, and 61 CI workflows.	2 months ago
Rene Cannao	b8cfa32c63	fix: address all critical and important code review findings (issue #5729) ... Critical fixes: - Move db_kind to end of ProxySQL_PluginRuntimeView (tail-append = free ABI-3 compat) - Add mutex to protect Anomaly_Detector::user_statistics from concurrent session threads - Replace system(mkdir -p) with pure mkdir() loop (command injection fix) - Replace 77 std::stoi/stoll with non-throwing parse_int_or_zero/parse_ll_or_zero - Fix proxysql.cnf filename: Mysqlx -> MySQLX - Add --whole-archive to non-ClickHouse link path (plugin symbol resolution) Important fixes: - Add default: case with warning to db_kind switch in ProxySQL_PluginManager.cpp - Add config_db dispatch and null-handle tests to plugin_runtime_views_unit-t (6 new tests) - Update CLAUDE.md with sqlite-vec PROXYSQLGENAI exception - Remove stale anomaly_detection-t Makefile target - Replace fixed 1024-byte buffer with dynamic sqlite3_mprintf in Anomaly_Detector Minor: update ABI.md, PLUGIN_API.md, stale comments, unit test plan counts	2 months ago
Rene Cannao	29402b4a42	docs(plugins/genai): close 3 small gaps from review ... # 1. Real prepared statement in local_proxy_endpoint.cpp read_global_variable's comment claimed "Parameterised lookup via prepared statement" but the implementation did string concatenation: "SELECT variable_value FROM main.global_variables WHERE variable_name='" + std::string(var_name) + "'" Today's only callers pass hardcoded "mysql-interfaces" / "pgsql-interfaces", so the SQL injection surface is zero — but the comment is a footgun for the next caller and we have a working prepare_v2 / bind_text pattern right next door in plugin_main.cpp. Replace with the proper prepared-statement form using proxy_sqlite3_step + proxy_sqlite3_column_text; semantics (empty-string on miss / error) are unchanged. # 2. Concurrency contract on genai_refresh_runtime_components The function (introduced in 9e4244321) tears down and reinitialises GloGATH and GloAI in place — with no rwlock around either global. Today it's called from: 1. genai_start (single-threaded, no traffic) — always safe 2. LOAD GENAI VARIABLES TO RUNTIME admin command — racy with active MCP listener / tool-handler threads 3. LOAD GENAI VARIABLES FROM CONFIG — same as (2) Document the contract honestly in the header: caller must guarantee no concurrent dereference of GloGATH/GloAI; today the practical mitigation is operator-side (quiesce traffic before issuing the LOAD). A proper rwlock around all GloGATH/GloAI consumers is a follow-up — too large for this PR's scope, requires audit of every listener/tool-handler reader. # 3. New plugins/genai/README.md The plugin is now ~30 source files spanning the MCP listener, GenAI worker pool, AI features manager, anomaly detector, schema discovery cache, FTS index, vector storage, LLM bridge, and 9 tool handlers — with no single doc explaining what it owns or how to operate it. Mirror the plugins/mysqlx pattern: a one-page README covering scope, building, loading, lifecycle, admin SQL surface, configuration pointer, observability, concurrency contract, testing, and known gaps. Cross-references doc/MCP/ and doc/LLM_Bridge/ for the deeper subsystem-internal documentation that already exists.	2 months ago
Rene Cannao	967ab401ec	genai: fix lifecycle reloads and drop dead MCP stats registration ... Load GenAI and MCP variables before runtime initialization, reinitialize the AI stack on reload, and make the anomaly hook obey the configured enable flags. Restore the missing MCP command aliases and the disk/config reload verbs, fix the MySQLX packaging filename mismatch, and update the unit harness for the plugin ABI. Also remove the stale stats_mcp_* registrations from the plugin surface. There is no plugin-side writer for those tables in this branch, so advertising them as live schema was misleading. Update the unit test and bootstrap comments to match the actual exposed surface.	2 months ago
Rene Cannao	978a6f8d2f	fix(plugins/genai): blocker fixes from PR-#5701 review ... Five concrete fixes uncovered by the four-agent review (architecture, code-safety, build/packaging, test coverage): # Blocker 1: per-row INSERT failure leaves transaction half-done In the six MCP_Threads_Handler methods that follow the BEGIN -> DELETE -> prepare -> loop INSERT -> COMMIT pattern, the SAFE_SQLITE3_STEP2 macro inside the loop didn't check `rc`. If a single INSERT failed (constraint violation, disk full, ...), the loop would continue binding subsequent rows and the COMMIT below would either succeed with partial data or be left dangling. Hoist the per-row binding into three insert_*_row(stmt, row) -> bool helpers in an anonymous namespace, then have every caller check the return value and `ROLLBACK; return false/return;` on failure. Affects save_auth_profiles_to_admin_table, save_target_profiles_to _admin_table, save_query_rules_to_admin_table, project_auth_profiles _to_runtime_view, project_target_profiles_to_runtime_view, project_query_rules_to_runtime_view, plus the new save_profiles_to _admin_table below. # Blocker 2: cross-table install/save atomicity for profiles target.auth_profile_id is an FK to auth.auth_profile_id. The old mcp_load_target_auth_map_from_admindb / mcp_save_target_auth_map_to_ admindb wrappers ran the two installs / two saves as separate operations under separate locks / transactions. A failure of the second step (or a concurrent LOAD/SAVE racing in between) left target_auth_map rebuilt from a (auth_v2, target_v1) mismatch, or left main.mcp_target_profiles referencing auth_profile_ids that didn't exist in main.mcp_auth_profiles. Add atomic combined methods: - install_profiles_from_admin: reads both source tables BEFORE taking the wrlock (so a slow admindb doesn't gate the listener), then under one wrlock swaps both vectors and rebuilds the joined target_auth_map exactly once. - save_profiles_to_admin_table: snapshots both vectors under one rdlock, then deletes target before auth (FK direction-aware) and inserts auth before target inside a single BEGIN ... COMMIT. The plugin_main.cpp wrappers now route through these atomic methods. The per-table install_*_from_admin / save_*_to_admin_table public methods stay (used by future granular admin verbs and unit tests), but each rebuilds target_auth_map after its own swap as before. # Blocker 3: unit-test PROXYSQLGENAI autodetect probe test/tap/tests/unit/Makefile had been probing libproxysql.a for the `MCP_Tool_Handler` symbol to autodetect PROXYSQLGENAI. Step 7 of the carve-out moved that symbol to plugins/genai/, so the probe silently found 0 matches and stopped defining -DPROXYSQLGENAI for unit-test builds. Switch the probe to invoke_register_schemas_phase (the chassis-exclusive symbol that's already used to probe PROXYSQL40) so the macro auto-defines whenever the chassis is available — which matches when genai-plugin unit tests can actually be built and exercised. # Blocker 4: triplet coverage through dispatch surface Extend genai_plugin_load_unit-t to drive the full install/save/project flow through the production dispatch path: - seed mcp_<X> tables in admindb - dispatch "LOAD MCP PROFILES TO RUNTIME" via mgr.dispatch_admin_ command (same path admin SQL takes) - manually invoke refresh_runtime_views_for_query (the chassis hook that fires on every admin SELECT) and assert runtime_mcp_<X> rows match the source - mutate main.<X> directly to simulate an operator stomp, then dispatch "SAVE MCP PROFILES TO MEMORY" and assert the in-memory snapshot was written back over the stomp This is the end-to-end coverage that plugin_runtime_views_unit-t can't provide (it uses synthetic callbacks) and that a standalone genai_mcp_config_store unit test would have provided at the cost of stubbing 9 plugin classes. Plan bumped from 25 -> 36. # Should-fix: atomic fence + GloGATH lifetime genai_anomaly_embed_fn was std::atomic with relaxed ordering; the comment claimed nothing else needed synchronising. Wrong: the embed function dereferences GloGATH, a non-atomic global mutated by genai_init / genai_stop. Pair the load with acquire and the stores with release so a reader that observes the non-null pointer also observes the prior `GloGATH = new ...`, and a reader that observes the post-stop nullptr short-circuits before GloGATH's deletion is visible. Acquire/release is free on x86_64 / arm64 — no perf regression. # Should-fix: harden DEB awk substitution + RPM glob DEB entrypoint: replace the inline awk -v repl="$..." substitution (which mangled paths containing awk metacharacters via gsub) with a file-backed awk getline that reads the replacement block verbatim. Add a fail-fast grep for the placeholder after substitution so a silently-broken substitution can't ship a malformed package. RPM (rhel + suse): the spec gates `/usr/lib/proxysql/*.so` on %if 0%{?with_plugins}, but rpmbuild aborts on "File not found by glob" if the directory is empty (silent plugin build failure + with_plugins=1). Have the entrypoint compgen-check whether any .so actually got staged and only pass --define "with_plugins 1" when true.	2 months ago
Rene Cannao	482ce386cd	chore(genai-carveout): strip dead PROXYSQLGENAI blocks + plugin hardening ... Two threads of cleanup that had to land together because they touch both core (where the dead code lives) and the plugin (where the new module-owned state lives): # Strip dead #ifdef PROXYSQLGENAI from lib/, include/, src/ (Spec Q9) Step 7 of the carve-out left PSQLGA empty in lib/Makefile, so every #ifdef PROXYSQLGENAI block in the core build became permanently false. The blocks remained as dead text (visible to readers, irrelevant to the binary). Strip ~600 lines spanning: src/main.cpp 2 sites lib/Admin_Bootstrap.cpp 3 sites (mcpdb open + attach + flush_*_variables hooks) lib/Admin_FlushVariables.cpp 2 sites (~250 lines of stub + #if 0 _ORIGINAL bodies) lib/Admin_Handler.cpp 5 sites (LOAD/SAVE GENAI VARIABLES dispatch + LOAD/SAVE MCP VARIABLES dispatch + MCP profiles handler) lib/ProxySQL_Admin.cpp ~10 sites (init_*_variables + load_mcp_server + load/save_mcp_query_rules + GenericRefreshStatistics MCP probes) lib/ProxySQL_Admin_Stats.cpp 3 stats___mcp_* function bodies include/proxysql_admin.h 7 method declarations include/ProxySQL_Admin_Tables_Definitions.h 4 unused DDL macros The plugin now owns LOAD/SAVE MCP VARIABLES, LOAD/SAVE GENAI VARIABLES, LOAD/SAVE MCP PROFILES, LOAD/SAVE MCP QUERY RULES via its admin command registry; the chassis dispatcher catches them before any of the dead core branches would have fired. Caveat — known break: the `stats_mcp_*` tables are still registered (empty schema) but the populator path that fed them is gone. SELECTs return empty result sets until a follow-up wires the populator into the plugin (likely via register_runtime_view or a parallel stats-side ABI). Tracked separately. # Plugin hardening (review-feedback items 4-7) * plugins/genai/Makefile: add -fvisibility=hidden, -fvisibility-inlines-hidden, -fstack-protector-strong (mirrors plugins/mysqlx). Tag proxysql_plugin_descriptor_v1 with __attribute__((visibility("default"))) so the loader's dlsym still resolves; without the explicit annotation the hidden-default flag would also hide the entry point. * plugins/genai/src/Anomaly_Detector.cpp: convert genai_anomaly_embed_fn to std::atomic<…> with relaxed loads/stores. The detector hot-path read races with init/stop on the lifecycle thread; atomics give us a portable no-torn-pointer guarantee without paying for acquire/release fences we don't need (no other state synchronises with the pointer). Replaces the hand-wavy "no mutex needed" comment with the actual reasoning. * plugins/genai/src/{plugin_main,plugin_commands,plugin_tables}.cpp: route the 17 fprintf(stderr, "genai plugin: ...") log lines through a new genai_log() helper that calls services->log_message when the chassis is wired up and falls back to stderr for early-init / unit-test paths. Lines now land in proxysql.log alongside the rest of the runtime. * plugins/genai/include/MCP_Thread.h: add get_variable_string(name, std::string&) returning a std::string, and switch mcp_save_variables_to_admindb (plugin_main.cpp) to it. The legacy get_variable(name, char*) had an unbounded sprintf into a stack buffer and the SAVE path was passing char val[256] — a real overflow risk for *_endpoint_auth bearer tokens. * plugins/genai/src/plugin_main.cpp: rewrite the genai_init / genai_stop comment block with the actual teardown ordering rationale (consumers before producers; atomic-clear-then-delete for the embed hook; why Prometheus counters stay registered).	2 months ago
Rene Cannao	16253a1c99	feat(plugins/genai): adopt ABI-3 separation-of-duties for runtime_mcp_* ... Wire the genai plugin into the plugin-chassis ABI-3 contract added in PR #5688: Admin owns the editable mcp_<X> tables, the module owns the in-memory snapshot under its own mutex, and runtime_mcp_<X> is a chassis-projected view repopulated lazily by a refresh callback before each admin SELECT. No more triple-storage of profile / query-rule data. # MCP_Threads_Handler: per-table install / save / project triplets Replaces the monolithic load_target_auth_map(joined-resultset) with three independent triplets, mirroring the mysqlx config_store rework: install_<X>_from_admin reads main.mcp_<X>, replaces the in-memory snapshot vector under wrlock, rebuilds the joined target_auth_map (for profiles) save_<X>_to_admin_table REPLACEs main.mcp_<X> from the snapshot project_<X>_to_runtime_view DELETE+INSERT runtime_mcp_<X> from the snapshot — invoked only by the chassis runtime-view refresh dispatcher Two new vectors (auth_profiles_, target_profiles_, query_rules_) hold the authoritative per-table rows; target_auth_map remains as the joined view consumed by the listener and is rebuilt whenever either profile vector changes. # Plugin descriptor + table registration genai_register_admin_tables now also calls services->register_runtime_view for runtime_mcp_auth_profiles, runtime_mcp_target_profiles, and runtime_mcp_query_rules, pointing each at a project_*_to_runtime_view trampoline. The runtime_mcp_<X> tables stay registered for their CREATE TABLE schema only — their rows are entirely owned by the per-SELECT projection callbacks from this point on. # Admin SQL surface LOAD MCP PROFILES TO RUNTIME / LOAD MCP QUERY RULES TO RUNTIME now go through install_<X>_from_admin and never touch runtime_mcp_<X>. SAVE MCP PROFILES TO MEMORY is added (was missing pre-ABI-3) and joins SAVE MCP QUERY RULES TO MEMORY in pulling from the module snapshot instead of the runtime view. Query_Tool_Handler::refresh_target_registry no longer re-reads runtime_mcp_<X> via SQLite on every refresh; it consumes the in-memory snapshot directly through GloMCPH->get_all_target_auth_contexts(). # Drop stale GENAI:-prefix tests test/tap/tests/genai_{async,embedding_rerank,live_validation}-t.cpp were left over from before Step 4.A removed the GENAI: query-prefix escape hatch — they would all have failed under ai-g1. Delete them and their groups.json / ai/README.md references; equivalent coverage now flows through the MCP /mcp/ai endpoint and the genai_module-t suite that already drives the plugin surface.	2 months ago
Rene Cannao	7f148940e3	feat(plugins/genai): Step 8b — wire LOAD/SAVE GENAI VARIABLES via plugin ... Mirror of the 4.F MCP variable wiring, applied to genai-* variables. Closes the gap that surfaced in the ai-g1 spot-check where genai_module-t saw "default value" / "CHECKSUM GENAI VARIABLES" empty because flush_genai_variables___* was stubbed in Step 5 and there was no plugin-side replacement. New helpers (plugins/genai/src/plugin_main.cpp + genai_plugin.h): genai_load_variables_from_admindb(ctx) SELECT variable_name, variable_value FROM main.global_variables WHERE variable_name LIKE 'genai-%' → GloGATH->set_variable(name + 6, value) for each row. Mirrors the pre-Step-5 flush_genai_variables___database_to_runtime. genai_save_variables_to_admindb(ctx) Walks GloGATH->get_variables_list() and REPLACEs the `genai-<name>` rows in main.global_variables. Mirrors the pre-Step-5 flush_genai_variables___runtime_to_database. Both helpers wrlock GloGATH for the duration of their walk, so callers don't have to coordinate. New plugin-registered admin SQL verbs (plugin_commands.cpp): LOAD GENAI VARIABLES TO RUNTIME aliases: FROM MEMORY, FROM MEM, TO RUN SAVE GENAI VARIABLES TO MEMORY aliases: TO MEM, FROM RUNTIME, FROM RUN End-to-end verified against the running ai-g1 infra: SET genai-llm_provider='openai'; LOAD GENAI VARIABLES TO RUNTIME; SAVE GENAI VARIABLES TO MEMORY; SELECT * FROM main.global_variables WHERE variable_name LIKE 'genai-%'; -> 32 rows (the full GenAI_Threads_Handler variable set, populated by the SAVE FROM RUNTIME walk, including the value just SET). This is a follow-up gap the unit tests don't catch (the pattern mirrors LOAD/SAVE MCP VARIABLES which 4.F already validated end-to-end); ai-g1 caught it because genai_module-t hits CHECKSUM GENAI VARIABLES which evaluates over the populated row set. The pre-Step-5 ProxySQL_Admin::flush_genai_variables___* functions remain stubbed in core (no callers reach them now — the chassis dispatcher routes LOAD/SAVE GENAI VARIABLES verbs to the plugin before admin_handler_command_load_or_save can fire). Step 7's dead- code cleanup pass can drop them entirely.	2 months ago
Rene Cannao	62ed0660a7	feat(plugins/genai): Step 7 — drop PROXYSQLGENAI from core build ... Last step of the GenAI plugin carve-out. The PROXYSQLGENAI macro is no longer compiled into core: lib/libproxysql.a and the proxysql binary build identically with or without `PROXYSQLGENAI=1`. The user-facing build flag stays — it now means "also build the genai plugin alongside proxysql" rather than "compile GenAI into the binary". What this commit does plugins/genai/Makefile Hardcodes -DPROXYSQLGENAI in the plugin's CXXFLAGS — the plugin IS the GenAI surface, so the macro is always-on inside its own compilation unit. Lets the existing `#ifdef PROXYSQLGENAI` guards inside plugin source files keep working without a massive guard-stripping pass (deferred follow-up cleanup). lib/Makefile, src/Makefile Drop the `PSQLGA := -DPROXYSQLGENAI` setter — core no longer sees the macro on the compile line. PSQLGA stays as an empty variable so the existing `$(PSQLGA)` cascades in CXXFLAGS references stay structurally identical (cleanup deferred). Also: lib/Makefile no longer compiles MCP_Tool_Handler.oo (the base class moved to the plugin in this commit too — see below). Top-level Makefile Plugin-build rules switched from `$(if $(filter 1,$(PROXYSQL40))` to `$(if $(filter 1,$(PROXYSQLGENAI))` — `PROXYSQL40=1` enables the chassis but not necessarily the genai plugin. Skip-message text updated to match. `PROXYSQLGENAI=1` still implies `PROXYSQL40=1` per the existing tier hierarchy, so the user invocation stays unchanged. MCP_Tool_Handler base class Moved to plugins/genai/ (`include/MCP_Tool_Handler.h` -> plugin include, `lib/MCP_Tool_Handler.cpp` -> plugin tool_handlers). Was the last GenAI-tier .cpp in core — only existed there because AI/RAG_Tool_Handler still inherited from it before Step 5. Now plugin-internal. lib/proxy_sqlite3_symbols.cpp Hook for sqlite_vec_init unconditional now (sqlite-vec extension object always linked into proxysql, since the genai plugin relies on it via this hook at dlopen time). lib/MySQL_Thread.cpp / lib/PgSQL_Thread.cpp `variables.ffto_enabled` default no longer flips to true when PROXYSQLGENAI is set (the gate was: PROXYSQLFFTO + PROXYSQLGENAI -> true; else false). With PROXYSQLGENAI removed, the default is false everywhere; operators opt in via admin SQL. CLAUDE.md Feature-tier table now lists Plugin Chassis (PROXYSQL40) and AI/MCP (PROXYSQLGENAI) separately, with a note explaining what PROXYSQLGENAI=1 means post-carve-out. Conditional Components section calls out that GenAI/MCP/RAG/LLM lives in plugins/genai/. What this commit does NOT do (deferred) - Stripping the `#ifdef PROXYSQLGENAI` guards from the plugin's own source files (LLM_Bridge.cpp, GenAI_Thread.cpp, etc. all have `#ifdef PROXYSQLGENAI ... #endif` brackets). The plugin's Makefile force-defines the macro so they keep expanding. A guard-stripping pass would be ~50 file edits with zero behaviour change. - Stripping the dead `#ifdef PROXYSQLGENAI` blocks from core (lib/Admin_Handler.cpp etc. still have FIXME-stubbed blocks inside `#ifdef PROXYSQLGENAI` from the 4.C / 5 surgery). With the macro undefined in core's compile line, those blocks silently expand to nothing — they're dead code already. Cleanup follow-up. - Removing the now-no-op stubs themselves (init_mcp_variables, flush_mcp_variables___*, load_mcp_server, stats___mcp_*, etc.). They linger as dead wrappers around early-returns; their declarations in proxysql_admin.h linger too. Cleanup follow-up. All three plugin unit tests pass: genai_plugin_load_unit-t : 25/25 genai_plugin_anomaly_unit-t : 6/6 genai_plugin_backend_client : 27/27 Carve-out scoreboard, FINAL: Step 0-2 : infrastructure (chassis cherry-pick, ABI extensions) Step 3 : ~1.1 K LOC Anomaly_Detector Step 4.A : ~1 K LOC GENAI: / LLM: query-prefix removal Step 4.B : ~0.5 K LOC backend_client helper Step 4.C : ~13 K LOC MCP subsystem + 8 tool handlers + MySQL_FTS Step 4.F : n/a runtime-reconfig admin SQL via plugin commands Step 4.G : n/a MCP admin/config/stats tables via plugin ABI Step 5 : ~7 K LOC GenAI/LLM/AI/RAG Step 6 : ~5 K LOC AI_Vector_Storage / Discovery_Schema / MySQL_Catalog / Static_Harvester pair Step 7 : n/a PROXYSQLGENAI macro dropped from core build Net core surface change: ~28 K LOC moved out of lib/ + include/ into plugins/genai/. src/proxysql binary shrunk from 197 MB (pre-carve-out, all PROXYSQLGENAI compiled in) to 152 MB; plugin .so grew to ~56 MB. Net process RSS roughly neutral.	2 months ago
Rene Cannao	4552236700	feat(plugins/genai): Step 6 — move RAG/Vector/discovery surface into the plugin ... Final file-move step in the carve-out: relocates the leaf modules that didn't depend on (and weren't depended on by) anything still in core. After this commit, the only PROXYSQLGENAI-guarded code remaining in libproxysql.a is `MCP_Tool_Handler` — the base class the plugin's AI_Tool_Handler / RAG_Tool_Handler inherit from — and that lives in core only because removing it would force pulling those derived plugin classes into core just for the inheritance edge. Step 7 deletes the PROXYSQLGENAI macro entirely. Files moved (git mv preserves history) Headers (include/ -> plugins/genai/include/) AI_Vector_Storage.h MySQL_Catalog.h Discovery_Schema.h Static_Harvester.h PgSQL_Static_Harvester.h Sources (lib/ -> plugins/genai/src/) AI_Vector_Storage.cpp MySQL_Catalog.cpp Discovery_Schema.cpp Static_Harvester.cpp PgSQL_Static_Harvester.cpp Core changes were minimal because these are leaf modules: include/proxysql.h: drop the AI_Vector_Storage.h include. include/cpp.h: drop AI_Vector_Storage / Discovery_Schema / MySQL_Catalog / Static_Harvester / PgSQL_Static_Harvester includes (replaced with a single carve-out history comment that Step 7 will delete entirely). lib/ProxySQL_Admin.cpp: drop the Discovery_Schema.h include — its only live use was via the now-#if-0'd save/load_mcp_query_rules blocks 4.C left behind for follow-up reference. lib/Makefile: drop AI_Vector_Storage.oo, MySQL_Catalog.oo, Discovery_Schema.oo, Static_Harvester.oo, PgSQL_Static_Harvester.oo from the GenAI _OBJ_CXX list. Test fixture probe update test/tap/tests/unit/Makefile: PROXYSQLGENAI auto-detect probe switched from `AI_Vector_Storage` (now plugin-only) to `MCP_Tool_Handler` (the last GenAI-tier symbol still in core). Step 7 deletes the auto-detect entirely. All three plugin unit tests pass: genai_plugin_load_unit-t : 25/25 genai_plugin_anomaly_unit-t : 6/6 genai_plugin_backend_client : 27/27 Binary size deltas: src/proxysql 159 MB -> 152 MB (-7 MB) plugins/genai/...Plugin.so 47 MB -> 55 MB (+8 MB) Net process RSS roughly neutral, as the design predicted. Carve-out scoreboard: Step 0-2 : infrastructure Step 3 : ~1.1 K LOC (Anomaly_Detector) Step 4.B : ~0.5 K LOC (backend_client) Step 4.C : ~13 K LOC (MCP subsystem + tool handlers + MySQL_FTS) Step 4.F : runtime-reconfig admin SQL routed through plugin commands Step 4.G : MCP admin / config / stats tables registered via plugin ABI Step 5 : ~7 K LOC (GenAI/LLM/AI/RAG) Step 6 : ~5 K LOC (AI_Vector_Storage + Discovery_Schema + MySQL_Catalog + Static_Harvester pair) Pending: Step 7 — delete the PROXYSQLGENAI macro from build flags and any remaining `#ifdef PROXYSQLGENAI` blocks.	2 months ago
Rene Cannao	28cf4ceab8	feat(plugins/genai): Step 5 — move GenAI/LLM/AI surface into the plugin ... Largest carve-out commit since 4.C: moves the GenAI / LLM / AI / RAG surface (about 7K LOC) out of core into plugins/genai/. After this commit, libproxysql.a no longer publishes any of GenAI_Threads_Handler, LLM_Bridge, LLM_Clients, AI_Features_Manager, AI_Tool_Handler, RAG_Tool_Handler, or the legacy GloGATH/GloAI globals. Files moved (git mv preserves history) Headers (include/ -> plugins/genai/include/) GenAI_Thread.h LLM_Bridge.h AI_Features_Manager.h AI_Tool_Handler.h RAG_Tool_Handler.h (promoted from Step 6 — bidirectional dep with AI_Features_Manager forced the early move, same way MySQL_FTS.h got promoted in 4.C) Sources (lib/ -> plugins/genai/src/ or …/tool_handlers/) GenAI_Thread.cpp LLM_Bridge.cpp LLM_Clients.cpp (.cpp without a header; class decls live in LLM_Bridge.h) AI_Features_Manager.cpp AI_Tool_Handler.cpp -> tool_handlers/ RAG_Tool_Handler.cpp -> tool_handlers/ Plugin lifecycle takeover plugins/genai/src/plugin_main.cpp: - Defines plugin-local GloGATH and GloAI alongside the existing GloMCPH (all three are .so-internal aliases for the contexts genai_init constructs). - genai_init now also constructs GenAI_Threads_Handler (calls ->init()) and AI_Features_Manager (calls ->init()), mirroring the pre-Step-5 ProxySQL_Main_init_GenAI_module sequence. - genai_stop tears them down in reverse order. plugins/genai/src/ProxySQL_MCP_Server.cpp: - AI / RAG endpoint registration re-enabled (was #if 0'd in 4.C because the constructors weren't reachable from the plugin then). /mcp/ai and /mcp/rag are once again live when the operator has GloAI configured. plugins/genai/src/Anomaly_Detector.cpp + plugin_main.cpp: - get_query_embedding() now calls through a function-pointer hook (genai_anomaly_embed_fn) that plugin_main installs in genai_init and clears in genai_stop. This indirection lets the test fixture (which compiles Anomaly_Detector.cpp directly into a test binary, without GenAI_Thread.cpp) link cleanly — the pointer stays null, the detector short-circuits to empty embedding, and the existing nullptr-guard in check_embedding_similarity does the rest. Core neutralization src/main.cpp: - Drops the GloGATH / GloAI globals, ProxySQL_Main_init_GenAI_module definition + call, the shutdown branch, and the `if (GloGATH) init_genai_variables()` call. lib/Admin_Bootstrap.cpp / Admin_FlushVariables.cpp / Admin_Handler.cpp / ProxySQL_Admin.cpp / MySQL_Session.cpp / ProxySQL_Admin_Stats.cpp: - Drop the corresponding `#include` and `extern` declarations. - flush_genai_variables___* stubbed (mirrors the 4.C MCP pattern; original bodies preserved in `#if 0` for the upcoming LOAD/SAVE GENAI verbs port). lib/Admin_Handler.cpp: - Loose-prefix accept for `genai-*` in is_valid_global_variable (mirrors the 4.F mcp-* fix), so SET genai-X=Y still reaches main.global_variables; runtime push happens via the plugin's LOAD GENAI VARIABLES TO RUNTIME (registered in a follow-up). include/cpp.h, include/proxysql.h: - Removed the AI_Features_Manager.h / LLM_Bridge.h / GenAI_Thread.h / AI_Tool_Handler.h / RAG_Tool_Handler.h includes that no longer resolve from core's -I path. test/tap/test_helpers/test_globals.cpp: - Drops the matching `#include`s and stub globals. Test fixture test/tap/tests/unit/Makefile: - PROXYSQLGENAI auto-detect probe switched from `GenAI_Thread` (now plugin-only) to `AI_Vector_Storage` (still in core for Step 6). After Step 6 the probe will switch again, and Step 7 deletes the auto-detect entirely. All three plugin unit tests pass: - genai_plugin_load_unit-t : 25/25 - genai_plugin_anomaly_unit-t : 6/6 - genai_plugin_backend_client : 27/27 Carve-out scoreboard: Step 0-2 : infrastructure Step 3 : ~1.1 K LOC (Anomaly_Detector) Step 4.B : ~0.5 K LOC (backend_client) Step 4.C : ~13 K LOC (MCP subsystem) Step 4.F : runtime-reconfig admin SQL routed through plugin commands Step 4.G : MCP admin / config / stats tables registered via plugin ABI Step 5 : ~7 K LOC (GenAI/LLM/AI/RAG) Pending: Step 6 (AI_Vector_Storage, MySQL_Catalog, Discovery_Schema, Static_Harvester, PgSQL_Static_Harvester) and Step 7 (delete the PROXYSQLGENAI macro entirely).	2 months ago
Rene Cannao	9078fd3150	feat(plugins/genai): Step 4.G — register MCP admin tables via plugin ABI ... Moves MCP admin / config / stats SQLite table registration from lib/Admin_Bootstrap.cpp into the plugin via the chassis register_table service. Pure refactor: the same DDL strings produce the same SQLite tables in the same SQLite DBs; only the registration path changes. New plugins/genai/src/plugin_tables.cpp registers 14 tables: admin DB (6) mcp_query_rules, runtime_mcp_query_rules mcp_auth_profiles, runtime_mcp_auth_profiles mcp_target_profiles, runtime_mcp_target_profiles config DB (3) mcp_query_rules, mcp_auth_profiles, mcp_target_profiles (no runtime_* in config; matches the pre-4.G layout) stats DB (5) stats_mcp_query_tools_counters, _reset stats_mcp_query_digest, _reset stats_mcp_query_rules The DDL strings still live in include/ProxySQL_Admin_Tables_Definitions.h for now — plugin_tables.cpp #includes the header. Migrating the macros into a plugin-private header is a follow-up gated on no remaining core caller of the MCP table macros (Step 7 territory). Core changes (lib/Admin_Bootstrap.cpp): - Deleted the 9 admin/config insert_into_tables_defs() entries inside `#ifdef PROXYSQLGENAI` for mcp_*. - Deleted the 5 stats insert_into_tables_defs() entries similarly. - Replaced both blocks with a one-line comment pointing at the new plugin owner. Plugin lifecycle: - genai_init() now calls genai_register_admin_tables() right after genai_register_admin_commands(). Both run during chassis Phase C (init), before admin module bootstrap so the chassis dispatcher merges plugin tables into the admin DB schema before any DML. Test extension (test/tap/tests/unit/genai_plugin_load_unit-t.cpp): - Updated the "skeleton registers no tables (yet)" assertions to expect 6 admin / 3 config / 5 stats — matching plugin_tables.cpp. - 25/25 tests passing. What this changes for operators (no-op): - Same SQL: SHOW TABLES still shows mcp_*, runtime_mcp_*, stats_mcp_*. - Same DDL: column lists, primary keys, defaults — all preserved. - Same admindb / configdb / statsdb attachment.	2 months ago
Rene Cannao	f90307ad75	feat(plugins/genai): Step 4.F — register LOAD/SAVE MCP QUERY RULES TO/FROM RUNTIME ... Adds the last LOAD/SAVE pair the pre-4.C admin SQL surface had: the MCP query rules cache held by Query_Tool_Handler's Discovery_Schema catalog. New helpers in plugins/genai/src/plugin_main.cpp: mcp_load_query_rules_to_runtime(ctx) SELECT * FROM main.mcp_query_rules WHERE active=1 ORDER BY rule_id → catalog->load_mcp_query_rules(rs). Mirrors the pre-4.C ProxySQL_Admin::load_mcp_query_rules_to_runtime. mcp_save_query_rules_from_runtime(ctx, runtime) Pulls catalog->get_mcp_query_rules() and REPLACEs the rows in main.mcp_query_rules (or runtime_mcp_query_rules when runtime=true). Mirrors the pre-4.C ProxySQL_Admin::save_mcp_query_rules_from_runtime. Both helpers fail cleanly (return false, log to stderr) when the MCP listener isn't running — there's no in-memory Query_Tool_Handler cache to read or write. Pre-4.C behaviour was to attempt a load_mcp_server() self-recovery in that case; we no longer have that shortcut from outside the listener lifecycle, so callers get a clear error instead of an opaque retry. New verbs registered in plugin_commands.cpp: LOAD MCP QUERY RULES TO RUNTIME aliases: FROM MEMORY, FROM MEM, TO RUN SAVE MCP QUERY RULES TO MEMORY aliases: TO MEM, FROM RUNTIME, FROM RUN Required new core-side includes in plugin_main.cpp for the Discovery_Schema and Query_Tool_Handler types these helpers traffic in. The .so already linked against them via the chassis --whole-archive trick on libproxysql.a; this is just a header dependency. Test extension (test/tap/tests/unit/genai_plugin_load_unit-t.cpp): - 6 new assertions for the QUERY RULES canonical + 5 aliases. - 25/25 tests passing. Carve-out scoreboard, runtime reconfig surface restored: LOAD MCP VARIABLES TO RUNTIME / FROM MEMORY / FROM MEM / TO RUN LOAD MCP PROFILES TO RUNTIME / FROM MEMORY / FROM MEM / TO RUN LOAD MCP QUERY RULES TO RUNTIME / FROM MEMORY / FROM MEM / TO RUN SAVE MCP VARIABLES TO MEMORY / TO MEM / FROM RUNTIME / FROM RUN SAVE MCP QUERY RULES TO MEMORY / TO MEM / FROM RUNTIME / FROM RUN Still pending: SET mcp-X=Y (chassis ABI gap, separate enhancement).	2 months ago
Rene Cannao	7a4f08416e	feat(plugins/genai): Step 4.F — register SAVE MCP VARIABLES TO MEMORY ... Adds the SAVE direction (runtime → main.global_variables) to the plugin's admin SQL surface, mirroring the LOAD direction wired in the previous commit. New helper plugins/genai/src/plugin_main.cpp::mcp_save_variables_to_admindb Walks ctx.mcp->get_variables_list(), reads each runtime value via ctx.mcp->get_variable, and REPLACEs the corresponding "mcp-<name>" rows in main.global_variables. Owns its own MCP_Threads_Handler wrlock so callers don't have to coordinate. Mirrors the pre-4.C flush_mcp_variables___runtime_to_database with runtime=false (no runtime_global_variables view write). New verb registered in plugins/genai/src/plugin_commands.cpp: SAVE MCP VARIABLES TO MEMORY (canonical) aliases: SAVE MCP VARIABLES TO MEM SAVE MCP VARIABLES FROM RUNTIME SAVE MCP VARIABLES FROM RUN Same alias convention as the LOAD verbs and as plugins/mysqlx. The on-disk write ("SAVE MCP VARIABLES TO DISK") stays in core's admin_handler_command_load_or_save — it's a pure SQL DML that copies main.* to disk.* and doesn't need plugin runtime state. Test extension (test/tap/tests/unit/genai_plugin_load_unit-t.cpp): - 4 new TAP assertions for the SAVE canonical + 3 aliases. - 19/19 tests passing. Carve-out scoreboard, runtime reconfig surface restored: LOAD MCP VARIABLES TO RUNTIME / FROM MEMORY / FROM MEM / TO RUN LOAD MCP PROFILES TO RUNTIME / FROM MEMORY / FROM MEM / TO RUN SAVE MCP VARIABLES TO MEMORY / TO MEM / FROM RUNTIME / FROM RUN Still pending: - SET mcp-X=Y (chassis ABI lacks per-variable namespace registration; would need a new ABI surface). - LOAD/SAVE MCP QUERY RULES TO/FROM RUNTIME (Query_Tool_Handler in plugin owns the runtime cache; helpers come in next commit).	2 months ago
Rene Cannao	ae13958995	feat(plugins/genai): Step 4.F (continued) — register LOAD MCP {VARIABLES,PROFILES} TO RUNTIME ... Adds plugins/genai/src/plugin_commands.cpp registering two admin SQL verbs through the chassis command registry, restoring runtime reconfiguration that 4.C disabled. Verbs registered (each with FROM MEMORY / FROM MEM / TO RUN aliases): LOAD MCP VARIABLES TO RUNTIME -> mcp_load_variables_from_admindb() -> mcp_start_listener_if_enabled() Re-pushes mcp-* values from main.global_variables into the running MCP_Threads_Handler, then re-evaluates the listener state (start if newly enabled). Replaces the FIXME-stubbed ProxySQL_Admin::flush_mcp_variables___database_to_runtime path. LOAD MCP PROFILES TO RUNTIME -> mcp_load_target_auth_map_from_admindb() Refreshes runtime_mcp_auth_profiles / runtime_mcp_target_profiles from main.* and rebuilds the in-memory target_auth_map. Replaces the FIXME-stubbed init_mcp_variables auth-map block. The chassis dispatcher in lib/Admin_Handler.cpp:3987 runs BEFORE admin_handler_command_load_or_save, so any admin SQL matching a registered plugin verb routes to the plugin and the (now no-op) hardcoded core handlers never fire. No core changes needed. What's still stubbed (deferred to follow-up commits): - SAVE MCP VARIABLES TO MEMORY / FROM RUNTIME (needs runtime -> admindb push in plugin) - SAVE MCP PROFILES TO DISK (writes runtime view back to disk.*) - LOAD/SAVE MCP QUERY RULES (Query_Tool_Handler in plugin owns them, but the runtime->admindb push isn't wired through plugin yet) - SET mcp-X=Y (chassis ABI has no register_variable_namespace; would need either a new ABI surface or per-variable register_command — defer to a chassis enhancement) Refactor: moved mcp_load_variables_from_admindb, mcp_load_target_auth_map_from_admindb, mcp_start_listener_if_enabled out of the anonymous namespace in plugin_main.cpp so plugin_commands.cpp can call them. Forward declarations added to genai_plugin.h. Test extension (test/tap/tests/unit/genai_plugin_load_unit-t.cpp): - 7 new TAP assertions verifying canonical SQL and aliases resolve through the plugin manager's resolve_alias_to_canonical, which is the same path admin SQL dispatch uses. - 15/15 tests passing.	2 months ago
Rene Cannao	5fe286e972	feat(plugins/genai): Step 4.C — move MCP subsystem into the plugin ... Per docs/superpowers/plans/2026-04-19-step4-mcp-subsystem-move.md "4.C / 4.D / 4.E merge" revision. This is the largest single carve-out commit so far: ~13 K LOC physically relocated, plus surgical edits across the core admin SQL surface to neutralize references to the moved code. What moved (git mv preserves history) Tool handlers (lib/<name>.cpp -> plugins/genai/src/tool_handlers/<name>.cpp; include/<name>.h -> plugins/genai/include/<name>.h) Admin_Tool_Handler, Cache_Tool_Handler, Config_Tool_Handler, MySQL_Tool_Handler, Observe_Tool_Handler, Query_Tool_Handler, Stats_Tool_Handler. MCP listener stack (lib/<name>.cpp -> plugins/genai/src/<name>.cpp; include/<name>.h(pp) -> plugins/genai/include/...) MCP_Endpoint, MCP_Thread, ProxySQL_MCP_Server. Promoted from Step 6 (bidirectional dep with MySQL_Tool_Handler): MySQL_FTS. MCP_Tool_Handler base stays in core because AI_Tool_Handler and RAG_Tool_Handler (Steps 5/6 scope) inherit from it; moving the base would force pulling those in too. Plugin lifecycle takeover plugins/genai/include/genai_plugin.h: GenAIPluginContext now holds the MCP_Threads_Handler*. plugins/genai/src/plugin_main.cpp: genai_init constructs the MCP_Threads_Handler and assigns to a plugin-local GloMCPH symbol (the legacy global name, kept for the moved tool handlers that still reference it; the symbol no longer exists in core). genai_stop tears down on shutdown. genai_start is a no-op for now; the MCP listener auto-start is gated on the admin-SQL surface that 4.F restores. plugins/genai/src/ProxySQL_MCP_Server.cpp: AI_Tool_Handler and RAG_Tool_Handler endpoint registration is disabled via stub assignments (the headers stay included so member-pointer types resolve). Steps 5/6 re-enable when those classes move into the plugin. Core neutralization (FIXME stubs awaiting 4.F/4.G) src/main.cpp: GloMCPH global removed; ProxySQL_Main_init_MCP_module removed; shutdown block deleted. lib/ProxySQL_Admin.cpp: init_mcp_variables, load_mcp_server, load_mcp_query_rules_to_runtime, save_mcp_query_rules_from_runtime stubbed to no-ops; original bodies preserved inside `#if 0` for 4.F reference. lib/Admin_FlushVariables.cpp: flush_mcp_variables___database_to_runtime and flush_mcp_variables___runtime_to_database stubbed similarly. lib/ProxySQL_Admin_Stats.cpp: stats___mcp_query_tools_counters, stats___mcp_query_digest, stats___mcp_query_rules stubbed. lib/Admin_Handler.cpp: has_variable("mcp-…") and the load_target_auth_map call site in LOAD MCP PROFILES … stubbed. Result: the MCP listener does NOT start automatically between this commit and 4.F (mcp-* admin variables are not pushed into the running plugin). Documented in the plan's "4.C/4.D/4.E merge" section. src link change: --whole-archive on libproxysql.a src/Makefile: the proxysql binary now links libproxysql.a with -Wl,--whole-archive so all symbols are exported via -Wl,--export-dynamic, allowing the genai plugin .so to dlopen cleanly even when core code paths don't reference moved-but-still- in-core classes (LLM_Bridge, Discovery_Schema, MySQL_Catalog, Static_Harvester, …) directly. test/tap/tests/unit/Makefile gets the same change so unit tests can dlopen the plugin. Without this, ld dead-code-eliminates the unreferenced symbols and dlopen fails with "undefined symbol". Tests passing - genai_plugin_anomaly_unit-t : 6/6 - genai_plugin_load_unit-t : 8/8 (lifecycle init/start/stop + admin table registration) - genai_plugin_backend_client_unit-t : 27/27 Carve-out scoreboard Step 0-2 : 0 (infrastructure only) Step 3 : ~1.1 K (Anomaly_Detector) Step 4.B : ~0.5 K (backend_client helper, no callers yet) Step 4.C : ~13 K (this commit; MCP subsystem) Pending : Steps 4.F, 4.G, 5, 6, 7	2 months ago
Rene Cannao	4080606672	feat(plugins/genai): Step 4.B — add backend_client helper for local-proxy connections ... Per docs/superpowers/plans/2026-04-19-step4-mcp-subsystem-move.md "Sub-step 4.B", introduces the helper that 4.D will use to rewire Query_Tool_Handler and MySQL_Tool_Handler from direct-to-backend dialing to dialing 127.0.0.1:<local-proxy-port>. This commit is functionally a no-op: nothing in the plugin or core calls the new helper yet. The .so links it in for future use; only the unit test exercises it. New files: plugins/genai/include/backend_client.h BackendTarget, MySQL/PgSQLDialResult, dial_*, dial_*_local, LocalProxyEndpoint, parse_interfaces_first_tcp, resolve_*_endpoint. plugins/genai/src/backend_client.cpp dial_mysql / dial_pgsql: thin wrappers around mysql_real_connect / PQconnectdb mirroring the dial sequence currently inlined in Query_Tool_Handler.cpp. No CLIENT_MULTI_STATEMENTS change of semantics relative to existing code. dial_*_local: resolve local proxy endpoint via admindb, then dial. plugins/genai/src/local_proxy_endpoint.cpp Reads mysql-interfaces / pgsql-interfaces from global_variables, parses the semicolon-separated host:port format (mirrors MySQL_Thread::listener_add's parser at lib/MySQL_Thread.cpp ~L240), canonicalises 0.0.0.0 -> 127.0.0.1 and :: -> ::1, skips Unix-socket tokens. No caching today; trivial to add when 4.D wires real callers. test/tap/tests/unit/genai_plugin_backend_client_unit-t.cpp 27 TAP assertions: parser variants (IPv4/IPv6/wildcards/multi/Unix- skip/whitespace/port-bounds), null-admindb guards on every public entry point, invalid-host/zero-port guards on direct-dial. Full MySQL handshake left for the 4.D integration coverage. Plumbing: - plugins/genai/Makefile: compile both new .cpp into the .so. - test/tap/tests/unit/Makefile: build genai_plugin_backend_client_unit-t using the same compile-twice trick as genai_plugin_anomaly_unit-t. - test/tap/groups/groups.json: register the new test in unit-tests-g1 with @proxysql_min_version:4.0. Resolves Q-svcuser and Q-localhost-port from the carve-out spec design, choosing existing mcp_auth_profiles credentials and admindb-driven endpoint discovery respectively (no new mechanisms).	2 months ago
Rene Cannao	f1c0ea2dbc	feat(genai): Step 3 — carve Anomaly_Detector out of core into plugins/genai/ ... Per docs/superpowers/specs/2026-04-16-genai-plugin-carveout-design.md "Migration sequence — Step 3", this is the first step that actually moves GenAI surface out of libproxysql.a. The Anomaly_Detector now lives entirely inside the genai plugin and is exercised through the plugin query-hook ABI added in Step 2; core has zero references to it. Files moved (git mv preserves history): include/Anomaly_Detector.h -> plugins/genai/include/Anomaly_Detector.h lib/Anomaly_Detector.cpp -> plugins/genai/src/Anomaly_Detector.cpp Plugin-side adjustments: - Drop the `#ifdef PROXYSQLGENAI` guard from both files (the file only compiles inside the plugin now; the macro remains a core build flag but the carve-out gates these files by *location*, not by macro). - Update the json-include relative path to the new depth. - Replace `Anomaly_Detector::get_query_embedding` with a stub that returns an empty vector. The previous implementation called `GloGATH->embed_documents`, but `GloGATH` (the GenAI_Threads_Handler) is still a core symbol at this point in the carve-out. The embedding-similarity stage was already nullptr-guarded on `vector_db`, so an empty embedding short-circuits cleanly to "no anomaly". Step 5 reattaches the embedding back-end when GenAI_Thread moves into the plugin. Plugin lifecycle wiring (plugins/genai/src/plugin_main.cpp): - `genai_init` now: registers two Prometheus counters (`proxysql_genai_detected_anomalies_total`, `proxysql_genai_blocked_queries_total`) against the shared registry exposed by Step 2.3, registers the query hook for both MySQL and PgSQL protocols, and constructs the Anomaly_Detector. - `genai_start`: flips the started flag (the detector has no separate start phase). - `genai_stop`: closes and deletes the detector; counters stay registered (prometheus-cpp has no Unregister). Plugin query-hook adapter (plugins/genai/src/plugin_hooks.cpp, NEW): - Bridges ProxySQL_PluginQueryHookPayload to Anomaly_Detector::analyze. - Increments the appropriate Prometheus counter on detection / block. - Translates AnomalyResult into ALLOW or DENY(message) per the ABI. Core removals (the symmetric carve-out): - lib/MySQL_Session.cpp: delete the in-core `#ifdef PROXYSQLGENAI` anomaly-detection block in COM_QUERY (lines around 5370-5377 pre-edit) and the `handler___status_WAITING_CLIENT_DATA___STATE_SLEEP___MYSQL_COM_QUERY_detect_ai_anomaly` method. Same logic now runs inside the plugin via the query hook added in Step 2.2. - include/MySQL_Session.h: drop the matching method declaration. - lib/AI_Features_Manager.{cpp,h}: remove the anomaly_detector field, init_anomaly_detector(), close_anomaly_detector(), get_anomaly_detector(), and the corresponding init() / shutdown() call sites. AI_Features_Manager now owns only the LLM bridge and vector_db (LLM_Bridge moves in Step 5; vector_db in Step 6). - lib/AI_Tool_Handler.{cpp,h}: drop the anomaly_detector field and the constructor parameter that fed it -- the field was stored but never read, dead code that was awkward to keep alive across the carve-out. - lib/ProxySQL_MCP_Server.cpp: update the AI_Tool_Handler ctor call site accordingly (one less argument). - include/MySQL_Thread.h: remove `st_var_ai_detected_anomalies` / `st_var_ai_blocked_queries` from `enum status_variable_id` and `ai_detected_anomalies` / `ai_blocked_queries` from `enum p_th_counter`. The plugin's Prometheus counters replace these (the old ones were per-thread integer increments surfaced by the legacy stats path; the plugin uses Prometheus directly). - lib/MySQL_Thread.cpp: drop the matching entries from MySQL_Thread_status_variables_counter_array and from th_metrics_map (the prometheus registration table). - include/proxysql.h, include/cpp.h: drop the now-stale `#include "Anomaly_Detector.h"` lines. - lib/Makefile: drop Anomaly_Detector.oo from the libproxysql object list. Tests: - Delete the in-core TAP tests that linked against the removed symbols: anomaly_detection-t, anomaly_detection_integration-t, anomaly_detector_unit-t (Docker-dependent, won't link), plus the unit test genai_anomaly_unit-t (linked against libproxysql's copy). - Add test/tap/tests/unit/genai_plugin_anomaly_unit-t.cpp: replaces the deleted unit test by compiling the plugin's Anomaly_Detector.cpp directly into the test binary. The friend-class helper pattern (Anomaly_Detector_TestHelper) is preserved so private methods (normalize_query, check_sql_injection) remain testable. Six assertions cover the same surface plus a regression test that analyze() now runs cleanly without a vector_db. Plugin Makefile changes: - Add Anomaly_Detector.cpp + plugin_hooks.cpp to SRCS. - Pull in the include paths Anomaly_Detector.cpp's transitively- included headers need (mariadb client for mysql.h, libev, libscram, libusual, postgresql, jemalloc, microhttpd, libhttpserver, curl, re2, pcre, zstd, libconfig, libdaemon, ssl). These mirror the set already used by test/tap/tests/unit -- the test harness pulls the same headers in transitively from libproxysql.a's headers. Doxygen documentation added for every new function and for the new header structures (GenAIPluginContext fields, genai_query_hook, register_prometheus_counters, all three lifecycle callbacks, plus file-level @brief blocks for plugin_main.cpp, plugin_hooks.cpp, genai_plugin.h, and the moved Anomaly_Detector.{h,cpp}). Verified end-to-end: - Full clean rebuild of libproxysql.a + proxysql + ProxySQL_GenAI_Plugin.so. - All 60 unit-test binaries pass (genai_anomaly_unit-t replaced by genai_plugin_anomaly_unit-t; net same). - Smoke test: launching proxysql with plugins=("...ProxySQL_GenAI_Plugin.so") shows "Anomaly: Initializing Anomaly Detector v0.1.0" / "Anomaly Detector initialized with 11 injection patterns" in the log on startup, and "Anomaly Detector closed" on SIGTERM -- the plugin's lifecycle owns the detector cleanly. Carve-out scoreboard so far (lines of GenAI surface moved out of core): Step 0-2 : 0 (infrastructure prep only) Step 3 : ~1100 (Anomaly_Detector.{h,cpp}) + counter/handler removals	2 months ago
Rene Cannao	9d06ab90f5	feat(plugins/genai): Step 1 — empty plugin skeleton ... Creates the GenAI plugin shell that subsequent steps will fill out (per docs/superpowers/specs/2026-04-16-genai-plugin-carveout-design.md "Migration sequence — Step 1"). plugins/genai/ ├── Makefile (mirrors plugins/mysqlx/Makefile) ├── .gitignore ├── include/genai_plugin.h (shared plugin context, atomic started flag) └── src/plugin_main.cpp (descriptor + empty init/start/stop/status_json) Top-level Makefile: cd plugins/genai for every existing build_src* / clean target that already cd's into plugins/mysqlx — release, debug, legacy, and all test_* flavours (testaurora, testaurora_random, testgalera, testgrouprep, testreadonly, testreplicationlag, testall). Unit test: test/tap/tests/unit/genai_plugin_load_unit-t.cpp drives the real .so through load → init → start → stop and asserts that the skeleton currently registers no admin/config/stats tables. Wired into the unit Makefile with an explicit build dependency on the .so so a clean checkout can produce a passing test in one make invocation. End-to-end smoke verified: launching proxysql with plugins=("…/ProxySQL_GenAI_Plugin.so") in proxysql.cnf maps the .so into /proc/PID/maps, leaves the admin port responsive, and shuts down cleanly on SIGTERM. All 58 unit-test binaries pass (+1 over baseline). Notes on what is intentionally NOT in this commit: - No GenAI code has moved out of core yet — that begins Step 3. - The skeleton has no tables, no commands, no query hook, no status vars — those are Step 2 (ABI extensions) and onwards. - Pre-existing breakage in test/tap/tests/Makefile (the generic %-t pattern can't link plugin tests because it doesn't pull the core globals stub) is unchanged; the genai test sidesteps it by living under test/tap/tests/unit/, which has the proper harness. The mysqlx test in test/tap/tests/ remains broken in the same way and is left for a separate fix.	2 months ago