2358 Commits (5eefca3ebfec6dec3f4aeae47dbc2e1787455502)

Author	SHA1	Message	Date
Rene Cannao	15c8bfcb78	refactor(admin): introduce FlushVariableStats and change flush_*_variables signatures ... Adds the FlushVariableStats struct and changes the return type of the flush_*_variables___database_to_runtime family of functions (and the generic process helper) from void to FlushVariableStats, so the next tasks can populate per-call counters (records/updated/rejected/unknown) and surface rejected-row counts in the LOAD ... TO RUNTIME response. Three public load_*_variables_to_runtime helpers are also updated to return the new struct, so callers can opt into the new counters. Build is intentionally broken at this point: the .cpp implementations still return void. The follow-up tasks will update the implementations to match the new signatures.	2 weeks ago
Rahim Kanji	eed52db897	Merge branch 'v3.0' into v3.0_partition-fairness	1 month ago
Rahim Kanji	6909011b67	perf(partition): promote longest-waiting B session via max_connect_time fold ... Under sustained contention ProcessAllSessions_Partition leaves the B (CONNECTING_SERVER) band unordered, so the pass can serve newer waiters ahead of older ones until they hit connect_timeout_server_max and abort. Track the oldest waiter during the classifier's is_B arm (no second pass) and swap it to running_end. Key on max_connect_time, which the is_B test already loads -- the min-compare is a register compare with no extra memory access, and the smallest value is the waiter closest to the timeout abort. Gated by |B| > PARTITION_FAIRNESS_MIN_B (=4) and partition_active.	1 month ago
René Cannaò	a5bb7f640f	Merge branch 'v3.0' into v3.0_partition-gate	1 month ago
René Cannaò	46ab989adb	Merge commit from fork ... fix(security): reject addresses after PROXY UNKNOWN (CVE-2026-48772 / GHSA-gw94-85m2-x8v2)	1 month ago
René Cannaò	3e7b77456e	Merge branch 'v3.0' into v3.0_partition-gate	1 month ago
René Cannaò	35cad4b4f4	Merge branch 'v3.0' into fix/kill-proxysqlgenai-build-flag-v2	1 month ago
Rene Cannao	0d39ac6991	build: kill PROXYSQLGENAI flag, PROXYSQL40=1 builds all plugins ... PROXYSQLGENAI was a leftover from before the genai plugin carve-out. All genai/MCP/AI/RAG code now lives in plugins/genai/ as a runtime loadable .so. Since the plugin chassis (PROXYSQL40=1) already builds and loads the genai .so, a separate flag serves no purpose. Entrypoint changes (all three -- rhel, suse, deb): - Build: pass PROXYSQL40=1 (not PROXYSQLGENAI=1) to make. - Packaging: copy genai .so alongside mysqlx when PROXYSQL40=1. - protobuf-devel install: gate on PROXYSQL40=1 (needed for mysqlx). Automated packaging verification added to all entrypoints: rpm -qpl / dpkg -c confirms both .so files are in the package. ELF smoke test: validates .so is a valid shared object exporting proxysql_plugin_descriptor_v1 symbol. Fails build on mismatch. Test Makefiles: PROXYSQLGENAI -> PROXYSQL40 for the vec.o link gate (test/tap/tests/Makefile and test/deps/cluster_simulator/Makefile). Updated remaining PROXYSQLGENAI references to PROXYSQL40 in: CLAUDE.md, include/cpp.h, lib/proxy_sqlite3_symbols.cpp, src/main.cpp (3x comments), proxysql.spec, test/infra/control/start-proxysql-isolated.bash, test/infra/control/run-unit-tests-asan-coverage.bash, and 14 genai test file comments.	1 month ago
Rene Cannao	031f6fa1f5	fix(build): adapt callers of dns_cache counters to std::atomic ... The previous commit promoted MySQL_Monitor::dns_cache_{queried, lookup_success,record_updated} to std::atomic<unsigned long long> but missed several call sites that pass them through implicit copy: * lib/MySQL_Thread.cpp sprintf("%llu", GloMyMon->dns_cache_*) — varargs cannot copy std::atomic. * lib/PgSQL_Thread.cpp sprintf("%llu", GloPgMon->dns_cache_*) — same, for the PgSQL counters. * lib/MySQL_Monitor.cpp p_update_counter(..., GloMyMon->dns_cache_*) — works via implicit operator T(), but explicit .load() is clearer and matches the rest of the file. Promote PgSQL_Monitor::dns_cache_* to std::atomic for the same race that motivated the MySQL side (resolver workers increment, stats readers and p_update_metrics read from another thread), and add .load() at every read site so we don't depend on implicit conversion. Verified with a local 'make -j32 debug' build in the integration worktree.	1 month ago
Rene Cannao	ca145d5026	fix(pgsql): atomic shutdown flag; honor dns_lookup fallback contract ... GloPgMon->shutdown was a plain bool, written by main.cpp's teardown and read by the monitor / DNS resolver loops on worker threads — a data race that can let the loops spin past shutdown or observe a stale value indefinitely. Promote to std::atomic<bool> and switch the readers/writer to acquire/release ordering. PgSQL_Monitor::dns_lookup() was also returning an empty string when GloPgMon / dns_cache_thread weren't initialized yet, ignoring the return_hostname_if_lookup_fails contract. Move the fallback out of the dns_cache_thread branch so startup / shutdown behave like a normal cache miss for callers that asked for the hostname fallback. Addresses CodeRabbit review feedback on PR #5806.	1 month ago
Rene Cannao	039d538c2a	fix(dns): correct inet_pton check; make DNS cache counters atomic ... validate_ip() treated inet_pton(...) != 0 as success, but inet_pton returns 1 / 0 / -1; a -1 (e.g. unsupported family) would be reported as a valid IP. Compare against == 1 instead. The DNS cache counters (dns_cache_queried / dns_cache_lookup_success / dns_cache_record_updated) were declared as plain unsigned long long on MySQL_Monitor. Resolver workers update them atomically via __sync_fetch_and_add, but p_update_metrics() reads them from a different thread without a barrier, racing the writers. Promote to std::atomic<unsigned long long> and switch DNS_Cache to fetch_add / the atomic pointer type so both ends are consistent. Addresses CodeRabbit review feedback on PR #5806.	1 month ago
Rene Cannao	a62d1b4e99	Address CodeRabbit + Gemini review feedback ... Fixes for actionable findings on PR #5806: DNS_Cache.cpp / hpp: - lookup(): initialize *ip_count = 0 on the disabled-cache fast path so callers don't observe a stale count from a previous lookup (CodeRabbit). - add_if_not_exist(): only bump counter_record_updated_ when an insert actually happened, and return that signal to callers. Previously a no-op call inflated the cache-update stats (CodeRabbit). - get_connected_peer_ip_from_socket(): drop the malloc, switch to a stack sockaddr_storage, and only assign result on AF_INET / AF_INET6 with a successful inet_ntop. Previously, on an unknown address family, we'd copy the uninitialized ip_addr buffer into result (Gemini + CodeRabbit). - monitor_dns_resolver_thread(): cache_ttl is signed int and at the max configured TTL (7*24*3600*1000 ms) `1000 * cache_ttl` overflows before being added to monotonic_time(). Force the multiply into unsigned long long space with 1000ULL (Gemini). - DNS_Cache::IP_ADDR::counter is now mutable so get_next_ip() (const) can __sync_fetch_and_add it without a const_cast (Gemini). PgSQL_Monitor.cpp: - monitor_dns_cache(): refresh pgsql_thread___monitor_local_dns_* before the loop, not just on the first version-bump. Otherwise the resolver runs its first pass against zero-initialized TTL / refresh interval and starts in the wrong enabled/disabled state until a later config bump (CodeRabbit). test/tap/tests/pgsql-test_dns_cache-t.cpp: - Step 7 ("Cache disabled by refresh_interval=0") was inserting an IP literal (0.0.0.0), which bypasses DNS regardless of the cache state. Switched to a resolvable hostname (example.com) so a regression in the cache-disable path would actually move counters and fail the assertion. Added a third assertion on dns_cache_lookup_success for symmetry. Plan adjusted to 17. Deliberately skipped: - CR comment about mysql_thread___resolution_family change requiring a cache flush: pre-existing MySQL behavior, out of scope for this PR. - CR comment about pthread_attr_init / setstacksize return checks in main.cpp: matches the existing un-checked pattern in MySQL_Monitor::run(). No need to diverge here. - Gemini comment about persistent thread pool vs. per-iteration spawn: MySQL_Monitor::monitor_dns_cache() also recreates the resolver threads each loop iteration; the PgSQL variant follows the same pattern. - Gemini comment about 2048-byte stack size: Thread::start takes ss in KB (lib/thread.cpp:46 `pthread_attr_setstacksize(&attr, ss*1024)`), so 2048 here means 2 MB, matching the MySQL resolver pool.	1 month ago
Rene Cannao	a19894427e	Add independent DNS cache for PgSQL ... Fixes the watchdog asserts under DNS degradation reported in #5768. Symptom: with a PgSQL backend addressed by hostname, an unhealthy resolver made every PQconnectStart() block ~5s on getaddrinfo inside libpq. Under load, all PgSQL_Thread workers spent the bulk of each loop iteration in that synchronous DNS call, atomic_curtime fell behind the watchdog threshold (poll_timeout + 1s), and after restart_on_missing_heartbeats misses the watchdog asserts. MySQL avoids this because mysql-monitor's DNS cache lets MySQL_Connection::connect_start pass an IP directly to libmariadb; PgSQL had no equivalent. This change adds the equivalent for PgSQL, with state fully independent of the MySQL cache: * PgSQL_Monitor gains its own DNS_Cache, force_dns_cache_update flag, and dns_cache_{queried,lookup_success,record_updated} counters. PgSQL_Monitor::dns_lookup / update_dns_cache_from_pgsql_conn / trigger_dns_cache_update mirror the MySQL_Monitor surface. * A new background thread (PgSQL_monitor_dns_cache_pthread) drives PgSQL_Monitor::monitor_dns_cache(), which walks PgHGM->pgsql_servers_to_monitor and feeds the shared DNSResolverWorker pool. Launched alongside PgSQL_monitor_scheduler and lives independently of pgsql-monitor_enabled, matching MySQL. * PgSQL_Connection::connect_start now consults the cache. On a hit it appends `hostaddr=<ip>` to the libpq conninfo while keeping `host=<hostname>`; libpq documents this combo specifically to skip name resolution while preserving TLS hostname verification. On a miss / IP literal / disabled cache we fall back to the previous behavior (libpq does getaddrinfo). * Successful PgSQL connects seed the cache via update_dns_cache_from_pgsql_conn, so the second connect avoids getaddrinfo even before the resolver loop has visited the host. * The pgsql-monitor_local_dns_cache_ttl / _refresh_interval / _resolver_queue_maxsize variables (already accepted by Admin via PgSQL_Thread's variable registry) now actually propagate to pgsql_thread___monitor_local_dns_* globals. The propagation block in PgSQL_Thread::refresh_variables was previously commented out, so the values never reached anywhere. * The runtime stats output (SQL3_GlobalStatus) now exposes the PgSQL_Monitor_dns_cache_* counters alongside the existing PgSQL_Monitor_* counters. DNSResolverWorker is shared infrastructure added to DNS_Cache.hpp/cpp, used here by PgSQL_Monitor. MySQL_Monitor still uses its existing ConsumerThread-based pool; nothing about the MySQL cache changes.	1 month ago
Rene Cannao	a2fff3725f	Extract DNS cache into shared DNS_Cache module ... DNS_Cache, DNS_Cache_Record, DNS_Resolve_Data, the resolver worker, and the validate_ip / get_connected_peer_ip_from_socket / debug_iplisttostring helpers used to live inside MySQL_Monitor.{hpp,cpp}. Move them to a new DNS_Cache.{hpp,cpp} so the same plumbing can back an independent PgSQL_Monitor cache. DNS_Cache no longer references GloMyMon directly; counters are wired in per-instance via set_counters(). monitor_dns_resolver_thread no longer reads mysql_thread___resolution_family; the desired ai_family is now a field on DNS_Resolve_Data set by the caller. MySQL_Monitor wires both up in its constructor and resolver loop respectively; behavior is identical. No change to the MySQL DNS cache semantics.	1 month ago
Rene Cannao	3ad60f6881	fix(security): hygiene follow-ups for PP1 UNKNOWN handling (GHSA-gw94-85m2-x8v2) ... Four small fixes that came out of post-fix review of the original GHSA-gw94 commit: 1. lib/proxy_protocol_info.cpp — defensively clear `header_was_unknown` at the top of parseProxyProtocolHeader(). The default constructor zero-inits it, but callers may reuse a ProxyProtocolInfo across multiple parse attempts; a stale `true` from a prior UNKNOWN parse would cause the caller to take the UNKNOWN branch on a later malformed parse, suppressing the legitimate warning. 2. lib/mysql_data_stream.cpp — in the UNKNOWN branch of buffer2array, populate PROXY_info->proxy_address and proxy_port from the real TCP peer (mirroring the TCP4/TCP6 branch at lines 1290-1296). The PROXYSQL INTERNAL SESSION JSON output previously reported empty proxy_address / proxy_port=0 for UNKNOWN frames even though the real upstream LB peer was known. 3. lib/mysql_data_stream.cpp — emit header_was_unknown in the PROXY_V1 JSON block so audit/observability consumers can distinguish a spec-compliant UNKNOWN observation (with intentionally-empty source/destination address fields) from any other state where those fields happen to be empty. 4. include/proxy_protocol_info.h — document the dual-return contract above parseProxyProtocolHeader(): returning false with header_was_unknown=true means "valid UNKNOWN, caller must NOT override client_addr". This guards against a future maintainer flipping the return value to true and accidentally letting the caller's success-branch strdup an empty source_address into addr.addr. No regression-test changes were needed — the existing unit test already pins header_was_unknown=true on UNKNOWN, and the integration test pins client.client_addr.address (not proxy_address) which is the real security boundary.	1 month ago
Rene Cannao	1d2e080a66	fix(security): reject address fields in PROXY-Protocol v1 UNKNOWN frames (GHSA-gw94-85m2-x8v2) ... The PP1 parser accepted "PROXY UNKNOWN <src> <dst> <sport> <dport>\r\n" as a well-formed header and let the parsed source address overwrite addr.addr, which is what Query_Processor reads for client_addr rule matching and what event_log / stats_mysql_processlist record. A peer that can reach the MySQL frontend (default mysql-proxy_protocol_networks = '*') could forge any source IP and bypass any client_addr-pinned ACL or routing rule. The HAProxy PROXY-protocol v1 spec is explicit: when the protocol token is UNKNOWN, the receiver MUST ignore any address fields that follow. Split UNKNOWN from TCP4/TCP6 in the parser: - For UNKNOWN, validate the byte after the token is ' ' or '\r' (rejects bogus tokens like "UNKNOWNFOO"), set a new header_was_unknown flag, and return false so the caller does not override addr.addr. - For TCP4/TCP6, require a single space delimiter after the 4-byte token before running sscanf. Previously a 4-byte memcmp accepted "TCP4xyz ..." as TCP4. Update mysql_data_stream.cpp to branch on header_was_unknown so it records the frame in PROXY_info without overriding addr.addr, and logs the skip at info level (not warning). PgSQL frontend is unaffected — only MySQL calls parseProxyProtocolHeader.	1 month ago
Rahim Kanji	53eea80aaf	feat(partition-gate): suppress low-volume ticks via min-attempts floor ... Add PARTITION_GATE_MIN_ATTEMPTS (=4). When a tick has fewer pool acquires than this, leave both gate state and streak counter untouched. Filters the "2/2 NULL = 100% stressed" noise case without drifting the gate OFF on transient idle ticks (which would happen if low- volume ticks were treated as not-stressed). Pairs with the existing mysql_sessions->len > 3 guard at the partition call site.	1 month ago
Rahim Kanji	878ba3b1e6	refactor(partition-gate): consolidate into Base_Thread; drain every iteration; widen ratio to uint64_t ... Three closely related changes: * Move the four partition_* members and note_pool_attempt() from MySQL_Thread / PgSQL_Thread into Base_Thread. Eliminates 22 lines of duplicated gate logic across the two protocol implementations. * Run update_partition_gate() once per process_all_sessions outer iteration, before the sess_sort / len > 3 / partition_active check. Per-tick counters can no longer accumulate stale across ticks where partition is skipped (sess_sort off, IDLE_THREADS maintenance thread, or session count below threshold), which in 5b5b937 could cause spurious gate flips after sess_sort toggled back on. * Widen the ratio comparison to uint64_t so the multiplications cannot overflow unsigned int (reachable in combination with the staleness issue above). Function-local HI_NUM/HI_DEN/STREAK locals are promoted to named static constexpr class members (PARTITION_GATE_NULL_RATIO_NUM/DEN/STREAK). No new behavior introduced -- purely correctness fixes plus a behavior-preserving refactor.	1 month ago
Rene Cannao	5b5b9373e3	feat(pgsql,mysql): gate ProcessAllSessions_Partition by pool-acquire NULL ratio ... ProcessAllSessions_Partition (Pass 1, the 3-way A/B/C session classification) is cheap and beneficial when block B (sessions waiting for a backend) is non-empty: it lets block A finish queries and release conns to block B waiters in the same outer iteration. When no sessions are contending for backends, partition is unnecessary work. Detect the contention state with a per-worker counter pair tracked at the get_MyConn_from_pool() call site: every call increments partition_pool_attempts; calls that returned NULL (couldn't acquire, session must wait) also increment partition_pool_nulls. At the top of each process_all_sessions outer iteration, the worker computes the NULL ratio and updates a hysteresis state machine: 3 consecutive iterations with ratio >= 5% flips partition_active to true; 3 consecutive iterations below the threshold flips it back. Mechanism: * Cheapest possible signal: 1 int increment per pool acquire, 1 branch per outer iteration. * Hysteresis (3 consecutive iterations either direction) filters transient bursts without needing min-sample / asymmetric-threshold tuning. * Single threshold 5% works in both directions by virtue of the streak filter -- the ratio has to be sustained near the boundary for flapping to occur, which doesn't match any realistic workload. Empirical verification at 1 worker thread, 500c clients, 4 KB rows, 50-conn backend pool, SSL on, 120 s: 20c (no contention): 1576 tps, 13 ms avg latency, gate stays OFF 100c (some): 1487 tps, 67 ms, gate stays ON 500c (heavy): 1487 tps, 336 ms, gate stays ON The gate transitions automatically; no admin variable to learn. Applied symmetrically to MySQL_Thread / MySQL_Session and PgSQL_Thread / PgSQL_Session. See #5791 for analysis.	1 month ago
Rene Cannao	e5beb0d65a	fix(pgsql,mysql): bounded local connection cache 1-in-N (#5791) ... The per-thread cached_connections cache in PgSQL_Thread/MySQL_Thread greedily kept every released backend conn local to the thread that released it. At high client count this self-reinforced into one thread hoarding all backend conns while sibling worker threads starved on the shared pool, manifesting as a throughput cliff at ~200 clients with 2 worker threads and a 50-conn backend pool. Change push_MyConn_local() to cache only 1 release in every N (N = mysql_threads / pgsql_threads); push the rest straight to the shared HGM pool. At N=1 (single worker) the behaviour is unchanged. At N>1 the cache still amortizes the bulk of the HGM wrlock cost while ensuring released conns flow to peer threads regularly enough that the pool is never drained on one side. See #5791 for the empirical analysis. Mechanism eliminates the 2-thread cliff at c=200 and recovers ~80% throughput at c=200, ~50% at c=1000 vs the always-cache baseline.	1 month ago
René Cannaò	6968de7ce8	Merge branch 'v3.0' into v3.0_latency_consistency_improvement	1 month ago
René Cannaò	43d3ee7cd2	Merge branch 'v3.0' into test/cluster-simulator	1 month ago
Rene Cannao	798c243fc4	fix(tsdb): serve dashboard from REST API port for same-origin fetch (#5684) ... The TSDB dashboard HTML was served from admin-web_port (default 6080) but its JS issues relative-URL fetches: <script src="/Chart.bundle.js"></script> fetch('/api/tsdb/metrics') fetch('/api/tsdb/query?...') Those URLs resolve against the page origin, so the browser tried to read /api/tsdb/* from :6080 -- but the endpoints live on admin-restapi_port (default 6070). Result: every fetch 404'd and the dashboard rendered "Error loading metrics". Move the dashboard and its Chart.bundle.js asset to the REST API server so the dashboard's relative URLs are same-origin with the endpoints they need. This is the smallest change that makes the feature actually work; backwards-compatibility isn't a constraint since the feature was completely broken at :6080. * lib/ProxySQL_RESTAPI_Server.cpp: register /tsdb and /Chart.bundle.js resources alongside the existing /api/tsdb/* handlers. * lib/ProxySQL_HTTP_Server.cpp: remove the now-redundant /tsdb route. Chart.bundle.js stays here too because other web-port dashboards (e.g. /, /stats) still reference it. * test/tap/tests/test_tsdb_api-t.cpp: rewrite the existing test to pin the new wiring. In particular, parse the live dashboard HTML, extract every src/href/fetch URL, and GET each one against the dashboard's origin -- this assertion fails on the pre-fix code and passes after. Verified locally: bug build fails 5/10 assertions, fixed build passes 10/10. User-facing change: the dashboard URL moves from http://host:admin-web_port/tsdb (broken) to http://host:admin-restapi_port/tsdb (working) The dashboard now requires admin-restapi_enabled=1 instead of admin-web_enabled=1; in practice both were required for the dashboard to be useful since the JS needed the API anyway.	1 month ago
Rahim Kanji	ca202ceba1	Refactor: partition sessions by backend state instead of sorting ... Replace ProcessAllSessions_SortingSessions with ProcessAllSessions_Partition. The old function did an implicit one-pass swap-sort over mysql_sessions by max_connect_time; the new one explicitly partitions sessions into three contiguous blocks in pdata according to their backend state: [0, running_end) A: running a query against the backend [running_end, idle_begin) B: acquiring a backend (max_connect_time != 0) [idle_begin, len) C: idle (no backend, or holding a conn while parked in WAITING_CLIENT_DATA)	2 months ago
Wazir Ahmed	66820fc176	chore(lint): Make `SQLite3DB` table helpers const-correct ... - `check_table_structure`, `build_table` and `check_and_build_table` never mutate their string arguments, so widen the parameter types from `char *` to `const char *`. - Drop the matching C-style casts at every call site that stripped const from string literals. Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	2 months ago
Wazir Ahmed	6c5d0fd5db	Merge branch 'v3.0' into cluster-simulator	2 months ago
Rene Cannao	0f8e014a33	Merge branch v3.0 into issue-5686-galera-vars ... Resolve groups.json conflict: keep test_set_statement_for-t from PR and set_parser_algorithm=3-g1 additions from v3.0.	2 months ago
Wazir Ahmed	eaa57d0f58	Merge branch 'v3.0' into cluster-simulator	2 months ago
René Cannaò	44c33cd2d3	Merge branch 'v3.0' into feature/parsersql-integration	2 months ago
Rene Cannao	5fea82e617	fix: resolve remaining SonarCloud warnings for PR #5741 ... - Replace #define with constexpr for ABI version constants - Replace strdup/free with std::string in AI_Features_Manager - Change mkdir permissions from 0755 to 0700 - Reduce nesting from 4 to 3 levels via early continue	2 months ago
Rene Cannao	b8cfa32c63	fix: address all critical and important code review findings (issue #5729) ... Critical fixes: - Move db_kind to end of ProxySQL_PluginRuntimeView (tail-append = free ABI-3 compat) - Add mutex to protect Anomaly_Detector::user_statistics from concurrent session threads - Replace system(mkdir -p) with pure mkdir() loop (command injection fix) - Replace 77 std::stoi/stoll with non-throwing parse_int_or_zero/parse_ll_or_zero - Fix proxysql.cnf filename: Mysqlx -> MySQLX - Add --whole-archive to non-ClickHouse link path (plugin symbol resolution) Important fixes: - Add default: case with warning to db_kind switch in ProxySQL_PluginManager.cpp - Add config_db dispatch and null-handle tests to plugin_runtime_views_unit-t (6 new tests) - Update CLAUDE.md with sqlite-vec PROXYSQLGENAI exception - Remove stale anomaly_detection-t Makefile target - Replace fixed 1024-byte buffer with dynamic sqlite3_mprintf in Anomaly_Detector Minor: update ABI.md, PLUGIN_API.md, stale comments, unit test plan counts	2 months ago
René Cannaò	222a5d1ec8	Merge branch 'v3.0' into feature/parsersql-integration ... Signed-off-by: René Cannaò <rene@proxysql.com>	2 months ago
Rene Cannao	86859e5743	fix: vendor parsersql properly - only track tarball + symlink, not extracted source ... The extracted parsersql-1.0.1/ source directory was committed to git, which violates the dependency convention. All other deps (re2, curl, libev, etc.) only track the tarball and a versioned symlink, with the extracted dir generated at build time. Changes: - git rm --cached parsersql-1.0.1/ (90 files removed from tracking) - Added symlink parsersql -> parsersql-1.0.1 (tracked) - Added deps/parsersql/parsersql-*/ to .gitignore - Updated deps/Makefile to use symlink path - Updated makefiles_paths.mk to use symlink path	2 months ago
René Cannaò	80b93f11a1	Merge branch 'v3.0' into feature/parsersql-integration	2 months ago
Rene Cannao	910a92b833	refactor: remove MCP stats DDL macros and design comments from core (issue #5729)	2 months ago
Rene Cannao	383655202a	feat(chassis): dispatch correct DB handle by db_kind in runtime view refresh (issue #5729)	2 months ago
Rene Cannao	8c47b41845	feat(abi): add db_kind to ProxySQL_PluginRuntimeView, bump ABI v4 (issue #5729)	2 months ago
Rene Cannao	391afb524f	upgrade: vendor ParserSQL 1.0.1 with SET parser fixes, update test expectations ... - Vendored ParserSQL 1.0.1 (was 1.0.0) with fixes for issues #34, #35, #36: - #34: Comma-separated assignments after SET NAMES/CHARSET - #35: ON/OFF keyword values accepted as SET RHS - #36: Empty AST detection for incomplete SET statements - New: NAMES/CHARSET/CHARACTER SET recognized after commas - Updated deps/Makefile and makefiles_paths.mk for parsersql-1.0.1 - Added -lsqlparser to TAP test link flags (MYLIBS_STATIC_PART, LDIRS) - Updated setparser_test_common.h expectations: - Invalid subquery tests now produce extracted paren values (sql_mode 23-25) - SET with empty RHS skips invalid assignment, returns valid ones (Set1_v2 0) - Complex nested function calls now fully parsed (Set1_v2 1) - Syntax errors partially parsed (syntax_errors 0-1) - All 224 SET tests pass, all 51 digest tests pass	2 months ago
René Cannaò	bd40e3bc2b	Merge branch 'v3.0' into issue-5686-galera-vars	2 months ago
Rene Cannao	deb1c46173	fix: address code review findings C1-C3, I1, I3 ... C1: NODE_SET_CHARSET map key corrected to character_set_results C2: query_processor_parser==1 now also routes SET through ParserSQL C3: qp->digest initialized to 0 on parse failure I1: header docs corrected (tokenizer-based, not libinjection) I3: defensive NODE_SET_TRANSACTION case added to SET walker	2 months ago
Rene Cannao	482ce386cd	chore(genai-carveout): strip dead PROXYSQLGENAI blocks + plugin hardening ... Two threads of cleanup that had to land together because they touch both core (where the dead code lives) and the plugin (where the new module-owned state lives): # Strip dead #ifdef PROXYSQLGENAI from lib/, include/, src/ (Spec Q9) Step 7 of the carve-out left PSQLGA empty in lib/Makefile, so every #ifdef PROXYSQLGENAI block in the core build became permanently false. The blocks remained as dead text (visible to readers, irrelevant to the binary). Strip ~600 lines spanning: src/main.cpp 2 sites lib/Admin_Bootstrap.cpp 3 sites (mcpdb open + attach + flush_*_variables hooks) lib/Admin_FlushVariables.cpp 2 sites (~250 lines of stub + #if 0 _ORIGINAL bodies) lib/Admin_Handler.cpp 5 sites (LOAD/SAVE GENAI VARIABLES dispatch + LOAD/SAVE MCP VARIABLES dispatch + MCP profiles handler) lib/ProxySQL_Admin.cpp ~10 sites (init_*_variables + load_mcp_server + load/save_mcp_query_rules + GenericRefreshStatistics MCP probes) lib/ProxySQL_Admin_Stats.cpp 3 stats___mcp_* function bodies include/proxysql_admin.h 7 method declarations include/ProxySQL_Admin_Tables_Definitions.h 4 unused DDL macros The plugin now owns LOAD/SAVE MCP VARIABLES, LOAD/SAVE GENAI VARIABLES, LOAD/SAVE MCP PROFILES, LOAD/SAVE MCP QUERY RULES via its admin command registry; the chassis dispatcher catches them before any of the dead core branches would have fired. Caveat — known break: the `stats_mcp_*` tables are still registered (empty schema) but the populator path that fed them is gone. SELECTs return empty result sets until a follow-up wires the populator into the plugin (likely via register_runtime_view or a parallel stats-side ABI). Tracked separately. # Plugin hardening (review-feedback items 4-7) * plugins/genai/Makefile: add -fvisibility=hidden, -fvisibility-inlines-hidden, -fstack-protector-strong (mirrors plugins/mysqlx). Tag proxysql_plugin_descriptor_v1 with __attribute__((visibility("default"))) so the loader's dlsym still resolves; without the explicit annotation the hidden-default flag would also hide the entry point. * plugins/genai/src/Anomaly_Detector.cpp: convert genai_anomaly_embed_fn to std::atomic<…> with relaxed loads/stores. The detector hot-path read races with init/stop on the lifecycle thread; atomics give us a portable no-torn-pointer guarantee without paying for acquire/release fences we don't need (no other state synchronises with the pointer). Replaces the hand-wavy "no mutex needed" comment with the actual reasoning. * plugins/genai/src/{plugin_main,plugin_commands,plugin_tables}.cpp: route the 17 fprintf(stderr, "genai plugin: ...") log lines through a new genai_log() helper that calls services->log_message when the chassis is wired up and falls back to stderr for early-init / unit-test paths. Lines now land in proxysql.log alongside the rest of the runtime. * plugins/genai/include/MCP_Thread.h: add get_variable_string(name, std::string&) returning a std::string, and switch mcp_save_variables_to_admindb (plugin_main.cpp) to it. The legacy get_variable(name, char*) had an unbounded sprintf into a stack buffer and the SAVE path was passing char val[256] — a real overflow risk for *_endpoint_auth bearer tokens. * plugins/genai/src/plugin_main.cpp: rewrite the genai_init / genai_stop comment block with the actual teardown ordering rationale (consumers before producers; atomic-clear-then-delete for the embed hook; why Prometheus counters stay registered).	2 months ago
Rene Cannao	62ed0660a7	feat(plugins/genai): Step 7 — drop PROXYSQLGENAI from core build ... Last step of the GenAI plugin carve-out. The PROXYSQLGENAI macro is no longer compiled into core: lib/libproxysql.a and the proxysql binary build identically with or without `PROXYSQLGENAI=1`. The user-facing build flag stays — it now means "also build the genai plugin alongside proxysql" rather than "compile GenAI into the binary". What this commit does plugins/genai/Makefile Hardcodes -DPROXYSQLGENAI in the plugin's CXXFLAGS — the plugin IS the GenAI surface, so the macro is always-on inside its own compilation unit. Lets the existing `#ifdef PROXYSQLGENAI` guards inside plugin source files keep working without a massive guard-stripping pass (deferred follow-up cleanup). lib/Makefile, src/Makefile Drop the `PSQLGA := -DPROXYSQLGENAI` setter — core no longer sees the macro on the compile line. PSQLGA stays as an empty variable so the existing `$(PSQLGA)` cascades in CXXFLAGS references stay structurally identical (cleanup deferred). Also: lib/Makefile no longer compiles MCP_Tool_Handler.oo (the base class moved to the plugin in this commit too — see below). Top-level Makefile Plugin-build rules switched from `$(if $(filter 1,$(PROXYSQL40))` to `$(if $(filter 1,$(PROXYSQLGENAI))` — `PROXYSQL40=1` enables the chassis but not necessarily the genai plugin. Skip-message text updated to match. `PROXYSQLGENAI=1` still implies `PROXYSQL40=1` per the existing tier hierarchy, so the user invocation stays unchanged. MCP_Tool_Handler base class Moved to plugins/genai/ (`include/MCP_Tool_Handler.h` -> plugin include, `lib/MCP_Tool_Handler.cpp` -> plugin tool_handlers). Was the last GenAI-tier .cpp in core — only existed there because AI/RAG_Tool_Handler still inherited from it before Step 5. Now plugin-internal. lib/proxy_sqlite3_symbols.cpp Hook for sqlite_vec_init unconditional now (sqlite-vec extension object always linked into proxysql, since the genai plugin relies on it via this hook at dlopen time). lib/MySQL_Thread.cpp / lib/PgSQL_Thread.cpp `variables.ffto_enabled` default no longer flips to true when PROXYSQLGENAI is set (the gate was: PROXYSQLFFTO + PROXYSQLGENAI -> true; else false). With PROXYSQLGENAI removed, the default is false everywhere; operators opt in via admin SQL. CLAUDE.md Feature-tier table now lists Plugin Chassis (PROXYSQL40) and AI/MCP (PROXYSQLGENAI) separately, with a note explaining what PROXYSQLGENAI=1 means post-carve-out. Conditional Components section calls out that GenAI/MCP/RAG/LLM lives in plugins/genai/. What this commit does NOT do (deferred) - Stripping the `#ifdef PROXYSQLGENAI` guards from the plugin's own source files (LLM_Bridge.cpp, GenAI_Thread.cpp, etc. all have `#ifdef PROXYSQLGENAI ... #endif` brackets). The plugin's Makefile force-defines the macro so they keep expanding. A guard-stripping pass would be ~50 file edits with zero behaviour change. - Stripping the dead `#ifdef PROXYSQLGENAI` blocks from core (lib/Admin_Handler.cpp etc. still have FIXME-stubbed blocks inside `#ifdef PROXYSQLGENAI` from the 4.C / 5 surgery). With the macro undefined in core's compile line, those blocks silently expand to nothing — they're dead code already. Cleanup follow-up. - Removing the now-no-op stubs themselves (init_mcp_variables, flush_mcp_variables___*, load_mcp_server, stats___mcp_*, etc.). They linger as dead wrappers around early-returns; their declarations in proxysql_admin.h linger too. Cleanup follow-up. All three plugin unit tests pass: genai_plugin_load_unit-t : 25/25 genai_plugin_anomaly_unit-t : 6/6 genai_plugin_backend_client : 27/27 Carve-out scoreboard, FINAL: Step 0-2 : infrastructure (chassis cherry-pick, ABI extensions) Step 3 : ~1.1 K LOC Anomaly_Detector Step 4.A : ~1 K LOC GENAI: / LLM: query-prefix removal Step 4.B : ~0.5 K LOC backend_client helper Step 4.C : ~13 K LOC MCP subsystem + 8 tool handlers + MySQL_FTS Step 4.F : n/a runtime-reconfig admin SQL via plugin commands Step 4.G : n/a MCP admin/config/stats tables via plugin ABI Step 5 : ~7 K LOC GenAI/LLM/AI/RAG Step 6 : ~5 K LOC AI_Vector_Storage / Discovery_Schema / MySQL_Catalog / Static_Harvester pair Step 7 : n/a PROXYSQLGENAI macro dropped from core build Net core surface change: ~28 K LOC moved out of lib/ + include/ into plugins/genai/. src/proxysql binary shrunk from 197 MB (pre-carve-out, all PROXYSQLGENAI compiled in) to 152 MB; plugin .so grew to ~56 MB. Net process RSS roughly neutral.	2 months ago
Rene Cannao	4552236700	feat(plugins/genai): Step 6 — move RAG/Vector/discovery surface into the plugin ... Final file-move step in the carve-out: relocates the leaf modules that didn't depend on (and weren't depended on by) anything still in core. After this commit, the only PROXYSQLGENAI-guarded code remaining in libproxysql.a is `MCP_Tool_Handler` — the base class the plugin's AI_Tool_Handler / RAG_Tool_Handler inherit from — and that lives in core only because removing it would force pulling those derived plugin classes into core just for the inheritance edge. Step 7 deletes the PROXYSQLGENAI macro entirely. Files moved (git mv preserves history) Headers (include/ -> plugins/genai/include/) AI_Vector_Storage.h MySQL_Catalog.h Discovery_Schema.h Static_Harvester.h PgSQL_Static_Harvester.h Sources (lib/ -> plugins/genai/src/) AI_Vector_Storage.cpp MySQL_Catalog.cpp Discovery_Schema.cpp Static_Harvester.cpp PgSQL_Static_Harvester.cpp Core changes were minimal because these are leaf modules: include/proxysql.h: drop the AI_Vector_Storage.h include. include/cpp.h: drop AI_Vector_Storage / Discovery_Schema / MySQL_Catalog / Static_Harvester / PgSQL_Static_Harvester includes (replaced with a single carve-out history comment that Step 7 will delete entirely). lib/ProxySQL_Admin.cpp: drop the Discovery_Schema.h include — its only live use was via the now-#if-0'd save/load_mcp_query_rules blocks 4.C left behind for follow-up reference. lib/Makefile: drop AI_Vector_Storage.oo, MySQL_Catalog.oo, Discovery_Schema.oo, Static_Harvester.oo, PgSQL_Static_Harvester.oo from the GenAI _OBJ_CXX list. Test fixture probe update test/tap/tests/unit/Makefile: PROXYSQLGENAI auto-detect probe switched from `AI_Vector_Storage` (now plugin-only) to `MCP_Tool_Handler` (the last GenAI-tier symbol still in core). Step 7 deletes the auto-detect entirely. All three plugin unit tests pass: genai_plugin_load_unit-t : 25/25 genai_plugin_anomaly_unit-t : 6/6 genai_plugin_backend_client : 27/27 Binary size deltas: src/proxysql 159 MB -> 152 MB (-7 MB) plugins/genai/...Plugin.so 47 MB -> 55 MB (+8 MB) Net process RSS roughly neutral, as the design predicted. Carve-out scoreboard: Step 0-2 : infrastructure Step 3 : ~1.1 K LOC (Anomaly_Detector) Step 4.B : ~0.5 K LOC (backend_client) Step 4.C : ~13 K LOC (MCP subsystem + tool handlers + MySQL_FTS) Step 4.F : runtime-reconfig admin SQL routed through plugin commands Step 4.G : MCP admin / config / stats tables registered via plugin ABI Step 5 : ~7 K LOC (GenAI/LLM/AI/RAG) Step 6 : ~5 K LOC (AI_Vector_Storage + Discovery_Schema + MySQL_Catalog + Static_Harvester pair) Pending: Step 7 — delete the PROXYSQLGENAI macro from build flags and any remaining `#ifdef PROXYSQLGENAI` blocks.	2 months ago
Rene Cannao	28cf4ceab8	feat(plugins/genai): Step 5 — move GenAI/LLM/AI surface into the plugin ... Largest carve-out commit since 4.C: moves the GenAI / LLM / AI / RAG surface (about 7K LOC) out of core into plugins/genai/. After this commit, libproxysql.a no longer publishes any of GenAI_Threads_Handler, LLM_Bridge, LLM_Clients, AI_Features_Manager, AI_Tool_Handler, RAG_Tool_Handler, or the legacy GloGATH/GloAI globals. Files moved (git mv preserves history) Headers (include/ -> plugins/genai/include/) GenAI_Thread.h LLM_Bridge.h AI_Features_Manager.h AI_Tool_Handler.h RAG_Tool_Handler.h (promoted from Step 6 — bidirectional dep with AI_Features_Manager forced the early move, same way MySQL_FTS.h got promoted in 4.C) Sources (lib/ -> plugins/genai/src/ or …/tool_handlers/) GenAI_Thread.cpp LLM_Bridge.cpp LLM_Clients.cpp (.cpp without a header; class decls live in LLM_Bridge.h) AI_Features_Manager.cpp AI_Tool_Handler.cpp -> tool_handlers/ RAG_Tool_Handler.cpp -> tool_handlers/ Plugin lifecycle takeover plugins/genai/src/plugin_main.cpp: - Defines plugin-local GloGATH and GloAI alongside the existing GloMCPH (all three are .so-internal aliases for the contexts genai_init constructs). - genai_init now also constructs GenAI_Threads_Handler (calls ->init()) and AI_Features_Manager (calls ->init()), mirroring the pre-Step-5 ProxySQL_Main_init_GenAI_module sequence. - genai_stop tears them down in reverse order. plugins/genai/src/ProxySQL_MCP_Server.cpp: - AI / RAG endpoint registration re-enabled (was #if 0'd in 4.C because the constructors weren't reachable from the plugin then). /mcp/ai and /mcp/rag are once again live when the operator has GloAI configured. plugins/genai/src/Anomaly_Detector.cpp + plugin_main.cpp: - get_query_embedding() now calls through a function-pointer hook (genai_anomaly_embed_fn) that plugin_main installs in genai_init and clears in genai_stop. This indirection lets the test fixture (which compiles Anomaly_Detector.cpp directly into a test binary, without GenAI_Thread.cpp) link cleanly — the pointer stays null, the detector short-circuits to empty embedding, and the existing nullptr-guard in check_embedding_similarity does the rest. Core neutralization src/main.cpp: - Drops the GloGATH / GloAI globals, ProxySQL_Main_init_GenAI_module definition + call, the shutdown branch, and the `if (GloGATH) init_genai_variables()` call. lib/Admin_Bootstrap.cpp / Admin_FlushVariables.cpp / Admin_Handler.cpp / ProxySQL_Admin.cpp / MySQL_Session.cpp / ProxySQL_Admin_Stats.cpp: - Drop the corresponding `#include` and `extern` declarations. - flush_genai_variables___* stubbed (mirrors the 4.C MCP pattern; original bodies preserved in `#if 0` for the upcoming LOAD/SAVE GENAI verbs port). lib/Admin_Handler.cpp: - Loose-prefix accept for `genai-*` in is_valid_global_variable (mirrors the 4.F mcp-* fix), so SET genai-X=Y still reaches main.global_variables; runtime push happens via the plugin's LOAD GENAI VARIABLES TO RUNTIME (registered in a follow-up). include/cpp.h, include/proxysql.h: - Removed the AI_Features_Manager.h / LLM_Bridge.h / GenAI_Thread.h / AI_Tool_Handler.h / RAG_Tool_Handler.h includes that no longer resolve from core's -I path. test/tap/test_helpers/test_globals.cpp: - Drops the matching `#include`s and stub globals. Test fixture test/tap/tests/unit/Makefile: - PROXYSQLGENAI auto-detect probe switched from `GenAI_Thread` (now plugin-only) to `AI_Vector_Storage` (still in core for Step 6). After Step 6 the probe will switch again, and Step 7 deletes the auto-detect entirely. All three plugin unit tests pass: - genai_plugin_load_unit-t : 25/25 - genai_plugin_anomaly_unit-t : 6/6 - genai_plugin_backend_client : 27/27 Carve-out scoreboard: Step 0-2 : infrastructure Step 3 : ~1.1 K LOC (Anomaly_Detector) Step 4.B : ~0.5 K LOC (backend_client) Step 4.C : ~13 K LOC (MCP subsystem) Step 4.F : runtime-reconfig admin SQL routed through plugin commands Step 4.G : MCP admin / config / stats tables registered via plugin ABI Step 5 : ~7 K LOC (GenAI/LLM/AI/RAG) Pending: Step 6 (AI_Vector_Storage, MySQL_Catalog, Discovery_Schema, Static_Harvester, PgSQL_Static_Harvester) and Step 7 (delete the PROXYSQLGENAI macro entirely).	2 months ago
Rene Cannao	5fe286e972	feat(plugins/genai): Step 4.C — move MCP subsystem into the plugin ... Per docs/superpowers/plans/2026-04-19-step4-mcp-subsystem-move.md "4.C / 4.D / 4.E merge" revision. This is the largest single carve-out commit so far: ~13 K LOC physically relocated, plus surgical edits across the core admin SQL surface to neutralize references to the moved code. What moved (git mv preserves history) Tool handlers (lib/<name>.cpp -> plugins/genai/src/tool_handlers/<name>.cpp; include/<name>.h -> plugins/genai/include/<name>.h) Admin_Tool_Handler, Cache_Tool_Handler, Config_Tool_Handler, MySQL_Tool_Handler, Observe_Tool_Handler, Query_Tool_Handler, Stats_Tool_Handler. MCP listener stack (lib/<name>.cpp -> plugins/genai/src/<name>.cpp; include/<name>.h(pp) -> plugins/genai/include/...) MCP_Endpoint, MCP_Thread, ProxySQL_MCP_Server. Promoted from Step 6 (bidirectional dep with MySQL_Tool_Handler): MySQL_FTS. MCP_Tool_Handler base stays in core because AI_Tool_Handler and RAG_Tool_Handler (Steps 5/6 scope) inherit from it; moving the base would force pulling those in too. Plugin lifecycle takeover plugins/genai/include/genai_plugin.h: GenAIPluginContext now holds the MCP_Threads_Handler*. plugins/genai/src/plugin_main.cpp: genai_init constructs the MCP_Threads_Handler and assigns to a plugin-local GloMCPH symbol (the legacy global name, kept for the moved tool handlers that still reference it; the symbol no longer exists in core). genai_stop tears down on shutdown. genai_start is a no-op for now; the MCP listener auto-start is gated on the admin-SQL surface that 4.F restores. plugins/genai/src/ProxySQL_MCP_Server.cpp: AI_Tool_Handler and RAG_Tool_Handler endpoint registration is disabled via stub assignments (the headers stay included so member-pointer types resolve). Steps 5/6 re-enable when those classes move into the plugin. Core neutralization (FIXME stubs awaiting 4.F/4.G) src/main.cpp: GloMCPH global removed; ProxySQL_Main_init_MCP_module removed; shutdown block deleted. lib/ProxySQL_Admin.cpp: init_mcp_variables, load_mcp_server, load_mcp_query_rules_to_runtime, save_mcp_query_rules_from_runtime stubbed to no-ops; original bodies preserved inside `#if 0` for 4.F reference. lib/Admin_FlushVariables.cpp: flush_mcp_variables___database_to_runtime and flush_mcp_variables___runtime_to_database stubbed similarly. lib/ProxySQL_Admin_Stats.cpp: stats___mcp_query_tools_counters, stats___mcp_query_digest, stats___mcp_query_rules stubbed. lib/Admin_Handler.cpp: has_variable("mcp-…") and the load_target_auth_map call site in LOAD MCP PROFILES … stubbed. Result: the MCP listener does NOT start automatically between this commit and 4.F (mcp-* admin variables are not pushed into the running plugin). Documented in the plan's "4.C/4.D/4.E merge" section. src link change: --whole-archive on libproxysql.a src/Makefile: the proxysql binary now links libproxysql.a with -Wl,--whole-archive so all symbols are exported via -Wl,--export-dynamic, allowing the genai plugin .so to dlopen cleanly even when core code paths don't reference moved-but-still- in-core classes (LLM_Bridge, Discovery_Schema, MySQL_Catalog, Static_Harvester, …) directly. test/tap/tests/unit/Makefile gets the same change so unit tests can dlopen the plugin. Without this, ld dead-code-eliminates the unreferenced symbols and dlopen fails with "undefined symbol". Tests passing - genai_plugin_anomaly_unit-t : 6/6 - genai_plugin_load_unit-t : 8/8 (lifecycle init/start/stop + admin table registration) - genai_plugin_backend_client_unit-t : 27/27 Carve-out scoreboard Step 0-2 : 0 (infrastructure only) Step 3 : ~1.1 K (Anomaly_Detector) Step 4.B : ~0.5 K (backend_client helper, no callers yet) Step 4.C : ~13 K (this commit; MCP subsystem) Pending : Steps 4.F, 4.G, 5, 6, 7	2 months ago
Rene Cannao	2b781ffbfd	feat(session): Step 4.A — remove the GENAI:/LLM: query-prefix escape hatches ... Per decision Q2 in the GenAI plugin carve-out design doc, the in-line MySQL-protocol prefixes that bypassed routing, ACLs, and the query processor are removed. Users reach GenAI features through MCP, admin SQL, or the REST endpoint -- as the rest of the carve-out finishes, nothing else needs the in-core GENAI:/LLM: handlers. Removals: lib/MySQL_Session.cpp (-933 lines) - handler___status_WAITING_CLIENT_DATA___STATE_SLEEP___MYSQL_COM_QUERY___genai - handler___status_WAITING_CLIENT_DATA___STATE_SLEEP___MYSQL_COM_QUERY___llm - handler___status_WAITING_CLIENT_DATA___STATE_SLEEP___genai_send_async - handler___status_WAITING_CLIENT_DATA___STATE_SLEEP___handle_genai_response - genai_cleanup_request - check_genai_events - The COM_QUERY prefix-detection block that dispatched to those handlers - The check_genai_events poll-loop hook in WAITING_CLIENT_DATA include/MySQL_Session.h (-67 lines) - All matching method declarations, with their doxygen include/Base_Session.h (-15 lines) - GenAI_PendingRequest struct - pending_genai_requests_ map - next_genai_request_id_ counter - genai_epoll_fd_ per-session epoll fd lib/Base_Session.cpp (-9 lines) - The init block that created the per-session genai_epoll_fd_ LLM: was deleted alongside GENAI: even though only GENAI: is named in the design. They share the same `#ifdef PROXYSQLGENAI` gate, the same async-genai socketpair plumbing, and the same rationale. Leaving LLM: behind would have left a half-block referencing functions that no longer exist; if we want LLM: back later the plugin can re-implement it (or a more general request-multiplexer ABI extension can land). The async-genai socketpair protocol (GenAI_RequestHeader / GenAI_ResponseHeader, defined in include/GenAI_Thread.h) is now unreferenced from outside lib/GenAI_Thread.cpp itself; those structs disappear when GenAI_Thread moves to plugins/genai/ in Step 5. Verified: - clean rebuild of libproxysql.a + proxysql + ProxySQL_GenAI_Plugin.so - all 60 unit-test binaries pass - smoke test: daemon starts, SELECT through admin works, SIGTERM shuts down cleanly Net diff: -1010 / +28 lines. Largest deletion of the carve-out so far.	2 months ago
Rene Cannao	f1c0ea2dbc	feat(genai): Step 3 — carve Anomaly_Detector out of core into plugins/genai/ ... Per docs/superpowers/specs/2026-04-16-genai-plugin-carveout-design.md "Migration sequence — Step 3", this is the first step that actually moves GenAI surface out of libproxysql.a. The Anomaly_Detector now lives entirely inside the genai plugin and is exercised through the plugin query-hook ABI added in Step 2; core has zero references to it. Files moved (git mv preserves history): include/Anomaly_Detector.h -> plugins/genai/include/Anomaly_Detector.h lib/Anomaly_Detector.cpp -> plugins/genai/src/Anomaly_Detector.cpp Plugin-side adjustments: - Drop the `#ifdef PROXYSQLGENAI` guard from both files (the file only compiles inside the plugin now; the macro remains a core build flag but the carve-out gates these files by *location*, not by macro). - Update the json-include relative path to the new depth. - Replace `Anomaly_Detector::get_query_embedding` with a stub that returns an empty vector. The previous implementation called `GloGATH->embed_documents`, but `GloGATH` (the GenAI_Threads_Handler) is still a core symbol at this point in the carve-out. The embedding-similarity stage was already nullptr-guarded on `vector_db`, so an empty embedding short-circuits cleanly to "no anomaly". Step 5 reattaches the embedding back-end when GenAI_Thread moves into the plugin. Plugin lifecycle wiring (plugins/genai/src/plugin_main.cpp): - `genai_init` now: registers two Prometheus counters (`proxysql_genai_detected_anomalies_total`, `proxysql_genai_blocked_queries_total`) against the shared registry exposed by Step 2.3, registers the query hook for both MySQL and PgSQL protocols, and constructs the Anomaly_Detector. - `genai_start`: flips the started flag (the detector has no separate start phase). - `genai_stop`: closes and deletes the detector; counters stay registered (prometheus-cpp has no Unregister). Plugin query-hook adapter (plugins/genai/src/plugin_hooks.cpp, NEW): - Bridges ProxySQL_PluginQueryHookPayload to Anomaly_Detector::analyze. - Increments the appropriate Prometheus counter on detection / block. - Translates AnomalyResult into ALLOW or DENY(message) per the ABI. Core removals (the symmetric carve-out): - lib/MySQL_Session.cpp: delete the in-core `#ifdef PROXYSQLGENAI` anomaly-detection block in COM_QUERY (lines around 5370-5377 pre-edit) and the `handler___status_WAITING_CLIENT_DATA___STATE_SLEEP___MYSQL_COM_QUERY_detect_ai_anomaly` method. Same logic now runs inside the plugin via the query hook added in Step 2.2. - include/MySQL_Session.h: drop the matching method declaration. - lib/AI_Features_Manager.{cpp,h}: remove the anomaly_detector field, init_anomaly_detector(), close_anomaly_detector(), get_anomaly_detector(), and the corresponding init() / shutdown() call sites. AI_Features_Manager now owns only the LLM bridge and vector_db (LLM_Bridge moves in Step 5; vector_db in Step 6). - lib/AI_Tool_Handler.{cpp,h}: drop the anomaly_detector field and the constructor parameter that fed it -- the field was stored but never read, dead code that was awkward to keep alive across the carve-out. - lib/ProxySQL_MCP_Server.cpp: update the AI_Tool_Handler ctor call site accordingly (one less argument). - include/MySQL_Thread.h: remove `st_var_ai_detected_anomalies` / `st_var_ai_blocked_queries` from `enum status_variable_id` and `ai_detected_anomalies` / `ai_blocked_queries` from `enum p_th_counter`. The plugin's Prometheus counters replace these (the old ones were per-thread integer increments surfaced by the legacy stats path; the plugin uses Prometheus directly). - lib/MySQL_Thread.cpp: drop the matching entries from MySQL_Thread_status_variables_counter_array and from th_metrics_map (the prometheus registration table). - include/proxysql.h, include/cpp.h: drop the now-stale `#include "Anomaly_Detector.h"` lines. - lib/Makefile: drop Anomaly_Detector.oo from the libproxysql object list. Tests: - Delete the in-core TAP tests that linked against the removed symbols: anomaly_detection-t, anomaly_detection_integration-t, anomaly_detector_unit-t (Docker-dependent, won't link), plus the unit test genai_anomaly_unit-t (linked against libproxysql's copy). - Add test/tap/tests/unit/genai_plugin_anomaly_unit-t.cpp: replaces the deleted unit test by compiling the plugin's Anomaly_Detector.cpp directly into the test binary. The friend-class helper pattern (Anomaly_Detector_TestHelper) is preserved so private methods (normalize_query, check_sql_injection) remain testable. Six assertions cover the same surface plus a regression test that analyze() now runs cleanly without a vector_db. Plugin Makefile changes: - Add Anomaly_Detector.cpp + plugin_hooks.cpp to SRCS. - Pull in the include paths Anomaly_Detector.cpp's transitively- included headers need (mariadb client for mysql.h, libev, libscram, libusual, postgresql, jemalloc, microhttpd, libhttpserver, curl, re2, pcre, zstd, libconfig, libdaemon, ssl). These mirror the set already used by test/tap/tests/unit -- the test harness pulls the same headers in transitively from libproxysql.a's headers. Doxygen documentation added for every new function and for the new header structures (GenAIPluginContext fields, genai_query_hook, register_prometheus_counters, all three lifecycle callbacks, plus file-level @brief blocks for plugin_main.cpp, plugin_hooks.cpp, genai_plugin.h, and the moved Anomaly_Detector.{h,cpp}). Verified end-to-end: - Full clean rebuild of libproxysql.a + proxysql + ProxySQL_GenAI_Plugin.so. - All 60 unit-test binaries pass (genai_anomaly_unit-t replaced by genai_plugin_anomaly_unit-t; net same). - Smoke test: launching proxysql with plugins=("...ProxySQL_GenAI_Plugin.so") shows "Anomaly: Initializing Anomaly Detector v0.1.0" / "Anomaly Detector initialized with 11 injection patterns" in the log on startup, and "Anomaly Detector closed" on SIGTERM -- the plugin's lifecycle owns the detector cleanly. Carve-out scoreboard so far (lines of GenAI surface moved out of core): Step 0-2 : 0 (infrastructure prep only) Step 3 : ~1100 (Anomaly_Detector.{h,cpp}) + counter/handler removals	2 months ago
Rene Cannao	62491629bb	docs: add documentation to ParserSQL adapter layer	2 months ago
Rene Cannao	6099d06c1e	ci(tsan): address review findings on PR #5725 ... Two small follow-ups from the CodeRabbit + Gemini review: * include/makefiles_vars.mk — emit a $(warning ...) for vm.mmap_rnd_bits=28 from the WITHTSAN=1 branch, mirroring the existing ASAN warning. Both sanitizers need the same 28-bit ASLR ceiling and the original commit's claim that "the warning above already covers both" was wrong: the ASAN warning only fires under WITHASAN=1, so a TSAN-only build was running silent. * .github/workflows/CI-unit-tests-tsan.yml — fix the compose project-name transform to match the Makefile. The Makefile's binaries/proxysql% rule uses `${GIT_VERSION/./}` (strip first dot only); the workflow was using `${GIT_VERSION//./}` (strip all dots), which put `docker compose run` into a different project namespace than `make ubuntu24-tap` had declared. In practice the Makefile tears down its project before exiting so the visible effect was just a redundant fresh network/volume on the run step, but the build-chain contract is "same project name throughout" and we should honour it. Comment block updated so it stops claiming the two transforms are the same operation.	2 months ago
Rene Cannao	90b1897764	feat: implement ParserSQL adapter layer	2 months ago