239 Commits (ee6d4e506ac47a9ef69b60efa5ddfc9cde2cd71d)

Author	SHA1	Message	Date
Rene Cannao	57c3e3fa28	docs: excise sqlite-rembed sections from GENAI.md and SQLite3-Server.md ... - GENAI.md: Remove two rembed-related links from Related Documentation section - SQLite3-Server.md: Delete Embedding Generation section (was lines 72-103) - SQLite3-Server.md: Remove rembed-dependent use cases (embedding generation, AI pipelines)	2 months ago
Rene Cannao	f821ebcdc7	docs: remove sqlite-rembed guides, examples, and embeddings script	2 months ago
Rene Cannao	81b406a428	doc(GH-Actions): address gemini review nits on §12 vocabulary section ... Four minor corrections from the PR #5607 review: 1. §12 intro: "from issue" → "from the issue" (missing article) 2. §12.1.1: comma splice "(... — ProxySQL does exactly this, see #7 below)" → semicolon for grammatical correctness 3. §12.1.1 YAML example: replaced the `← ← ←` arrow annotations with `# <- ...` comments so the snippet is valid YAML if copy-pasted 4. §12.1.1: "has runs on 10 commits" → "has run on 10 commits" (past participle) No change to meaning or structure. The `←` arrows that remain in the ASCII box diagrams at §12.2 / §12.3 / §12.4 are untouched - those diagrams are illustrative text inside un-typed fenced blocks, not copy-pasteable code, and the gemini review only flagged the yaml-typed block. Coderabbit's "Critical" finding about anchor links using `--` instead of `-` is a false alarm: the em-dash in the §12 heading is stripped by GitHub's slugger, leaving two adjacent spaces which become two adjacent hyphens. I verified this by asking GitHub's own `/markdown` API to render the heading and inspecting the generated `<a id="...">`, which contains exactly `...vocabulary--read-this-first-if-confused` (double hyphen). No anchor fix is needed.	2 months ago
Rene Cannao	90c103a345	doc(GH-Actions): add §12.6 terminal flow — how to actually see what ran ... Follow-up to the vocabulary section added in ca4da932e. A maintainer hit GitHub's longstanding check-run UX papercut: clicking a row in the PR "Checks" tab lands on /runs/<check_id>, which shows only a status card and whose "View more details on GitHub Actions" link often points back at the same page because details_url on auto-created check runs is self-referential. The same dead-end is reached via the PR checks table's "Details" button (an earlier draft of this doc claimed otherwise — the claim was wrong and is corrected here). This commit inserts §12.6 "Seeing what actually ran — the terminal flow" between §12.5 (Common confusions) and what is now §12.7 (Sanity check, formerly §12.6). The new section walks the exact four-command terminal flow needed to go from a single row of `gh pr checks` output to the log lines of the failing step, using the real row from PR 5596 as the walked example: ✓ CI-trigger/CI-legacy-g1 / tests (mysql57) → gh run list -R sysown/proxysql --workflow CI-legacy-g1 → gh run view 24281031512 -R sysown/proxysql → gh run view --log-failed --job=70902846188 -R sysown/proxysql The section covers: - A character-by-character breakdown of the check row, identifying which fields are useful (workflow name) and which are dead ends (the /runs/<id> URL). - Why `gh run list`'s `branch:` column lies ("v3.0" rather than the PR's actual branch) because workflow_run-triggered runs inherit the default branch's metadata, not the triggering PR's. The documented gotcha in §10.2 cross-referenced. - How the run's display-title is the ONLY place where the PR head SHA appears, and the idiom to grep / `jq .displayTitle | contains(...)` against it. - How job name and check-run name differ ("run / tests (mysql57)" vs "CI-legacy-g1 / tests (mysql57)") because the prefixes come from different namespaces (caller job key vs workflow name). - Three flavors of log retrieval (--log-failed, --log | less, full-run --log) with concrete awk / grep idioms for the tab-separated log format. - A copy-pasteable condensed cheat sheet parameterised over PR number and workflow name. - A "why the web UI cannot do this" explanation stacking the three separate GitHub design issues: check-runs vs job-runs are different objects with no explicit link, details_url is self-referential, and workflow_run runs are tagged with the default branch's SHA not the PR's. Also: - §12.7 Sanity-check renumbered from §12.6, internal cross-refs in its "if you didn't get this, re-read X" pointer updated. - §12.7 gains a sixth self-test question covering the terminal flow. - No TOC change needed: the TOC lists only top-level sections (§12 as a whole), subsections are reached via the heading anchors. Doc-only change. 1547 → 1818 lines. Code fences balanced at 100.	2 months ago
Rene Cannao	ca4da932ef	doc(GH-Actions): add detailed GitHub Actions vocabulary section with diagrams ... The existing CI architecture doc assumes the reader already knows the GitHub Actions terminology and can parse check-run labels like 'CI-maketest / builds (testgalera)'. A maintainer asked what that label means and where to find it by grep, and the only answer was "re-read the whole doc and work it out yourself, most of the pieces are in different sections". That's not good enough. This commit adds a new §12 'Understanding GitHub Actions vocabulary' that: - Walks through the seven core terms (workflow, workflow run, job, matrix cell, step, check run, caller/reusable split) bottom-up, using ci-maketest.yml as a single concrete example throughout. - Includes an ASCII nesting diagram that shows how workflow -> run -> job -> matrix cell -> steps slot into each other, with check runs as separate objects attached to the commit SHA. - Includes a second ASCII diagram showing the ProxySQL two-branch caller/reusable split: one logical trigger -> two workflow runs, one on v3.0 and one on GH-Actions, sharing the same workflow name. - Includes a third diagram tracing exactly how the check-run label 'CI-maketest / builds (testgalera)' is assembled at runtime from github.workflow + github.job + env.MATRIX, explaining why the literal string exists nowhere on disk and grepping for it or for 'galera' will fail. - Answers five common confusion questions directly (why click-throughs land on GH-Actions, why two check rows per cell, where 'make testgalera' is defined, which file produced a given check row, CI-legacy-g2-genai vs CI-legacy-g2). - Ends with a five-question self-test so the reader can check they have the model right before closing the section. Also: - Added a 'New to GitHub Actions terminology?' pointer at the top of the doc so a confused reader jumps directly to the new section instead of wading through the two-branch architecture explanation first. - Renamed the existing compact Glossary table to 'Glossary (quick reference)' to make clear it is the lookup-by-term reference and the new narrative section is the learn-by-walkthrough explanation. - Both sections are kept: §12 for people who need the model, §13 for people who just need a word defined quickly. No source-code or workflow changes. Doc only.	2 months ago
Rene Cannao	22e0ec2c91	docs(ci): document matrix.infradb gotcha; tag fenced blocks with text ... Addresses review feedback on #5598: 1. Adds `matrix.infradb` to the "What a sibling differs in" checklist. Gemini-code-assist flagged it -- and it was also a real bug in the merged ci-mysql84-g{1..5}.yml files, fixed in the separate #5600. The doc now explains that `infradb` is cosmetic for routing but displayed in the PR UI via env.MATRIX, so leaving it at the template default creates a misleading "CI-mysql84-g3 / tests (mysql57)" label. 2. Updates the sed example in "Adding a new test group" to substitute the infradb string alongside the group name, so copy-pasting from the doc produces a correct file on the first try. 3. Adds `text` language tag to seven previously untyped fenced code blocks (ASCII diagrams, directory listings, plain-text output) to satisfy markdownlint MD040. CodeRabbit flagged these. The repo doesn't enforce markdownlint, but the tag is harmless and silences the bot for future contributors editing the doc.	2 months ago
Rene Cannao	5aad576125	ci: rewire group callers to dedicated reusables; add CI-unittests; document CI architecture ... Rewiring -------- Following the landing of sysown/proxysql#5597 on the GH-Actions branch (which adds the nine dedicated reusable workflows), point each of the eight existing group callers at its dedicated reusable instead of the broken generic ci-taptests-groups.yml: CI-legacy-g1.yml → ci-legacy-g1.yml@GH-Actions CI-legacy-g3.yml → ci-legacy-g3.yml@GH-Actions CI-legacy-g5.yml → ci-legacy-g5.yml@GH-Actions CI-mysql84-g1.yml → ci-mysql84-g1.yml@GH-Actions CI-mysql84-g2.yml → ci-mysql84-g2.yml@GH-Actions CI-mysql84-g3.yml → ci-mysql84-g3.yml@GH-Actions CI-mysql84-g4.yml → ci-mysql84-g4.yml@GH-Actions CI-mysql84-g5.yml → ci-mysql84-g5.yml@GH-Actions Each edit also drops the now-useless `testgroup: '["<group>"]'` input that the new dedicated reusables do not accept (and the old generic reusable silently ignored). This is the other half of the fix started in #5597. Before this change, the eight group workflows were routed to ci-taptests-groups.yml, whose tests job strategy evaluates from a cached proxysql/tap-matrix.json that CI-builds has been populating as "[ ]". With an empty matrix vector GitHub Actions rejects the strategy with "Matrix vector 'testgroup' does not contain any values" and the tests job never instantiates -- visible for example in run 24240099115. With this PR merged (after #5597), these eight workflows will route to their own dedicated reusables and start running real tests on the next push. CI-unittests caller ------------------- Adds a new v3.0 caller CI-unittests.yml targeting ci-unittests.yml@GH-Actions (also added in #5597). This runs the 42 unit-tests-g1 binaries listed in test/tap/groups/groups.json directly on the runner, via the SKIP_PROXYSQL=1 host-only branch in run-tests-isolated.bash -- no Docker, no backends. Documentation ------------- Replaces doc/GH-Actions/README.md with an expanded, current CI architecture reference (~800 lines). Covers: * The two-branch caller/reusable split and the workflow_run default-branch rule that forces it * The CI-trigger / CI-builds babysitter pattern and why test workflows chain on CI-trigger completion (and not on CI-builds directly -- see 78b8f5ac6 for the incident that taught us why) * The dedicated-reusable pattern, with ci-legacy-g4.yml walked through step by step as the canonical template * Build cache layout (_bin / _src / _test / _matrix) * The TAP groups system: groups.json, BASE_GROUP stripping, infras.lst, env.sh, SKIP_PROXYSQL mode for unit tests * Full workflow catalogue as of 2026-04-11 with status per file * End-to-end walkthrough for adding a new test group * Common pitfalls: merge ordering, workflow_run head_sha propagation, the empty-matrix wedge, the LouisBrunner/checks-action false-positive permissions hotspot * Debugging guide: how to trace a failing run through CI-trigger → CI-builds → test workflow, and how to reproduce locally * Glossary Adds a Mermaid sequence diagram for the execution flow and inline ASCII diagrams throughout. Also updates CLAUDE.md to point agents at doc/GH-Actions/README.md before they touch anything under .github/workflows/.	2 months ago
Rene Cannao	80ff2242e6	merge: resolve v3.0 conflict and add pgsql_servers_ssl_params cluster sync ... v3.0 added pgsql_servers_ssl_params table. Integrate into pgsql cluster sync path: - Add CLUSTER_QUERY_PGSQL_SERVERS_SSL_PARAMS to Admin_Handler interception block alongside other pgsql cluster queries - Add ssl_params fetch query to pull_pgsql_servers_v2_from_peer - Add update_pgsql_servers_ssl_params() for applying synced data - Update convert_pgsql_servers_resultsets to handle 5 result sets - Include ssl_params in v2 checksum computation (4 tables)	2 months ago
Rene Cannao	822f4aaa6e	doc: remove stale pgsql cluster sync PR status document ... The pgsql_cluster_sync_pr5297_status.md was written during development and is now outdated — the TAP test passes fully with cluster sync validation including disk persistence.	2 months ago
Rene Cannao	47758a468d	merge: resolve groups.json conflict with v3.0 ... Take v3.0's groups.json as base (legacy-gN naming, mysql84-gN groups, new test entries) and add test_cluster_sync_pgsql-t from this branch. Drop duplicate pgsql_admin_metacmds-t (already exists as pgsql-admin_metacmds-t in v3.0).	2 months ago
Rahim Kanji	50221943a3	Fix numbering in ssl_keylog user guide troubleshooting section (#5281)	2 months ago
Rahim Kanji	f2bd222145	Fixed compilation issue	2 months ago
Rahim Kanji	e7ee68dda5	Add SSL/TLS traffic decryption for PostgreSQL backend connections (#5281) ... Extends the existing NSS keylog-based SSL decryption to PostgreSQL backend connections by patching libpq with a global PQsetSSLKeyLogCallback() API. When admin-ssl_keylog_file is configured, TLS secrets from all PgSQL backend SSL handshakes are written to the keylog file in NSS Key Log Format, enabling traffic decryption with Wireshark/tshark. Approach: - Patch libpq with global callback following the PQsslKeyPassHook pattern - Single startup call covers all backend paths (regular, monitor, kill, harvester) - No new admin variables — existing admin-ssl_keylog_file covers PgSQL backends - Zero changes to ProxySQL PgSQL connection code Includes 9 integration tests (pgsql-ssl_keylog-t) and user guide updates.	2 months ago
Rene Cannao	bc42730061	Add GitHub Actions CI/CD documentation ... Comprehensive documentation covering: - Two-tier architecture (caller + reusable workflows) - Execution flow and trigger cascade - Build matrix (debian12-dbg, ubuntu22-tap, ubuntu24-tap-genai-gcov) - Cache strategy and key naming - All 25 workflows with status, purpose, and dependencies - Third-party integration testing (10 workflows) - Test infrastructure pipeline (ensure-infras → run-tests → cleanup) - TAP group system and hooks - Guide for adding new test workflows Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2 months ago
René Cannaò	586fb734f8	Move git workflow to top of task template — agents skip buried instructions ... Learned from agent post-mortem: agents prioritize technical content over administrative instructions. Even when branch info is present, agents skim past it while focusing on code requirements, then use heuristics to fill the gap they don't realize they have. Fix: git workflow is now the FIRST section with "do this before reading anything else" label. Also updates common-mistakes.md with the expanded root cause analysis.	2 months ago
René Cannaò	c54e6e1805	Expand task assignment template with learnings from CodeRabbit plans ... Adds four new checklist items based on analyzing CodeRabbit's auto- generated plans for issues #5491 and #5492: - #9: Write ready-made prompts for executing agents — pre-digest the task into imperative, sequential, agent-optimized format - #10: Document design decisions with rationale — prevent agents from re-litigating settled architecture choices - #11: Show the research — make codebase analysis visible for verification and context transfer - #12: For refactoring tasks, specify what to replace — exact line ranges, replacement calls, include dependencies Also adds two new entries to the common mistakes table: - "Recreates existing infrastructure" (research contradicts plan) - "Uses wrong infrastructure" (multiple similar patterns exist) Updates the good-vs-bad example to demonstrate the new items.	2 months ago
René Cannaò	bf250c6307	Add agent guidelines documentation ... Adds doc/agents/ with three guidance documents for AI coding agents, based on lessons learned from Milestone 2-3 unit testing work: - project-conventions.md: ProxySQL-specific rules — directory layout, build system, unit test harness usage, header conventions, circular include dependency warnings, git workflow, dual-protocol coverage - task-assignment-template.md: Generic template and orchestrator checklist for writing GitHub issues that agents can execute correctly. Covers: specifying WHERE/HOW/environment, DO NOT lists, binary acceptance criteria, and the good-vs-bad issue example - common-mistakes.md: Catalog of 9 observed failure patterns with root cause analysis, prevention strategies, and shell commands for detection in agent PRs Also updates CLAUDE.md with a reference to the agent guidelines and a note about the unit test harness.	2 months ago
Rene Cannao	859c89b30e	cluster: harden PostgreSQL cluster sync parity paths after real two-node validation ... This commit turns the current working branch follow-up into one coherent changeset after reviewing the PostgreSQL cluster sync implementation against real multi-node behavior instead of relying only on checksum presence and basic TAP accessibility checks. The main implementation change is that PostgreSQL cluster sync now updates the replica admin-memory tables in the places where the previous branch state was still weaker than the established MySQL behavior. In the PostgreSQL servers path, the replica now rewrites pgsql_servers, pgsql_replication_hostgroups, and pgsql_hostgroup_attributes before promoting the fetched state into runtime. That closes the most important parity gap that showed up during replica-side validation, because the cluster sync operation no longer leaves runtime state correct while replica admin-memory tables lag behind. In the PostgreSQL query rules path, the fetched query-rules and fast-routing resultsets are now hashed using the same converted representation that runtime loading consumes, then passed directly into load_pgsql_query_rules_to_runtime(), and finally written back from runtime into the replica admin-memory tables. In the PostgreSQL users path, the branch now relies on PostgreSQL-specific runtime checksum logic and persists accepted rows into the replica pgsql_users admin-memory table. This commit also fixes a real correctness bug that was discovered only after running a true two-node manual validation flow. The refactored module table in ProxySQL_Cluster::set_checksums() passed the full module names "pgsql_variables" and "ldap_variables" into pull_global_variables_from_peer(), but that shared helper only accepts the short dispatch keys "pgsql" and "ldap". In practice, changing a PostgreSQL variable on the primary caused the replica to abort on an assertion as soon as the cluster monitor attempted the variable pull. This commit restores the correct short dispatch keys so the PostgreSQL and LDAP variable sync paths actually execute instead of crashing. On the TAP side, the PostgreSQL cluster sync test is strengthened so it checks what the implementation is supposed to guarantee rather than what the earlier test happened to probe. The server sync test now mutates the real pgsql_servers admin table, then validates replica runtime_pgsql_servers, replica pgsql_servers, and disk.pgsql_servers when save_to_disk is enabled. The users sync test now validates runtime, main, and optional disk state on the replica. The query-rules sync test now validates both query rules and fast-routing rows across runtime, main, and optional disk persistence. The polling helpers were also hardened to support fresh replica admin sessions, because during local validation a long-lived admin session did not reliably observe synced rows even though fresh sessions and the manual harness did. The test now asserts the externally observable replica state seen by new admin clients. The documentation is updated to reflect the branch as it actually exists after this review pass. The PostgreSQL cluster sync status document now records the remaining non-CI TODO, the module-by-module behavior, the real two-node test plan, the outcomes for save_to_disk=false and save_to_disk=true, the variable sync dispatcher crash that was found and fixed locally, and the current MySQL parity assessment. This makes the branch easier to continue without having to reconstruct the reasoning from terminal logs or partial memory. Validation performed for this commit: - git diff --check - make build_src -j2 - PostgreSQL cluster-sync TAP two-node validation with save_to_disk=false - PostgreSQL cluster-sync TAP two-node validation with save_to_disk=true - manual two-node validation of pgsql_servers, pgsql_replication_hostgroups, pgsql_hostgroup_attributes, and a real PostgreSQL variable with save_to_disk=false - manual two-node validation of the same paths with save_to_disk=true Residual caveat intentionally documented but not treated as a blocker in this commit: - a long-lived replica admin session did not reliably observe synced PostgreSQL rows during polling, while fresh admin sessions and the manual validation harness did observe the expected state - the underlying cluster-sync apply path now validates correctly in real two-node runs, but the admin-session visibility quirk should still be understood better before calling the branch completely finished	2 months ago
Rene Cannao	c2e383701d	doc: add non-CI todo for pgsql cluster sync	2 months ago
Rene Cannao	d702e7d3e9	doc: summarize pgsql cluster sync branch status	2 months ago
Rene Cannao	c98f544db5	Enhance infrastructure stability and dynamic configuration ... - Implement clean slate logic in docker-compose-init.bash (stop containers + wipe data dirs). - Dynamically patch Orchestrator configurations with INFRA_ID-derived ROOT_PASSWORD. - Ensure all literal placeholders are preserved across all infrastructures. - Fix permission issues for PostgreSQL and MySQL log directories. - Verified successful infra-mysql57 initialization and ProxySQL registration.	3 months ago
Rene Cannao	70647fb426	fix: Remove unused legacy mcp-mysql_* variables ... These variables were deprecated in favor of profile-based routing but were never removed from the code. They existed in the struct definition, getters, and setters but were not consumed anywhere - Query_Tool_Handler uses target_auth_map via mcp_auth_profiles and mcp_target_profiles. Removed: - mcp-mysql_hosts, mcp-mysql_ports, mcp-mysql_user, mcp-mysql_password, mcp-mysql_schema from MCP_Thread.h struct - Variable names from mcp_thread_variables_names array - Initialization, cleanup, getters, and setters in MCP_Thread.cpp - Documentation in doc/MCP/VARIABLES.md	3 months ago
Rene Cannao	e53d2c76f9	Address PR #5410 review findings across FFTO, sessions, TAP, and docs ... Apply actionable bot review feedback from PR #5410 without behavior drift:\n- fix MySQL/PgSQL FFTO digest normalization and max-length hashing\n- add early FFTO packet/message size guards for bypass safety\n- restore MySQL session GTID reset and harden GenAI async error ownership\n- add null-check for stripped KILL query normalization path\n- call FFTO on_close() before observer reset/bypass in session flows\n- fix processlist query-options default pagination limit semantics\n- fix TAP noise_failures locking race and noise-aware plan count\n- extend wait_get_enpoint_ready with optional basic-auth passthrough\n- update MCP architecture docs from observe->stats naming\n- clean corrupted character in pgsql query-cache test comment\n\nValidation run:\n- git diff --check\n- make -C test/tap/tap tap.o utils_mariadb.o utils_mysql57.o utils_mysql8.o -j4\n- make -C lib obj/MySQLFFTO.oo obj/PgSQLFFTO.oo obj/MySQL_Session.oo obj/PgSQL_Session.oo obj/Query_Processor.oo -j4	3 months ago
Rene Cannao	79c0b383e8	integration: fix pre-CI issues in unified branch	3 months ago
Rene Cannao	acef17a8c1	Merge branch 'v3.0-ff_inspect' into v3.0-unified-large-pr ... # Conflicts: # test/tap/tests/Makefile	3 months ago
Rene Cannao	e2ebc0c449	Merge branch 'v4.0-mcp-stats2' into v3.0-unified-large-pr	3 months ago
Rene Cannao	90462f6d5a	ffto: harden MySQL/PgSQL observer state handling and align tests/docs ... Address outstanding review findings for FFTO on v3.0-ff_inspect and tighten protocol-state correctness for both engines. MySQL FFTO - Restrict on_close() reporting to true in-flight states and always clear query tracking state after close. - Add explicit active-query cleanup helpers and invoke them on state transitions to IDLE. - Preserve accounting on mid-resultset server ERR packets by reporting current query in READING_COLUMNS/READING_ROWS before reset. - Keep prepared-statement lifecycle cleanup robust (pending prepare cleared on prepare completion paths). MySQL session integration - Extract duplicated FAST_FORWARD client FFTO feed logic into observe_ffto_client_packet() and reuse it from all call sites. PostgreSQL FFTO - Replace regex-based CommandComplete parsing with lightweight token parsing, including NUL/whitespace trimming and strict numeric validation. - Add queued tracking for pipelined extended-protocol executes so query text and response attribution stay aligned under Parse/Bind/Execute pipelining. - Distinguish finalize semantics (execute-finalize on CommandComplete vs sync-finalize on ReadyForQuery) and centralize finalize/activation helpers. - Add frontend Close ('C') handling to evict statement/portal mappings. - Harden client/server message parsing with additional length checks. - Extend affected-row command tag coverage to COPY and MERGE. TAP tests - Stabilize test plans for failure paths by replacing early returns with a fail-and-skip-remaining flow and shared cleanup labels. - Ensure both MySQL and PgSQL FFTO tests preserve planned assertion counts under setup/prepare/execute failures. Documentation - Align FFT0 design doc state/response descriptions with current implementation (ReadyForQuery handling, pipelined queueing, supported PG command tags). - Fix wording/typo issues in protocol section. Validation performed - make -C lib -j4 - make -C test/tap/tests test_ffto_mysql-t test_ffto_pgsql-t -j4 Runtime execution of the two TAP binaries remains environment-dependent (admin endpoint connectivity required).	3 months ago
René Cannaò	9c6945fae8	Merge branch 'v3.0' into v3.0-tsdb-feature ... Signed-off-by: René Cannaò <rene@proxysql.com>	3 months ago
Rene Cannao	33fce1ca35	MCP stats: address review findings and add detailed docs	3 months ago
René Cannaò	4a6809b60f	Merge branch 'v3.0' into v3.0-tsdb-feature ... Signed-off-by: René Cannaò <rene@proxysql.com>	3 months ago
Rene Cannao	eba795d729	fix(pgsql-eventslog): make dump-to-memory sizing protocol-agnostic across admin ports ... Context: - Admin commands are exposed by one shared Admin module over two protocols: - MySQL admin endpoint (`6032`) - PostgreSQL admin endpoint (`6132`) - `DUMP PGSQL EVENTSLOG ...` must behave identically regardless of which admin protocol executes the command. Issue fixed: - PGSQL events dump path used thread-local PGSQL sizing variables when draining to memory (`stats_pgsql_query_events`). - When command execution happened from a non-PGSQL protocol admin worker, TLS values could be out of sync for PGSQL sizing decisions. Changes: - In `lib/PgSQL_Logger.cpp`, add runtime-size helper functions that read canonical values from `GloPTH->variables` with TLS fallback: - `eventslog_buffer_max_query_length` - `eventslog_table_memory_size` - Use these helpers in: - `PgSQL_Event` copy constructor (buffered query truncation) - `PgSQL_Logger::processEvents()` (in-memory retention for `stats_pgsql_query_events`) - Update internal architecture doc to explicitly state command availability from both admin protocol endpoints. Why: - Ensure `DUMP PGSQL EVENTSLOG ...` semantics remain consistent across `6032` and `6132`, matching the shared-admin-module design.	3 months ago
Rene Cannao	6f5a22f669	docs(internal): update PGSQL advanced logging architecture validation section with auto-dump E2E coverage ... Context: - The architecture document was rewritten as an as-built reference, but latest test coverage additions were not fully reflected. Changes: - Update validation mapping in `doc/internal/pgsql_advanced_query_logging_architecture.md` to include: - `pgsql_query_logging_autodump-t` TAP test - scheduler-driven auto-dump acceptance coverage - both relevant TAP group registrations Why: - Keep internal architecture/validation documentation aligned with implemented and wired test coverage. - Avoid stale validation references when reviewing branch readiness.	3 months ago
Rene Cannao	1edf1f8e86	Add PostgreSQL advanced logging TAP coverage and rewrite architecture doc	3 months ago
Rene Cannao	3ca390dacb	Fix binary protocol support for prepared statements in FFTO ... - Implemented statement ID tracking for MySQL COM_STMT_PREPARE/EXECUTE - Implemented Parse/Bind/Execute portal mapping for PostgreSQL Extended Query - Corrected documentation regarding binary protocol support - Ensured metric parity for prepared statements in Fast Forward mode	3 months ago
Rene Cannao	83a631bb18	Enhance FFTO documentation with verification steps and performance details	3 months ago
Rene Cannao	3a3be5a559	Implement Fast Forward Traffic Observer (FFTO) for MySQL and PostgreSQL ... - Added TrafficObserver base interface - Implemented MySQLFFTO and PgSQLFFTO classes for protocol-aware observation - Integrated FFTO hooks into MySQL and PostgreSQL session handlers in FAST_FORWARD state - Added configuration variables: mysql-ffto_enabled, mysql-ffto_max_buffer_size, pgsql-ffto_enabled, pgsql-ffto_max_buffer_size - Implemented query digest extraction and reporting parity with internal stats - Added session-level bypass when packet size exceeds buffer threshold - Included architectural documentation in doc/ffto_design.md	3 months ago
René Cannaò	762afaa44f	Merge branch 'v3.0' into v4.0-mcp-stats ... Signed-off-by: René Cannaò <rene@proxysql.com>	3 months ago
Rene Cannao	700459f67b	tsdb: address AI code review feedback, fix memory safety and logic bugs ... Summary of changes: - Fixed allocator mismatches (sqlite3_free vs free) across multiple files. - Converted manual SQL construction to prepared statements for TSDB variables. - Resolved memory leaks in statistics and monitoring loops. - Fixed critical 'LOAD TSDB VARIABLES FROM CONFIG' command mismatch. - Improved command parsing robustness using string searching. - Standardized SQL LIKE patterns with escaped wildcards (\%). - Switched to data-driven variable management in ProxySQL_Statistics. - Improved web dashboard security with URL encoding and const correctness. - Clarified documentation regarding configuration lifecycle and schema names. - Updated TAP tests for better reliability and correct persistence verification.	3 months ago
Rene Cannao	9685cdaa4b	mcp discovery: enforce target-scoped run model and add protocol-aware static harvesting (mysql+pgsql) ... Refactor the MCP query/discovery stack to remove prototype-era MySQL-only assumptions and make discovery/catalog semantics explicitly target-scoped. This commit is intentionally not backward compatible with legacy single-target catalog metadata. Key implementation details: - Discovery_Schema: - runs table now stores target_id, protocol, and server_version - resolve_run_id() now requires target_id and resolves schemas within target scope - create_run() now records target/protocol/server version - added legacy schema detection + destructive catalog table rebuild when old runs layout is found - fixed resultset lifetime/escaping issues in target-aware run resolution - New PostgreSQL static harvester: - added PgSQL_Static_Harvester class and build wiring - harvests schemas/objects/columns/indexes/fks/view definitions into Discovery_Schema - maps pg metadata to object_id-based insert APIs used by Discovery_Schema - Query_Tool_Handler: - constructor simplified to catalog_path only (no mcp-mysql_* runtime ctor deps) - discovery.run_static now requires target_id and dispatches protocol-aware harvester at runtime - catalog.*, agent.run_start, and llm.* tool contracts now require target_id when resolving run_id - all run_id resolution call sites switched to resolve_run_id(target_id, ...) - list_schemas catalog query now scoped by target_id via runs join - digest logging path now resolves run_id in target context - ProxySQL_MCP_Server: - updated Query_Tool_Handler construction to new signature - Docs updated: - added explicit target_id scoping guidance for discovery/catalog/agent/llm workflows - clarified protocol-aware routing and noted legacy examples still present in large script docs Build validation: - Recompiled changed objects with PROXYSQLGENAI=1: - Query_Tool_Handler.oo - Discovery_Schema.oo - PgSQL_Static_Harvester.oo - ProxySQL_MCP_Server.oo	3 months ago
Rene Cannao	af0411bd46	MCP: add target-aware rules/stats tests, explain_sql rule coverage, and AI local docker TAP infra ... This commit completes end-to-end MCP query-rules validation for multi-target routing and introduces a self-contained TAP infra for the `ai` group that can run both in Jenkins and manually. Main MCP/runtime changes: - Extended MCP query-rule matching context to include both `target_id` and resolved backend `username`. - Added `target_id` column to `mcp_query_rules` and `runtime_mcp_query_rules` table definitions. - Extended `stats_mcp_query_rules` to include `username` and `target_id` alongside `rule_id` and `hits`. - Updated load/save/runtime refresh paths to persist and rehydrate the expanded MCP rule schema. - Wired the rule engine into `explain_sql` so MCP rules apply consistently across `run_sql_readonly` and `explain_sql`. - Included startup-order fix in `src/main.cpp` to initialize MCP/GenAI thread handlers early, preventing startup crashes in PROXYSQLGENAI builds. Test coverage changes: - Updated existing MCP TAP phases to assert target-aware and username-aware behavior: - `test_phase4_stats.sh` - `test_phase6_eval_block.sh` - Added new MCP TAP phases: - `test_phase10_eval_explain.sh` (rule engine coverage for `explain_sql`) - `test_phase11_pgsql_target.sh` (pgsql target routing/rule/stats coverage; graceful skip if no pgsql target is configured) - Updated `test_mcp_query_rules-t.sh` runner to execute new phases. AI group isolated infra (manual + CI compatible): - Added `test/tap/groups/ai/docker-compose.yml` with MySQL 9.0 and PostgreSQL 16 backends. - Added lifecycle scripts: - `docker-compose-init.bash` - `docker-compose-destroy.bash` - `pre-proxysql.bash` - `post-proxysql.bash` - Extended `test/tap/groups/ai/env.sh` with local default ports/credentials/target IDs used by MCP TAP tests. - Added `test/tap/groups/ai/README.md` documenting manual execution flow outside Jenkins. Outcome: - MCP tests now validate routing-aware rule enforcement and stats attribution for both MySQL and PostgreSQL targets. - The `ai` TAP group can be run with an isolated local backend stack without relying on external Jenkins infra repositories.	3 months ago
Rene Cannao	4a8b224038	MCP TAP/docs: migrate tests and documentation from legacy MCP mysql vars to profile-based routing ... This commit migrates MCP TAP assets and related docs away from deprecated `mcp-mysql_*` style configuration toward the profile-based model centered on auth profiles, target profiles, and `target_id` routing. Changes included: - Updated TAP configuration helpers and test inputs to build and load MCP auth/target profiles. - Replaced legacy variable-driven setup assumptions with runtime profile loading (`LOAD MCP PROFILES ...`). - Refreshed documentation snippets and examples to describe the profile-based configuration flow. Motivation: - Keep test and documentation surfaces aligned with the current MCP architecture. - Eliminate ambiguity between old POC paths and supported profile-driven routing. Outcome: - TAP and docs now consistently describe and exercise MCP routing through profile tables and `target_id` selection.	3 months ago
Rene Cannao	013864b36f	MCP: introduce profile-based target/auth routing and unified LOAD/SAVE MCP PROFILES commands ... This commit introduces the core MCP routing architecture that decouples client-facing target selection from backend connection details. Architecture introduced: - New MCP auth profile table to store backend credentials/policy (`mcp_auth_profiles`). - New MCP target profile table to map opaque `target_id` -> protocol/hostgroup/auth profile (`mcp_target_profiles`). - Runtime mirrors for both profile tables. Behavioral changes: - MCP query tools resolve execution context by `target_id` and route internally by protocol + hostgroup. - Client no longer needs backend-specific connection details; only logical target identifiers. - Unified admin commands for profile lifecycle management: - `LOAD MCP PROFILES TO/FROM MEMORY/RUNTIME/DISK` - `SAVE MCP PROFILES TO/FROM MEMORY/RUNTIME/DISK` Design goals: - Support multi-backend MCP execution (MySQL + PostgreSQL) behind one endpoint. - Provide server-side credential management and cleaner operational model. - Replace legacy one-backend POC configuration with scalable target/auth abstractions. Result: - MCP routing is now dynamic, protocol-agnostic at the client layer, and fully managed through profile tables and unified load/save commands.	3 months ago
Rene Cannao	2267ef2459	doc: update TSDB documentation ... - Documented TSDB Web Dashboard and REST API endpoints. - Updated configuration references to include LOAD/SAVE TSDB VARIABLES commands. - Corrected database aliases (stats_history vs statsdb_disk) in examples. - Removed outdated references to missing API features.	3 months ago
Rene Cannao	a236a13787	tsdb: dedicated module for TSDB variables with tsdb- prefix and optimized background tasks	3 months ago
Rene Cannao	da877c378b	tsdb: align admin variable lifecycle, full prometheus ingestion, docs and tap coverage	3 months ago
Rene Cannao	f17e999830	Implement SQLite-based TSDB subsystem ... Add time-series database extension to ProxySQL_Statistics with: - Three SQLite tables: tsdb_metrics, tsdb_metrics_hour, tsdb_backend_health - Automatic downsampling from raw to hourly aggregates - Retention management (7 days raw, 1 year hourly) - Prometheus metrics sampling - Backend health monitoring via TCP probes - SQL query interface with JSON label filtering - Admin variable integration Documentation: - embedded_tsdb_overview.md - Architecture and benefits - embedded_tsdb_architecture.md - System design and components - embedded_tsdb_reference.md - Configuration and API reference - embedded_tsdb_specs.md - Technical specifications - embedded_tsdb_quickstart.md - Getting started guide All 4 build combinations pass: - make debug / make release - PROXYSQLGENAI=1 make debug / make release	3 months ago
Rene Cannao	a69b925932	Minor typo fixes	3 months ago
Wazir Ahmed	2d3d12b9a0	Address review comments from gemini and coderabbit	4 months ago
Wazir Ahmed	ae92ae265f	Merge branch 'v3.0' into v4.0-mcp-stats	4 months ago
Wazir Ahmed	7f19b642c2	MCP: Rewrite stats tool handlers based on updated spec ... Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	4 months ago