283 Commits (feature/pgsql-native-backend-protocol)

Author	SHA1	Message	Date
Rene Cannao	b25ed9f804	fix(ci): broaden CI-unit-tests-asan-coverage to write-all for OIDC + checks ... Follow-up to 30b255339, which added `checks: write` to the narrow permissions block. That wasn't enough -- the next run still failed with "Resource not accessible by integration" at the LouisBrunner/checks-action step. The reason is the workflow_run trigger context. This workflow is spawned by CI-trigger's completion (workflow_run), and the GITHUB_TOKEN GitHub mints for that context is scoped to the default branch, not to the PR branch the cascade was actually testing. Trying to POST a check-run for a PR-branch SHA from that token fails even with `checks: write` granted -- the same wall the legacy-g2-genai pipeline hit during PR #5818 ratcheting (see the commit history on ci-legacy-g2-genai.yml under GH-Actions, where the author tried adding scopes one at a time before settling on write-all). Switch to `permissions: write-all` -- identical to what every TAP- group reusable workflow in this PR's fan-out already declares. The older minimal block's only motivation was a noted desire not to widen scope unnecessarily; in this case the wider scope is actually required for the workflow's last step to succeed.	1 month ago
Rene Cannao	30b2553398	fix(ci): grant checks:write to CI-unit-tests-asan-coverage ... The workflow's permissions block declared only `contents: read` and `id-token: write`. The final step in the job is a `LouisBrunner/checks-action@v2.0.0` call that updates the check-run status. POSTing a check run requires `checks:write`, and without it the action fails with: ##[error]Resource not accessible by integration Every run since at least 2026-05-26 has failed at this step even when build + ASAN unit tests + LCOV coverage capture + Codecov upload all succeeded -- the workflow has been showing red on the dashboard for purely cosmetic reasons. Found while investigating the #5828 fan-out cascade's red lines (this workflow isn't in the fan-out, but the same cascade re-runs it, so it appeared in the same failure list).	1 month ago
Rene Cannao	8878076cf8	ci(coverage): grant write-all on TAP-group callers for Codecov OIDC ... Fan-out companion to the reusable-side change on GH-Actions (feature/ci-codecov-tap-all-groups-callees). Every CI-<group>.yml caller now declares `permissions: write-all` on its `jobs.run` block so the `id-token:write` scope that codecov- action@v4 needs (`use_oidc: true`) actually reaches the callee. Reusable-workflow permissions are the intersection of caller + callee; without this grant on the caller side, the OIDC token mint fails even though the callee declares write-all. Mirrors what CI-legacy-g2-genai.yml already does (from PR #5818). Workflows touched (38): CI-legacy-clickhouse-g1 CI-legacy-g1, CI-legacy-g3..g9 CI-mariadb10-galera-g1..g9 CI-mysql56-single-g1 CI-mysql84-g1..g9 CI-mysql84-gr-g1..g9 CI-set_parser_algorithm_3-g1	1 month ago
Rene Cannao	4823a40766	ci(coverage): use write-all on legacy-g2-genai caller (match callee) ... Mirror of the callee switch from explicit-scope list to write-all. Reusable-workflow permissions intersect caller and callee, so the callee's write-all is no-op unless the caller also declares write-all. See the callee commit on GH-Actions for the full rationale.	1 month ago
Rene Cannao	7122523db1	ci(coverage): mirror actions:write to legacy-g2-genai caller ... Mirror of the callee scope bump. Reusable-workflow permissions intersect caller + callee, so the callee's actions:write is no-op without this matching grant on the caller.	1 month ago
Rene Cannao	af64f444d6	ci(coverage): mirror actions:read grant to legacy-g2-genai caller ... Mirror of the third callee permission addition (cache restore). Reusable-workflow permissions are intersected with the caller's, so the callee's actions:read grant is no-op without this. The full grant list is now: contents:read checkout id-token:write Codecov OIDC upload checks:write LouisBrunner/checks-action packages:read GHCR docker pull actions:read cache restore This is what GITHUB_TOKEN had implicitly before the `permissions:` block was added; we've now rebuilt that subset explicitly to keep id-token:write (which is not a default and so does need the explicit block).	1 month ago
Rene Cannao	3a594d2058	ci(coverage): mirror packages:read grant to legacy-g2-genai caller ... Same logic as the previous caller commit (65be0dc2c added checks:write to mirror the callee). Reusable-workflow permissions intersect caller + callee, so the callee's packages:read grant is no-op without this caller-side mirror. This should be the final permissions tweak: contents:read, id-token:write, checks:write, and packages:read together restore all defaults the workflow steps actually use, plus the OIDC scope codecov-action@v4 needs.	1 month ago
Rene Cannao	65be0dc2cc	ci(coverage): add checks:write to legacy-g2-genai caller (mirror callee) ... Mirror of GH-Actions side: declaring `permissions:` on the caller shrinks GITHUB_TOKEN to only the listed scopes, which then becomes the upper bound for the reusable workflow's GITHUB_TOKEN. The callee added `checks: write` to fix the LouisBrunner/checks-action permission failure; the caller has to grant the same scope or the intersection drops it again. See sysown/proxysql@<GH-Actions-sha> for the failure mode.	1 month ago
Rene Cannao	3846bbd1be	ci(coverage): grant id-token:write to legacy-g2-genai caller for Codecov OIDC ... The reusable workflow ci-legacy-g2-genai.yml@GH-Actions was just extended (6335dca47) to upload its LCOV coverage report to Codecov via codecov-action@v4 with `use_oidc: true`. That action needs the runner to mint a GitHub OIDC token, which requires `id-token: write` on the calling job. Reusable-workflow permissions are the intersection of caller and callee, so the callee declaring it is necessary but not sufficient -- this commit grants it on the v3.0 caller too. This is the first non-unit-test Codecov upload: real ProxySQL coverage from the legacy-g2 TAP integration suite running against a WITHGCOV=1 proxysql daemon. Once green, the same pattern extends to the other TAP groups (flip their build cache to -tap-genai-gcov, add the upload step + this grant).	1 month ago
René Cannaò	cda58357b2	Merge branch 'v3.0' into feature/ci-codecov-unit-tests	1 month ago
Rene Cannao	51c9dcd79f	ci(coverage): include event_name in concurrency group to stop dispatch cancellation ... Background: dispatching CI-unit-tests-asan-coverage on a feature branch (workflow_dispatch, head_branch=feature/X) while a workflow_run-triggered run for v3.0 is also active was repeatedly cancelling the dispatched run mid-build, even though the previous concurrency expression should have placed them in different groups (`CI-...-feature/X` vs. `CI-...-v3.0`). Three dispatches in a row were killed at ~25 min into a ~35 min build. In practice GitHub's concurrency comparison appears to coalesce the two events on this workflow despite the differing branch suffix. The simplest robust fix is to include `github.event_name` in the group, so a workflow_dispatch run is never in the same group as a workflow_run run regardless of branch. Two workflow_runs for the same branch still serialize via the existing branch suffix; so do two dispatches for the same branch -- the cancellation semantics within a single event type are unchanged.	1 month ago
Rene Cannao	48ae1918f5	ci(coverage): set use_oidc:true on codecov-action to engage OIDC upload ... Last attempt removed the `token:` parameter assuming the action would auto-detect the available `id-token: write` permission and switch to OIDC. It didn't -- codecov-cli 11.2.8 ran `create-commit` with no token, Codecov rejected it as `HTTP 400: Token required because branch is protected` (Codecov's own branch-protection concept, not GitHub's -- every branch defaults to "protected"), and the action moved on. codecov-action@v4 requires the explicit `use_oidc: true` input to mint the OIDC token from the runner and present it as the upload credential. With that flag set, the still-granted `id-token: write` permission becomes load-bearing rather than dormant. This is the only path that works without a static CODECOV_TOKEN secret (the per-repo upload token is hidden in the Codecov UI for public repos with the GitHub App installed -- they want everyone on OIDC, but the action does not enable it by default).	1 month ago
Rene Cannao	eec30125a5	ci(coverage): use OIDC tokenless upload for Codecov, drop token param ... The first run on this branch with non-empty lcov.info successfully produced the artifact but Codecov rejected the upload with `HTTP 400: Token required because branch is protected`. Tokenless upload via the Codecov GitHub App requires two things that the previous commit did not have: 1. `permissions: id-token: write` on the job, so GitHub Actions mints an OIDC token for the action to present to Codecov as proof of the workflow run identity. 2. *Omit* the `token:` parameter on codecov/codecov-action@v4. Passing `token: ${{ secrets.CODECOV_TOKEN }}` (even when the secret is undefined and resolves to empty string) forces the action into legacy token-mode, which is what triggers the "branch is protected" rejection. With no `token:` the action auto-selects OIDC. Codecov UI currently hides the per-repo upload token for public repos with the GitHub App installed -- they want everyone on the OIDC path -- so removing the token reference also closes a chicken-and-egg loop (you can't get a token without an active repo, the repo doesn't activate without an upload, an upload without a token requires OIDC, OIDC requires permissions+omitting the param). Validation runs in dispatched CI on this branch. Side effect: if we ever flip the repo to private, we'd need to re-add `token: ${{ secrets.CODECOV_TOKEN }}` (and provision the secret). Noted in the inline comment.	1 month ago
Rene Cannao	39d1d0feb4	ci(coverage): upload unit-test LCOV to Codecov ... Wire the existing CI-unit-tests-asan-coverage workflow's coverage/lcov.info output to Codecov via codecov/codecov-action@v4. The workflow already builds proxysql with WITHGCOV=1, runs every unit test under test/tap/tests/unit/, and produces a filtered coverage/lcov.info -- this commit just teaches it to also POST that file to https://app.codecov.io/gh/sysown/proxysql. After the first successful upload Codecov: * activates the repository on the dashboard, * starts a historical coverage graph on the v3.0 default branch, * comments on each PR with the per-file coverage delta of touched lines. Tokenless upload via the Codecov GitHub App (already installed at the org level). If a CODECOV_TOKEN repo secret is also present it is forwarded as well -- either path works for public repos. The step is guarded by `if: always()` so coverage uploads even when unit tests fail (partial coverage helps diagnose red runs) and `fail_ci_if_error: false` so a transient Codecov outage cannot turn a green CI run red on a third-party SaaS dependency. This is the smallest possible first step in the coverage plan: it touches exactly one workflow file on the v3.0 branch, leaves the 39 test-group caller/reusable pairs untouched, and unlocks Codecov's PR-comment-with-delta feature for the unit-test surface immediately. A follow-up will extend coverage capture to the TAP test groups (legacy-g*, mysql84-g*, etc.) once the Codecov plumbing is proven on this minimal path.	1 month ago
René Cannaò	35cad4b4f4	Merge branch 'v3.0' into fix/kill-proxysqlgenai-build-flag-v2	1 month ago
Rene Cannao	c648dc92d8	ci: verify package install on clean distro before release upload ... Add verify-package-install.bash that installs the built .deb/.rpm on a clean Docker image of the target distro and runs smoke tests: - proxysql --version catches missing runtime shared library deps - Plugin .so file presence at /usr/lib/proxysql/ (auto-detected) - ELF header validation on installed binaries The script auto-detects whether the package ships plugins by inspecting dpkg -c / rpm -qpl, so it works for both v3.0 (no plugins) and v4.0 genai/mysqlx packages. Wire it into all 28 CI-package-arm64-* workflows (14 genai + 14 base) between the Build and Upload steps. If verification fails, the build job fails and the package never reaches the release draft.	1 month ago
Rene Cannao	ccb5621ada	fix(ci): prevent zombie IN_PROGRESS checks when CI-unit-tests-asan-coverage is cancelled ... The previous pattern opened a LouisBrunner check_run with status=in_progress at job start and closed it via check_id at job end with if: always(). When the runner is killed mid-job (concurrency:cancel-in-progress, OOM, infra shutdown), the closing step never executes and the check_run is left in in_progress forever, blocking the PR rollup. Collapse to a single LouisBrunner call at the end: pass name: instead of check_id: so the action creates the check_run with a terminal conclusion in one shot. If the job is cancelled before the closing step runs, no check_run is ever created (PR rollup shows "missing" — recoverable via re-run) instead of "stuck" — unrecoverable without GitHub App auth. Refs: stuck check_run 77381366265 on PR #5752 (sha cf0b1d36) from 2026-05-22 12:39 UTC. Same pattern exists across ~163 other CI-* workflows; fixing only the workflow with the observed incident here. Sweep of the remaining files belongs in a separate PR.	1 month ago
Rene Cannao	d4a3e1fa7e	fix(ci): add build job to populate the build-<SHA> cache CI-mysqlx restores (#5796) ... CI-mysqlx has failed 100% of its last 500 workflow runs at the "Restore build cache" step. The three jobs (unit-tests, e2e-tests, soak-tests) each begin with an actions/cache@v4 restore keyed on `build-<SHA>` with `fail-on-cache-miss: true`, and no workflow in the repository ever writes that key. CI-trigger (which CI-mysqlx is triggered by) doesn't write any cache. CI-builds writes `<SHA>_<dist><type>_*` keys, completely different from `build-<SHA>`. Net effect: every workflow_run-triggered CI-mysqlx invocation aborts immediately before any build, infra setup, or test runs. The mysqlx test suite — chassis plugin loader, X-Protocol, route management, unit tests, the soak harness — has been dead code in CI since at least 2026-05-03 (the API window limit; likely earlier). This commit adds a `build` job at the top of CI-mysqlx.yml that: - does a full checkout (not sparse — needs deps/, lib/, src/), - tries the cache restore first (so a re-run on the same SHA short-circuits the build), - on cache miss, runs PROXYSQL40=1 PROXYSQL31=1 PROXYSQLFFTO=1 PROXYSQLTSDB=1 \ make -j$(nproc) debug && make -j$(nproc) build_tap_test_debug - lets actions/cache@v4's automatic post-job save write the cache. The three existing jobs gain `needs: build` (directly for unit-tests; transitively for e2e-tests and soak-tests via the existing `needs: unit-tests` chain). Their `fail-on-cache-miss` flags are flipped from the conditional `${{ github.event_name == 'workflow_run' }}` to unconditional `true`: the build job is also gated by the same `if:` condition as the rest of the workflow, so the cache is guaranteed to exist by the time any downstream job starts. The PROXYSQL40 tier flags in the build step deliberately match what each downstream job's "Build mysqlx plugin" step uses — see the ABI-mismatch comment on that step. A mismatch here would let the plugin link succeed but corrupt memory on first plugin dispatch. After this PR lands, CI-mysqlx will actually execute its test binaries. Downstream test failures (listener-bind timing in setup-infras.bash, X-Protocol handshake auth, the admin localhost-only restriction blocking the test-runner container) will then surface in CI for the first time. Each one should get its own follow-up issue; this PR is the unblocker. Fixes #5796	1 month ago
Rene Cannao	431c65476a	fix(ci): clear SonarCloud security-rating issues on new fedora44 workflows + tsdb test ... PR #5778 still failed the "D Security Rating on New Code" gate after the SHA-pinning commit, with 27 VULNERABILITY-typed issues remaining: 1. 24 × workflow files (MAJOR, rule S6571): "Move this write permission from workflow level to job level." Moved `permissions: { contents: write, checks: write }` from the top of each of the 12 new fedora44 workflows to each of the two jobs (init_release, build) that actually need them. Same effective permissions; principle-of-least-privilege satisfied per job scope. 2. 3 × test/tap/tests/test_tsdb_api-t.cpp (CRITICAL): - Line 75 (S4423 "Use stronger SSL and TLS versions"): added `CURLOPT_SSLVERSION = CURL_SSLVERSION_TLSv1_2` to pin the test client to TLS 1.2+. Real improvement. - Lines 83/84 (S5527 / S4830 "Enable cert/hostname verification"): `CURLOPT_SSL_VERIFYPEER=0` and `CURLOPT_SSL_VERIFYHOST=0` are by design — the test connects to a localhost proxysql that serves an auto-generated self-signed cert; installing the per-run cert into the system CA store is out of scope for a TAP test. Added NOSONAR + explanatory comment. The remaining 9 CODE_SMELL findings (void*, cognitive complexity, nested control flow in test_tsdb_api-t.cpp and the new libconfig regression test) do not factor into the Security Rating and are left for separate cleanup if desired.	2 months ago
Rene Cannao	043f947915	fix(ci): clear SonarCloud hotspots on new fedora44 workflows + tsdb test ... Two categories of S security-hotspot findings on PR #5778: 1. LouisBrunner/checks-action@v2.0.0 not pinned to a commit SHA (24 hotspots, lines 90 and 141 of each of the 12 new fedora44 workflow files). Pinned to `@6b626ffbad...` (the commit that v2.0.0 currently points to). The other ~78 workflow files in the repo still use the version-tag form; project-wide SHA pinning is out of scope for this PR. 2. Plain-HTTP literal in test/tap/tests/test_tsdb_api-t.cpp:196 (1 hotspot). The ProxySQL REST API listens on plain HTTP by design; the test must hit the real endpoint, not a TLS variant. Added a clarifying comment and a NOSONAR pragma to suppress the false-positive.	2 months ago
Rene Cannao	11412589b6	fix(deps): patch jemalloc for GCC 16; add fedora44 to build matrix ... Bundled jemalloc 5.2.0 uses the libstdc++ internal symbol std::__throw_bad_alloc() in src/jemalloc_cpp.cpp, which GCC 16's libstdc++ no longer exposes through <new>. This makes the build fail on Fedora 44 and any other distro shipping GCC 16. Backport the user-visible part of upstream jemalloc commit 1a15fe33, replacing `std::__throw_bad_alloc()` with `throw std::bad_alloc()`. The full upstream patch adds a configure-time check that compiles either a throw or std::terminate() depending on whether C++ exceptions are enabled; ProxySQL always builds with exceptions, so the simpler unconditional form is sufficient. Verified in a `gcc:16` container (Debian trixie + GCC 16.1.0): - without the patch: reproduces the reported error exactly. - with the patch: jemalloc builds clean, symbols reference std::bad_alloc instead of __throw_bad_alloc. Also add Fedora 44 (ships GCC 16.1.1 by default) to the packaging matrix as the automated regression net: - docker-compose.yml: fedora44_build, fedora44_clang_build, fedora44_dbg_build services. - Makefile: include fedora44 variants in amd64-fedora and arm64-fedora recipe lists. - .github/workflows/: 12 new CI-package-{amd64,arm64}-fedora44*.yml workflows, mirrors of the existing fedora43 set. All triggered by workflow_dispatch (no GH-Actions branch counterpart needed). The matching packaging images (`proxysql/packaging:build-fedora44-v4.0.0` and `proxysql/packaging:build-clang-fedora44-v4.0.0`) come from ProxySQL/docker-images#18. Fixes #5770	2 months ago
Rene Cannao	cfad7c8554	feat(ci): add mariadb10-galera TAP group with CI-mariadb10-galera-g[1-9] workflows	2 months ago
Rene Cannao	949e4cdc30	feat(ci): add CI-mysql84-gr-g[1-9] workflows for Group Replication test groups	2 months ago
Rene Cannao	af3c985a74	feat(ci): add workflow to build and push proxysql-ci-base image to GHCR ... Triggers on pushes to v3.0 that change test/infra/docker-base/Dockerfile. Pushes to ghcr.io/sysown/proxysql-ci-base with latest + SHA tags. Reusable workflows on GH-Actions will pull this image instead of building from scratch, eliminating apt-get mirror sync failures in CI.	2 months ago
Rene Cannao	6033c4e1ab	feat(ci): split legacy-g[1-4] and mysql84-g[1-4] into parallel groups for faster CI ... Split large test groups roughly in half to reduce wall-clock CI time: - legacy-g1 (88) → legacy-g1 (44) + legacy-g6 (44) - legacy-g2 (46) → legacy-g2 (23) + legacy-g7 (23) - legacy-g3 (41) → legacy-g3 (21) + legacy-g8 (20) - legacy-g4 (98) → legacy-g4 (49) + legacy-g9 (49) - mysql84-g1 (80) → mysql84-g1 (40) + mysql84-g6 (40) - mysql84-g2 (42) → mysql84-g2 (21) + mysql84-g7 (21) - mysql84-g3 (41) → mysql84-g3 (21) + mysql84-g8 (20) - mysql84-g4 (61) → mysql84-g4 (31) + mysql84-g9 (30) Adds: group directories, caller workflows, groups.json entries. Reusable workflows on GH-Actions branch in companion PR.	2 months ago
Rene Cannao	1946b1f728	fix(ci): rename set_parser_algorithm=3 group to avoid Docker network name issue ... The "=" character in the TAP group name "set_parser_algorithm=3-g1" is incompatible with Docker network names -- Docker interprets "=" as a key=value separator in the --network flag, causing the CI job to fail with "invalid field key ci-set_parser_algorithm". Rename the group to "set_parser_algorithm_3" everywhere: - Group directory: set_parser_algorithm=3/ → set_parser_algorithm_3/ - groups.json entries - Caller workflow file and name - Reusable workflow file, name, and INFRA_ID references Also fix SonarCloud cognitive-complexity Quality Gate failure in tools/bench_connect.c by extracting parse_args() and print_json() helpers from main(), reducing complexity from 31 to below 25.	2 months ago
René Cannaò	44c33cd2d3	Merge branch 'v3.0' into feature/parsersql-integration	2 months ago
Rene Cannao	5d6ba67d97	fix: add NOSONAR annotations for SonarCloud security hotspots ... - S5813 (strlen safety): all 3 instances use string literal prefixes or null-terminated array entries where strlen is compile-time evaluated or trivially safe - S7635/S7637 (GitHub Actions): branch ref and secrets:inherit match the pattern used by all other caller workflows in the repo	2 months ago
Rene Cannao	a9f2eaaa13	feat: remove PROXYSQLGENAI flag, genai now builds under PROXYSQL40 (issue #5722) ... PROXYSQLGENAI flag eliminated entirely. PROXYSQL40=1 now unconditionally builds the genai plugin alongside core. All #ifdef PROXYSQLGENAI replaced with #ifdef PROXYSQL40 in source, Makefiles, and 61 CI workflows.	2 months ago
René Cannaò	80b93f11a1	Merge branch 'v3.0' into feature/parsersql-integration	2 months ago
Rene Cannao	95f948de5d	feat: add set_parser_algorithm=3 TAP test group and CI workflow ... - New test group test/tap/groups/set_parser_algorithm=3/ with env.sh, infras.lst, and pre-proxysql.sql enabling both mysql and pgsql set_parser_algorithm=3 - 14 SET-related tests added to set_parser_algorithm=3-g1 in groups.json (12 MySQL + 2 PgSQL) - 2 ParserSQL unit tests added to unit-tests-g1 in groups.json - CI reusable workflow and caller workflow for set_parser_algorithm=3-g1 using Docker-safe INFRA_ID (ci-set-parser-algorithm-3-g1) - Locally verified: infrastructure starts, tests discovered correctly, unit tests pass (51/51 digest, 202/224 SET)	2 months ago
Rene Cannao	1d2eea260c	ci(asan-coverage): containerise unit-tests workflow + extract runner script ... Ports CI-unit-tests-asan-coverage from host-direct execution to Docker isolation, mirroring the architecture established for TSAN in PR #5725. Closes #5721 for the ASAN-coverage half. Why: Two parallel CI runs on a shared / self-hosted runner can no longer collide on the host filesystem (no /opt/proxysql contention, no toolchain skew between build env and run env). The runner stays clean — no host-direct `apt install lcov fastcov`, no host-direct `make build_deps_debug` polluting it. Architecture: * Build phase: `make ubuntu24-tap` with WITHASAN=1 WITHGCOV=1 NOJEMALLOC=1 PROXYSQLGENAI=1, runs entirely inside the ubuntu24_dbg_build container (same path TSAN uses). * Test+coverage phase: `docker compose run --rm` re-enters the same image and invokes the new canonical runner script test/infra/control/run-unit-tests-asan-coverage.bash. The workflow YAML is now a thin wrapper around the script — local repro and CI execute the exact same command. The extracted script: * Iterates every executable under test/tap/tests/unit/ (drop-in replacement for the old host-direct loop). Deliberately not routed through run-tests-isolated.bash because that runner's dual-directory test discovery (test/tap/tests/ in addition to .../unit/) pulls in misclassified entries from groups.json that aren't actually unit tests (e.g. unit-strip_schema_from_ query-t lives in test/tap/tests/, needs backend infra, fails silently as a host-direct binary). Cleaning that up is out of scope here. * Captures baseline (--initial) + post-test LCOV, merges, filters /usr + deps + test, runs genhtml. * Idempotent on lcov / libprotobuf-dev — only apt-installs when missing so re-running locally inside the same container is fast. * Honours ASAN_OPTIONS from the caller; defaults match the env block at the top of the workflow. PROXYSQLGENAI=1 stays (instead of dropping to PROXYSQL40=1) — the unit-test set includes genai_*_unit-t binaries that only build under the genai tier. Coverage scope is unchanged from the pre-Docker workflow. Verified locally — same `make ubuntu24-tap` + `docker compose run --rm ... run-unit-tests-asan-coverage.bash` invocation the workflow uses: 83/83 unit tests pass under ASAN, coverage report generated (13.1% lines / 21.3% functions, expected for the unit-only scope), lcov.info + coverage/html/ + unit-test-logs/ all land in the host workspace via the bind mount. Workflow YAML net diff: -185 lines (the test+coverage logic moved out of the YAML into the script).	2 months ago
Rene Cannao	540fb5b4e6	ci(tsan): drop bogus -p PROJECT from docker compose run ... Fixes a regression introduced in 6099d06c1 ("ci(tsan): address review findings on PR #5725"). That commit mirrored the literal expression `${GIT_VERSION/./}` from the Makefile into the workflow YAML, on the (incorrect) assumption that the Makefile expression was bash parameter expansion stripping the first dot. It isn't: the Makefile recipe runs through *make* first, which treats `${GIT_VERSION/./}` as a variable-name reference (the literal name `GIT_VERSION/./`), finds no such variable, and expands it to the empty string. So docker compose ends up invoked with `-p ""`, which falls back to the directory basename — the same default a `-p`-less invocation gets. The workflow YAML is interpreted by *bash*, where the same expression really does perform single-dot stripping, leaving results like "30.8-693-gXXXX". Docker compose v2's project-name validator rejects that: invalid project name "30.8-693-gXXXX": must consist only of lowercase alphanumeric characters, hyphens, and underscores as well as start with a letter or number so every CI-unit-tests-tsan run on a PR that triggers workflow_run on the v3.0 default branch fails before reaching the test step. Fix: drop the `-p PROJECT` flag entirely, matching what the Makefile effectively does. Comment rewritten to record the gotcha so the next reviewer doesn't try the same "consistency" fix again. Root cause for the merge-with-broken-workflow itself: a workflow that uses `workflow_run: workflows: [ CI-trigger ]` cannot be exercised on its own PR cycle — GitHub Actions only fires workflow_run-triggered workflows from files that already exist on the default branch. So the original PR's CI never actually ran the new workflow; only the post-merge CI on v3.0 did, and by then it was too late to catch this without a follow-up. Worth considering whether the next sanitizer/coverage workflow we add should additionally trigger on `pull_request` (perhaps gated on a label) so we get one validating run before merge.	2 months ago
Rene Cannao	6099d06c1e	ci(tsan): address review findings on PR #5725 ... Two small follow-ups from the CodeRabbit + Gemini review: * include/makefiles_vars.mk — emit a $(warning ...) for vm.mmap_rnd_bits=28 from the WITHTSAN=1 branch, mirroring the existing ASAN warning. Both sanitizers need the same 28-bit ASLR ceiling and the original commit's claim that "the warning above already covers both" was wrong: the ASAN warning only fires under WITHASAN=1, so a TSAN-only build was running silent. * .github/workflows/CI-unit-tests-tsan.yml — fix the compose project-name transform to match the Makefile. The Makefile's binaries/proxysql% rule uses `${GIT_VERSION/./}` (strip first dot only); the workflow was using `${GIT_VERSION//./}` (strip all dots), which put `docker compose run` into a different project namespace than `make ubuntu24-tap` had declared. In practice the Makefile tears down its project before exiting so the visible effect was just a redundant fresh network/volume on the run step, but the build-chain contract is "same project name throughout" and we should honour it. Comment block updated so it stops claiming the two transforms are the same operation.	2 months ago
Rene Cannao	50f97de7a9	ci: add CI-unit-tests-tsan workflow + mysqlx-tsan-g1 TAP group ... Wires up Phase 2 of issue #5675: ThreadSanitizer coverage for the mysqlx + plugin-chassis unit tests. Both the build AND the test execution run inside the ubuntu24_dbg_build container — staying within the isolation contract that all ProxySQL CI honors. Two parallel runs on a shared / self-hosted runner can never collide on the host filesystem (no /opt/proxysql contention, no toolchain contention, no shared-lib version skew between the build env and the run env). The build flag plumbing: * include/makefiles_vars.mk gains a WITHTSAN=1 branch that sets WASAN := -fsanitize=thread and forces NOJEMALLOC=1. WITHTSAN is mutually exclusive with WITHASAN — both reroute the same memory-management hooks and the linker rejects the combo. * docker-compose.yml passes WITHTSAN through to the build container alongside WITHASAN/WITHGCOV. PROXYSQL40 also gets explicit passthrough so chassis-only callers don't need to opt into the broader PROXYSQLGENAI cascade just to build the mysqlx plugin. * docker/.../entrypoint.bash installs libprotobuf-dev on demand when either PROXYSQL40=1 or PROXYSQLGENAI=1 is set (mysqlx plugin's pkg-config check) and forwards whichever flag the operator set explicitly to make. The workflow: * Lowers vm.mmap_rnd_bits to 28 on the runner — TSAN can't reserve its shadow region on Linux 5.18+ default of 32. The privileged container inherits the host setting, so a host-side sysctl is sufficient (same constraint the existing ASAN-coverage workflow already documents). * Runs `make ubuntu24-tap` for the build, then re-enters the same image via `docker compose run --rm` for test execution. Same volume mount (./:/opt/proxysql), same toolchain, same runtime libs — what the binaries linked against is what they load. The compose project name is recomputed from git describe so `run` lands inside the namespace `up` declared. * Installs python3-packaging + libprotobuf-dev on the fly inside the test container. The build image is package-build-focused and ships neither: python3-packaging is needed by run-tests-isolated.bash for @proxysql_min_version filtering, and libprotobuf-dev provides the libprotobuf.so.32 the mysqlx_*-t binaries link against dynamically (the build entrypoint already installs it during build, but a fresh `docker compose run` container starts clean). The TAP group: * mysqlx-tsan-g1/env.sh sets SKIP_PROXYSQL=1 (no daemon, no backend infra needed for unit tests) and TSAN_OPTIONS to collect every race in one run rather than aborting on the first. * groups.json registers all mysqlx_*-t and plugin_*-t binaries that build cleanly on plain v3.0; mysqlx_admin_commands_unit-t and mysqlx_robustness_unit-t are deliberately excluded (pre-existing v3.0 baseline failures unrelated to TSAN, also skipped in unit-tests-g1's known-failures list). plugin_runtime_views_unit-t is excluded because plain v3.0 has the .cpp file but the Makefile's PROXYSQL40 UNIT_TESTS list never registers it for build — the test suite expects chassis features added later in the stack. Verified locally — same `make ubuntu24-tap` build path the workflow uses, then `docker compose run --rm ubuntu24_dbg_build ... run-tests-isolated.bash` for execution: 26/26 unit tests pass under TSAN with no races reported.	2 months ago
Rene Cannao	d4427731b7	Merge remote-tracking branch 'origin/v3.0' into plugin-chassis ... Picks up the v3.0 fixes that unblock plugin-chassis CI: - 5d702b12b build(test): fix testgalera link after switch to src/SQLite3_Server.cpp — restores -Wl,--allow-multiple-definition, ClickHouse/zstd/lz4 link, and ProxySQL_Admin.cpp recompile under TEST_GALERA. Resolves CI-maketest red on plugin-chassis. - baa5069ea introduces test/tap/groups/mysql56-single/ which the CI-mysql56-single-g1 workflow expects. Resolves the missing-infra error. - macOS package upload + init_release race-tolerant grouping + amd64 package workflow expansion + various CI/release cleanups. CI-unit-tests-asan-coverage on plugin-chassis still depends on the v3.0 workflow YAML having libprotobuf-dev in its apt install step (workflow_run loads the YAML from the default branch). That fix is already on plugin-chassis (47ab14148) and will land on v3.0 via a separate small PR.	2 months ago
Rene Cannao	47ab14148f	ci: add libprotobuf-dev to unit-tests-asan-coverage workflow ... This workflow runs directly on the GitHub runner host (not inside one of the deb/rhel/suse-compliant docker containers whose entrypoints already install libprotobuf-dev on demand), so the dependency must be installed in the workflow itself. Without it, building with PROXYSQLGENAI=1 fails at Makefile parse time: the top-level Makefile recurses into plugins/mysqlx (PROXYSQL40 is implied by PROXYSQLGENAI), and plugins/mysqlx/Makefile's protobuf 3.x ABI guard aborts when it cannot find pkg-config metadata for protobuf. The rest of the dependency list mirrors INSTALL.md's Ubuntu section, to which libprotobuf-dev was added separately for general builds; this brings the workflow's package set in line with that.	2 months ago
Rene Cannao	99a745f6ed	ci(mysqlx): wire mysqlx-soak group into TAP harness end-to-end ... Completes the four follow-up items documented in the mysqlx-soak group's README. After this commit, the harness scripts run inside the proper docker-isolated TAP framework — no more ad-hoc invocations. ## (1) Add mysql-connector-python to proxysql-ci-base test/infra/docker-base/Dockerfile installs python3 + a few pip packages but lacked the X DevAPI bindings. Add `mysql-connector-python` to the existing `pip3 install` line. Image must be rebuilt (`docker build -t proxysql-ci-base:latest test/infra/docker-base`); the new soak-tests CI job rebuilds unconditionally per run, so CI gets the new package automatically. ## (2) TAP wrappers for the harness scripts Two new Bash TAP entries under test/tap/tests/: * test_mysqlx_soak_behavioral-t.sh — emits two TAP assertions: scenario 1 = SIGTERM-mid-traffic (the harness signals the proxysql container with `docker kill -s TERM proxysql.${INFRA_ID}` mid-run and verifies clients receive Mysqlx::Error 1053 instead of TCP RST); scenario 2 = LOAD MYSQLX ROUTES TO RUNTIME mid-traffic. Both fall back to "skip" if mysql-connector-python is missing, or if the proxysql container is unreachable after scenario 1. * test_mysqlx_soak_stress-t.sh — single TAP assertion that wraps stress.py. Defaults to 60s/20-clients to fit a CI timeout; long soaks invoke stress.py directly with --duration 24h per issue #5677. Both wrappers default the connection params to the docker-internal hostname `proxysql` (via network alias) so they work from inside the test-runner container; environment overrides let local invocations point elsewhere. ## (3) Register in groups.json Two new entries: "test_mysqlx_soak_behavioral-t" : [ "mysqlx-soak-g1", "@proxysql_min_version:4.0" ], "test_mysqlx_soak_stress-t" : [ "mysqlx-soak-g1", "@proxysql_min_version:4.0" ], Both use the @proxysql_min_version:4.0 tag (the harness only makes sense in chassis-aware builds). Lint passes (421 entries, sorted). ## (4) CI job Add `soak-tests` job to .github/workflows/CI-mysqlx.yml. Pattern mirrors CI-taptests-pgsql-cluster.yml: restore build cache, build plugin, build proxysql-ci-base, ensure-infras (TAP_GROUP=mysqlx-soak), run-tests-isolated (TAP_GROUP=mysqlx-soak-g1), cleanup, archive logs on failure. The job runs after unit-tests passes (same dependency as e2e-tests) and is independent of e2e-tests (parallel execution OK). ## What this covers * Builds the plugin and rebuilds the test image with the X DevAPI. * Stands up a real MySQL 8.4 backend (3-node replication via the existing infra-dbdeployer-mysql84 image, X protocol on port 23306-23308 inside the docker network). * Stands up ProxySQL in a container with the plugin .so bind-mounted and a per-group config that declares plugins=("..."). * The mysqlx-soak setup-infras hook provisions one route, one user, one endpoint, reloads, and verifies the listener bound on 6603. * Two TAP tests run inside the test-runner container against the freshly-stood-up ProxySQL, exercising the plugin end-to-end. ## What this does NOT cover * Long-running soaks (24-72h). The CI job runs a 60s stress for signal; the full soak per issue #5677 needs staging. * All compression/TLS combinations. The harness's defaults are uncompressed + clear-text; matrix expansion is future work. * Listener-port collisions across parallel CI runs. INFRA_ID isolates docker-network names but the TAP_GROUP-scoped MYSQLX_ PROXYSQL_PORT (default 6603) is a single value. CI runs are serial per workflow concurrency group; not a problem today but worth flagging if matrix-fanout is added.	2 months ago
Rene Cannao	df7e335e23	fix(ci,infra): pass PROXYSQL40 to plugin build, remove orphaned infra files ... Three CI/infra clean-ups that fall out of the independent review. 1) .github/workflows/CI-mysqlx.yml: pass PROXYSQL40=1 (and the implied tier flags) to `make` when building the mysqlx plugin. plugins/mysqlx/Makefile picks up tier flags from the environment and propagates them as -DPROXYSQL40 / -DPROXYSQL31 / -DPROXYSQLFFTO / -DPROXYSQLTSDB / -DPROXYSQLGENAI on every compile line. If the workflow runs `make` with no flags, the plugin is built without -DPROXYSQL40 — meaning the ProxySQL_PluginDescriptor and ProxySQL_PluginServices struct layouts compiled into the plugin .so differ from the layouts that the cached src/proxysql binary (built upstream by CI-trigger with the full tier flags) sees. The link succeeds and the first virtual dispatch into a plugin callback crashes or, worse, corrupts memory silently. The Makefile already warns about exactly this in lines 56-61. Pass the flags explicitly. 2) test/tap/groups/mysqlx-e2e/infras.lst + test/infra/docker-compose-mysqlx.yml: delete both — they are orphaned. `infras.lst` referenced `infra-mysqlx`, which has never existed under `test/infra/`. ensure-infras.bash would have errored on it — except `mysqlx-e2e/env.sh` exports SKIP_PROXYSQL=1, which makes ensure-infras.bash short-circuit at line 38 before it reaches the docker-compose loop. So infras.lst was both wrong AND unreachable — the worst combination for the next reader trying to figure out how the e2e group is wired. Similarly, `test/infra/docker-compose-mysqlx.yml` was a docker-compose file that no `infra-*/` directory points at, left over from an early plan to use Docker for the mysqlx backend that was abandoned in favour of dbdeployer. Both files are dead weight; delete them. 3) test/tap/groups/mysqlx-e2e/env.sh: comment the unconventional wiring so the next reader doesn't have to reverse-engineer it. Document why this group has no `infras.lst` / no `infra-mysqlx/` dir, why CI uses inline dbdeployer instead of ensure-infras.bash, and that setup-infras.bash + pre-cleanup.bash exist for local-only ad-hoc use today. 4) test/tap/tests/Makefile: drop the dangling `test_mysqlx_listener_smoke-t` target. That test was retired together with the dormant MysqlxWorker path in commit 98aee7db2; the unit/Makefile no longer builds it. The wrapper target in test/tap/tests/Makefile remained and would fail `make test_mysqlx_listener_smoke-t` with "no rule to make target". Replace with a NOTE pointing at where the listener-lifecycle coverage actually lives now (mysqlx_thread_unit-t, mysqlx_robustness_unit-t). Verified: the plugin still builds clean with the tier flags exported to the sub-make. No behavioural change for the docker-using groups.	2 months ago
Rene Cannao	78f06fcade	ci: fix macOS package naming and release upload ... The 6 macOS workflows had three interacting bugs: 1. All three tier variants (stable/v31/genai) produced tarballs with the same filename (proxysql-${GIT_VERSION}-macos-<arch>.tar.gz), where GIT_VERSION came straight from `git describe`. The Makefile bumps the version for v31/genai builds, but the workflow did not mirror that — so a v3.1 or v4.0 binary was packaged under a 3.0.x name. 2. `gh release upload --clobber` then overwrote the tarballs of the sibling tier jobs racing on the same filename, leaving only one surviving macOS artifact per arch across the three tiers. 3. `gh release create v3.0-head --draft 2>/dev/null || true` spawned extra duplicate drafts, and `gh release upload v3.0-head` picked a non-deterministic draft (often a stale SHA) rather than the canonical current-SHA draft used by the Linux package workflows. Fixes, matching the Linux CI-package-* workflows: - Add an `init_release` job that finds or creates the canonical draft for the current SHA+tag via a race-tolerant lowest-id-wins loop, and passes RELEASE_ID to the build job. - Compute the tarball version by mirroring the Makefile's tier bump (major+1 for GENAI, minor+1 for 31), so artifacts are named proxysql-3.0.8-…, proxysql-3.1.8-…, proxysql-4.0.8-… — consistent with the Linux RPM/DEB naming convention. - Upload by release ID with per-asset delete-if-exists, eliminating cross-tier clobber and stale-draft targeting.	2 months ago
Rene Cannao	ac6438955f	ci: drop shared concurrency group on init_release, use race-tolerant find-or-create ... The previous design serialized all init_release jobs via concurrency: group: release-init-${{ github.sha }} cancel-in-progress: false That ran into GitHub's "1 running + 1 pending" rule on concurrency groups: when a third job tries to enter the group, the currently- pending one is cancelled in favor of the newer arrival. Triggering all 156 package workflows at once caused ~142 of them to be cancelled before their init_release even got a chance to run. Replace serialization with a race-tolerant find-or-create loop that all 156 init jobs run in parallel: - Stagger start 0-9s to reduce thundering-herd on the first query. - Up to 6 attempts: look up drafts matching this SHA + tag_name (strongest key); use the one with the lowest id (deterministic winner). - If none exist, POST create; sleep 2-5s; re-query. Multiple creates may race during the initial burst, producing a handful of duplicate drafts, but every worker converges on the same lowest-id draft. - Fail hard after 6 attempts. Duplicates from races are not deleted here (deletion under contention risks a worker using a just-deleted release). A follow-up can sweep zero-asset duplicate drafts periodically if the list grows.	2 months ago
Rene Cannao	1dd62fc4aa	ci: expand amd64 package workflows to include clang and dbg variants ... ARM64 only has the plain gcc variant per distro (13 × 3 tiers = 39). amd64 supports three build variants per distro in the Makefile — plain gcc, -clang, and -dbg — so the per-distro workflow count should be 3× higher: 13 distros × 3 variants × 3 tiers = 117 amd64 workflows. Naming: CI-package-amd64-<distro>[-<tier>][-<variant>] ex: CI-package-amd64-almalinux10 CI-package-amd64-almalinux10-clang CI-package-amd64-almalinux10-dbg CI-package-amd64-almalinux10-v31 CI-package-amd64-almalinux10-v31-clang CI-package-amd64-almalinux10-v31-dbg CI-package-amd64-almalinux10-genai CI-package-amd64-almalinux10-genai-clang CI-package-amd64-almalinux10-genai-dbg Internally each workflow now exposes both DIST (distro) and MAKE_TARGET (distro + variant suffix) env vars; the build step uses MAKE_TARGET so the -clang / -dbg Makefile targets are invoked correctly. Shared init_release concurrency group and one-draft-per-SHA release dedup (Option A) unchanged and applies to all 156 workflows.	2 months ago
Rene Cannao	5bd72d1f28	ci: add amd64 package workflows, fix upload glob, dedupe release drafts ... - Add 39 CI-package-amd64-<distro>[-tier].yml workflows mirroring the existing ARM64 matrix (13 distros × 3 tiers: stable, v31, genai) on ubuntu-24.04 runners. - Fix the upload glob in all 78 package workflows: the previous `binaries/*.[mb]` matched only single-char extensions (nothing), so no artifacts were ever attached. Restored to `binaries/*[mb]` (matches .rpm / .deb trailing char). - Replace the single-job `gh release create --draft` pattern (which produced one brand-new draft per workflow run because GitHub does not dedupe drafts by tag name) with a two-job structure: - init_release: serialized across all 78 workflows via concurrency.group=release-init-<sha> with cancel-in-progress:false. Finds or creates one draft named "<branch>-head - <git describe>" and outputs its release_id. - build: needs init_release, uploads artifacts by release ID using the uploads.github.com endpoint (idempotent: deletes any existing asset with the same name before re-upload). Result: exactly one draft release per commit, hidden, containing all package artifacts from every distro × tier × arch combination.	2 months ago
Rene Cannao	6220ebf726	fix: macOS build - sqlite3 link and missing libtool ... - deps/Makefile: Use -dynamiclib -undefined dynamic_lookup for sqlite3 shared lib on Darwin (Linux -shared allows unresolved symbols, macOS does not). Also link OpenSSL for SHA1 symbol resolution. - workflows: Add libtool to brew install (provides glibtoolize needed by curl's autoreconf).	2 months ago
Rene Cannao	05848ef2e4	ci: fix macOS workflows - don't override PATH globally ... Homebrew's git/make shadow system tools causing checkout to fail (sed, basename, uname not found). Move PATH override into the build step only and remove make/git from brew install list (already on runner).	2 months ago
Rene Cannao	1c654339c9	ci: add 6 on-demand macOS build workflows ... Add workflow_dispatch-only workflows for building ProxySQL natively on macOS (no Docker). Covers Intel (macos-13) and Apple Silicon (macos-14) across 3 feature tiers (stable, PROXYSQL31, PROXYSQLGENAI). Each workflow installs Homebrew dependencies, builds with make -j, creates a tarball of the binary with sha256, and uploads to the <branch>-head draft prerelease.	2 months ago
Rene Cannao	4cd48b9db6	ci: replace artifact upload with GitHub Release upload ... Artifact storage limit (500MB) is too small for 39 packages (~40MB each). Switch to gh release upload to a <branch>-head draft prerelease, matching the existing ci-package-build.yml pattern. Create the release if it doesn't exist, then upload with --clobber. Update permissions from contents:read to contents:write for release uploads.	2 months ago
Rene Cannao	53e9716482	ci: add permissions block and remove unused BIN_PKG/BIN_HASH ... Add explicit permissions (contents: read, checks: write) to ensure checks-action works under read-only default token permissions. Remove dead BIN_PKG and BIN_HASH environment variable assignments that could corrupt $GITHUB_ENV with multi-line ls output.	2 months ago
Rene Cannao	cec72a8226	ci: add 39 on-demand ARM64 package build workflows ... Add workflow_dispatch-only workflows for building ARM64 packages across 13 distros and 3 feature tiers (stable/v3.0, PROXYSQL31/v3.1, PROXYSQLGENAI/v4.0). Each workflow is self-contained, runs on ubuntu-24.04-arm, and uploads the built .deb/.rpm as a 90-day workflow artifact.	2 months ago
René Cannaò	7ca0d153cc	Merge pull request #5654 from sysown/fix/pgsql-active-tx-on-broken-conn ... fix(pgsql): surface mid-transaction backend errors instead of silently replaying	2 months ago