15 Commits (dbb40a30baec7d23568ebe2d2b228737d31dfcba)

Author	SHA1	Message	Date
Rene Cannao	bee37b2cb8	fix(tester): abort when TAP_GROUP is set but has no tests registered ... Previously, if TAP_GROUP was set but the group had no entries in test/tap/groups/groups.json (typo, missing registration, etc.), the tester logged a warning and silently fell back to running ALL 325 TAP tests. In a multi-group parallel run this turns a misconfiguration into a multi-hour hang that only surfaces when the per-group TIMEOUT fires. We hit this in practice: mysql90-g[2-5] existed as launch targets in the runner but hadn't been populated in groups.json yet. The tester fell back to the full suite, and the groups TIMED OUT after chewing through pgsql tests they had no business running. Change behavior to log at CRITICAL and sys.exit(1) in two cases: 1. TAP_GROUP is set, groups.json is present, but the group has no tests registered. 2. TAP_GROUP is set and groups.json is missing entirely. The "no TAP_GROUP specified" path is unchanged — that's the legitimate way to run the full suite directly (outside the group runner). Error messages explicitly tell the operator what to do: either register tests for the group, or unset TAP_GROUP to run the full suite intentionally.	1 month ago
Rene Cannao	8c2275178a	test: fail proxysql-tester.py TAP runs when zero binaries are discovered ... Defence-in-depth against the silent false-green failure mode we just tracked down. Previously, if CI-builds produced no TAP test binaries on the host (for any reason -- wrong make target, broken volume mount, typo in a docker-compose override, misconfigured groups.json), the tester would walk three workdirs, find zero `*-t` binaries in each, report: SUMMARY: 'tests' PASS 0/0 : FAIL 0/0 : SKIP 0/0 SUMMARY: 'deprecate_eof_support' PASS 0/0 : FAIL 0/0 : SKIP 0/0 SUMMARY: 'unit' PASS 0/0 : FAIL 0/0 : SKIP 0/0 SUMMARY: ret_rc = [0] ...and exit 0. Zero failures out of zero tests is technically "success" by the old logic, so the workflow would turn green. Combined with a build-target regression on the GH-Actions branch, this masked the fact that the entire TAP test suite (CI-legacy-g*, CI-mysql84-g*, CI-basictests, CI-taptests-pgsql-cluster, CI-legacy-clickhouse-g1, CI-legacy-g2-genai) had been running zero tests per commit for several weeks while still reporting green. Safety net ---------- At the end of `run_tap_tests`, check whether `glob(*-t)` found ANY binaries across ALL workdirs. The summary tuple (cmd, rc) is populated for every discovered binary, including ones that end up skipped by version filter or INCL/EXCL regexes -- those get `rc=None` rather than being absent. So `len(ret_summary) == 0` is a strict "nothing was even on disk" signal. When that happens, check groups.json for how many tests TAP_GROUP is expected to run. If N > 0, log a SAFETY NET FAIL line and bump ret_rc to 1. Version-filter skips and INCL/EXCL skips are not affected -- they still produce summary entries, so ret_summary is non-empty in those cases. Only the actual "test binaries were never built" case trips the guard. If TAP_GROUP is unset (running the full TAP suite), any zero-discovery result is a failure regardless -- there's no legitimate configuration where proxysql-tester should run zero tests. This complements the ci-builds.yml fix on the GH-Actions branch that restores the TAP build target. With both in place, a future regression in either the build or the test discovery chain surfaces loudly instead of silently.	2 months ago
Rahim Kanji	625e068412	Add @proxysql_min_version tag support for version-aware test execution ... Skip tests that require a newer ProxySQL version than the one being tested, preventing false-positive CI failures across v3.0.x/v3.1.x/4.0.x branches. Tagged 19 FFTO/TSDB tests (>=3.1) and 46 GenAI/MCP/AI tests (>=4.0) in groups.json. Tests without a tag run on all versions.	2 months ago
Rene Cannao	0441d5026d	security: fix all 48 Dependabot alerts — bump Python test deps ... Update all vulnerable Python packages in test infrastructure: Package Old New Severity CVEs PyMySQL 0.9.2 >=1.1.1 CRITICAL CVE-2024-36039 (SQL injection) requests 2.26.0 >=2.33.0 HIGH CVE-2024-35195, CVE-2024-47081, CVE-2023-32681, CVE-2026-25645 urllib3 1.26.6 >=2.6.3 HIGH CVE-2023-43804, CVE-2023-45803, CVE-2024-37891, CVE-2025-50181, CVE-2025-66418, CVE-2025-66471, CVE-2026-21441 certifi 2021.5.30 >=2024.7.4 HIGH CVE-2022-23491, CVE-2023-37920, CVE-2024-39689 idna 2.10 >=3.7 MEDIUM CVE-2024-3651 Changes: - requirements.txt: bump all 6 packages, use >= instead of == - requirements-ubuntu24.txt: same bumps - Pipfile: bump pymysql/requests, update python_version 2.7 → 3 - Remove stale Pipfile.lock (generated for Python 2.7, all hashes outdated) API compatibility verified: - PyMySQL: pymysql.connect(), DictCursor, OperationalError, CLIENT.MULTI_STATEMENTS, defer_connect all work in 1.1.x. `passwd` kwarg is aliased to `password` internally. - requests: basic requests.post() usage unchanged - urllib3/certifi/idna: transitive deps, no direct API usage Note: CI Docker image (ubuntu:24.04) installs packages via apt-get, not pip. These requirements files are used for local dev and some CI fallback paths.	2 months ago
Rene Cannao	1ebe4de9b7	Fix mysql84 pre-proxysql hook, basictests setup, and move flaky tests ... - mysql84 pre-proxysql.bash: use docker exec for all ProxySQL admin commands (was using mysql -h127.0.0.1 which can't reach the container) - basictests setup-infras.bash: use INSERT OR REPLACE when remapping hostgroups 1300/1301 -> 0/1 to avoid UNIQUE constraint violation when target hostgroups already have entries - proxysql-tester.py: skip per-stage coverage generation when MULTI_GROUP=1 (was spawning 32 gcov processes on 35k .gcno files) - Move test_cluster_sync-t and test_read_only_actions_offline_hard_servers-t to legacy-g5 pending investigation of cluster sync algorithm issues. Move reg_test_3765_ssl_pollout-t from legacy-g5 to legacy-g4.	2 months ago
Rene Cannao	d0872851a6	Enable coverage when either WITHGCOV or COVERAGE_MODE is set ... proxysql-tester.py only checked WITHGCOV to decide whether to run PROXYSQL GCOV DUMP. Since WITHGCOV now defaults to 0, passing COVERAGE=1 to run-multi-group.bash had no effect — the tester skipped coverage collection. Now checks both WITHGCOV and COVERAGE_MODE (which is passed into the container by run-tests-isolated.bash). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2 months ago
Rene Cannao	9cf64fd7ff	Fix failover test: use DEFAULT_MYSQL_INFRA for orchestrator DNS names ... The failover test had hardcoded 'infra-mysql57' hostnames and short aliases like 'mysql1' that don't resolve in the isolated Docker network. Now uses DEFAULT_MYSQL_INFRA env var to construct the correct DNS names (e.g. orc1.infra-mysql57, mysql1.infra-mysql57). All three basictests now pass locally: benchmark, change_user, failover. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2 months ago
Rene Cannao	b93a1a164c	Remove jenkins-build-scripts dependencies from test scripts ... - Update proxysql-tester.yml: benchmark script path uses $WORKSPACE - Remove all JENKINS_SCRIPTS_PATH references from proxysql-tester.py - Use TAP env vars in local-docker-benchmark.bash for host/port/password Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2 months ago
Rene Cannao	c3004c1039	Fix code coverage collection for ProxySQL core code ... Two fixes required: 1. Mount gcov directory in test runner container so it can access the .gcda files written by ProxySQL 2. Call 'PROXYSQL GCOV DUMP' after each TAP test to flush coverage counters from memory to disk Without these fixes, coverage data was never flushed from the running ProxySQL process, resulting in 0% coverage for ProxySQL core files.	2 months ago
Rene Cannao	be290c5aeb	Address PR review feedback on test infrastructure ... - Remove redundant host-side coverage collection in run-tests-isolated.bash (coverage is already collected via trap inside the container) - Initialize TAP_GROUP, group_has_tests, groups before try block in proxysql-tester.py to avoid UnboundLocalError on early exceptions - Differentiate skipped tests from missing tests in TAP_GROUP validation so intentionally skipped tests don't count as missing	2 months ago
Rene Cannao	0bc75cd589	Fix test validation and coverage HTML generation ... - Fix false positive "missing tests" errors when processing secondary workdirs (e.g., deprecate_eof_support). The validation now only checks tests that actually exist in the current workdir rather than all tests in groups.json for the TAP_GROUP. - Add --ignore-errors negative,source --synthesize-missing flags to genhtml calls in run-tests-isolated.bash to match the multi-group runner and prevent HTML generation failures on coverage data with negative branch counts.	2 months ago
Rene Cannao	5004d827b1	Fix proxysql-tester.py to fail when expected tests don't run	2 months ago
Rene Cannao	33aae3b1c2	Fix reg_test_3223 RESTAPI scripts path and add group env inheritance ... Fix reg_test_3223-restapi_return_codes-t failure by properly configuring REGULAR_INFRA_DATADIR in group environment files: 1. test/tap/groups/legacy/env.sh: Add REGULAR_INFRA_DATADIR pointing to test/tap/tests where reg_test_3223_scripts directory exists 2. test/tap/groups/mysql84/env.sh: Same REGULAR_INFRA_DATADIR setting 3. Create env.sh for feature-specific groups: - mysql-auto_increment_delay_multiplex=0 - mysql-multiplexing=false - mysql-query_digests=0 - mysql-query_digests_keep_comment=1 4. test/scripts/bin/proxysql-tester.py: Add fallback logic to load base group env.sh when subgroup env.sh doesn't exist (e.g., legacy-g1 falls back to legacy). This follows the same pattern as ensure-infras.bash. This follows the architectural principle that test-specific setup belongs in TAP group configuration, not in the generic test runner.	2 months ago
Rene Cannao	e96f02083e	Fix indentation in proxysql-tester.py	2 months ago
Rene Cannao	00b87013cc	Migrate and localize proxysql-tester.py logic ... - Migrate test-scripts from jenkins-build-scripts to test/scripts/. - Refactor proxysql-tester.py to skip tests missing from groups.json instead of failing. - Ensure correct path resolution and dependency links in localized environment. - Disable automatic pre-hook execution in the Python tester.	2 months ago