proxysql

Commit Graph

Author	SHA1	Message	Date
Rahim Kanji	e7ee68dda5	Add SSL/TLS traffic decryption for PostgreSQL backend connections (#5281 ) Extends the existing NSS keylog-based SSL decryption to PostgreSQL backend connections by patching libpq with a global PQsetSSLKeyLogCallback() API. When admin-ssl_keylog_file is configured, TLS secrets from all PgSQL backend SSL handshakes are written to the keylog file in NSS Key Log Format, enabling traffic decryption with Wireshark/tshark. Approach: - Patch libpq with global callback following the PQsslKeyPassHook pattern - Single startup call covers all backend paths (regular, monitor, kill, harvester) - No new admin variables — existing admin-ssl_keylog_file covers PgSQL backends - Zero changes to ProxySQL PgSQL connection code Includes 9 integration tests (pgsql-ssl_keylog-t) and user guide updates.	1 month ago
Rene Cannao	b2f4b4f9a0	Fix crash when COM_BINLOG_DUMP received with idle backends from other hostgroups (#5556 ) When a session has backend connections to multiple hostgroups and receives COM_BINLOG_DUMP, GPFC_Replication_SwitchToFastForward() transitions the session to FAST_FORWARD mode but only manages the active backend. Idle backends from other hostgroups remain in the session's mybes array with async_state_machine==ASYNC_IDLE. If MySQL's wait_timeout closes one of these idle connections, the resulting POLLIN event causes handler() to be called with ASYNC_IDLE, which has no case in the switch and hits assert(0). Fix: Clean up idle backends from other hostgroups during the FAST_FORWARD transition in GPFC_Replication_SwitchToFastForward(). These connections will never be used again, so they are destroyed immediately. Includes a regression test (test_binlog_dump_multi_backend_crash-t) that: - Sets up two hostgroups with query rules routing SELECTs to reader HG - Creates backend connections to both hostgroups on one session - Sends COM_BINLOG_DUMP to enter FAST_FORWARD mode - Waits for wait_timeout to close the idle backend - Verifies ProxySQL does not crash	2 months ago
René Cannaò	d0625dc243	Merge pull request #5549 from sysown/v3.0-CodeCov0325 Add 32 unit test files (2963 assertions) and CI workflow improvements	2 months ago
Rene Cannao	acaa100dc8	fix: address PR review feedback from coding agents Actionable fixes from CodeRabbit / agent reviews on PR #5549: 1. include/Stats_Tool_Handler.h: add #include <cstdint> — uint64_t used in parse_digest_filter() without direct include 2. lib/MCP_Endpoint.cpp: use strncasecmp() for Bearer token prefix check — RFC 7235 Section 2.1 says auth-scheme is case-insensitive. This is a real bug fix in production code. 3. test/tap/tests/unit/sqlite3db_unit-t.cpp: guard result->rows_count dereference with null check to avoid segfault if execute_statement returns null 4. test/tap/tests/unit/genai_mcp_endpoint_unit-t.cpp: remove memset() calls on MCP_Threads_Handler that leaked constructor's strdup("") allocations for auth fields 5. test/tap/groups/check_groups.py: use os.walk() instead of hardcoded directory list so new test subdirectories are discovered 6. test/tap/tests/unit/admin_disk_upgrade_unit-t.cpp: add comment documenting intentional calloc pattern (ProxySQL_Admin constructor has too many daemon deps for unit tests)	2 months ago
Rene Cannao	4fb7aac73f	fix: restore groups.json to compact one-line-per-entry format Previous commits inadvertently reformatted groups.json with Python's json.dump(indent=2), breaking the compact format required by test/tap/groups/README.md. The correct format is one entry per line with compact arrays so grep works for quick lookups.	2 months ago
Rene Cannao	e806b0219c	test: register 5 new unit tests in groups.json for CI	2 months ago
Rene Cannao	8df688dd8a	test: register 5 new unit tests in groups.json for CI	2 months ago
Yuji Hatakeyama	eda537ac6c	Add reg_test_5306 to test groups, excluding digest-disabled configs	2 months ago
Rene Cannao	093693ba06	test: add check_groups.py to verify groups.json completeness Python tool that checks consistency between groups.json and compiled TAP test binaries: - WARNING (exit 0): tests in groups.json that aren't compiled yet - ERROR (exit 1): compiled tests missing from groups.json Scans test/tap/tests/, test/tap/tests/unit/, and test/tap/tests_with_deps/ for ELF binaries matching *-t pattern. Usage: python3 test/tap/groups/check_groups.py	2 months ago
Rene Cannao	fa668d646a	test: register 21 new unit tests in groups.json for CI Add all new unit test binaries to the CI test group configuration: - 13 core tests → unit-tests-g1 - 8 GenAI tests → ai-g1 This ensures CI pipelines discover and run all new tests.	2 months ago
Rene Cannao	1ebe4de9b7	Fix mysql84 pre-proxysql hook, basictests setup, and move flaky tests - mysql84 pre-proxysql.bash: use docker exec for all ProxySQL admin commands (was using mysql -h127.0.0.1 which can't reach the container) - basictests setup-infras.bash: use INSERT OR REPLACE when remapping hostgroups 1300/1301 -> 0/1 to avoid UNIQUE constraint violation when target hostgroups already have entries - proxysql-tester.py: skip per-stage coverage generation when MULTI_GROUP=1 (was spawning 32 gcov processes on 35k .gcno files) - Move test_cluster_sync-t and test_read_only_actions_offline_hard_servers-t to legacy-g5 pending investigation of cluster sync algorithm issues. Move reg_test_3765_ssl_pollout-t from legacy-g5 to legacy-g4.	2 months ago
Rene Cannao	7739b86745	Move test_auth_methods-t from legacy-g2 to mysql84-g2 only Test requires MySQL 8.0+ (detects 5.7 and skips, but crashes with 'terminate called without an active exception' during noise thread cleanup). The crash-on-skip is a separate test bug.	2 months ago
Rene Cannao	42f4507e72	Fix test_tls_stats plan count and move caching_sha2 to mysql84 - test_tls_stats-t: plan(29) -> plan(27). The test executes 27 ok() calls on the happy path; plan was miscounted during authoring. - reg_test_4935-caching_sha2-t: remove from legacy-g4 (MySQL 5.7 doesn't support caching_sha2_password). Keep in mysql84-g4.	2 months ago
René Cannaò	d1c86b141f	Merge pull request #5525 from sysown/v3.0-ci260322 Migrate CI-basictests from jenkins-build-scripts to in-repo test/infra	2 months ago
Rene Cannao	ba309e9c89	Fix groups.json: restore compact one-line-per-entry format	2 months ago
Rene Cannao	801a44399a	Split clickhouse tests into separate legacy-clickhouse group Move 3 clickhouse TAP tests (clickhouse_php_conn-t, test_clickhouse_server-t, test_clickhouse_server_libmysql-t) from legacy-g1/g3 to legacy-clickhouse-g1. Remove infra-clickhouse23 from legacy/infras.lst. This eliminates the clickhouse container from all legacy-g1..g4 runs, reducing container count per group.	2 months ago
Rene Cannao	6769ea4845	Make legacy/pre-proxysql.bash a no-op	2 months ago
Rene Cannao	6fcb8fd631	Run ProxySQL cluster nodes inside a single container Instead of spawning 1+9 Docker containers for the ProxySQL cluster, run all instances as background processes inside the primary container. Each node gets its own port pair (admin=6032+i10, mysql=6033+i10) and data directory. This eliminates 9 Docker containers per group, avoiding Docker networking/iptables contention that caused "no route to host" errors under parallel test execution. Changes: - start-proxysql-isolated.bash: starts cluster nodes as background processes inside the container, then initializes the cluster and installs the scheduler — all in one script - check_all_nodes.bash: polls 127.0.0.1 on different ports instead of container hostnames - env-isolated.bash: TAP_CLUSTER_NODES uses proxysql:PORT format - default/pre-proxysql.bash, legacy/pre-proxysql.bash: no longer call cluster_start.bash/cluster_init.bash (handled by start-proxysql) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2 months ago
Rene Cannao	09e9207da9	Move reg_test_3765_ssl_pollout-t from legacy-g2 to legacy-g5 This test does 122-second CPU measurement waits, making it too slow for the legacy-g2 group which needs to complete in a reasonable time.	2 months ago
Rene Cannao	f2db55f0c8	Fix basictests hostgroup hook: move instead of copy to avoid monitor conflicts Previously the hook copied servers from 1300/1301 into 0/1, leaving both pairs active. The replication hostgroup monitor managed both pairs simultaneously, causing race conditions that removed servers from 0/1. Now moves servers in-place (UPDATE) and deletes the original 1300/1301 replication entry, so only one pair (0/1) exists for the monitor. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2 months ago
Rene Cannao	d1d10188cc	Register PgSQL FFTO tests in Makefile and groups.json (#5517 ) Add Makefile rules for the two pg_lite_client-based tests: - test_ffto_pgsql_pipeline-t (links pg_lite_client.cpp) - test_ffto_pgsql_stmt_portal-t (links pg_lite_client.cpp) The other 5 PgSQL FFTO tests use the default wildcard rule (%-t: %-t.cpp) and don't need explicit Makefile entries. Register all 7 new PgSQL FFTO tests in groups.json under legacy-g4 and mysql84-g4 (same groups as test_ffto_pgsql-t): - test_ffto_pgsql_command_types-t - test_ffto_pgsql_concurrent-t - test_ffto_pgsql_errors-t - test_ffto_pgsql_large_resultsets-t - test_ffto_pgsql_mixed_protocol-t - test_ffto_pgsql_pipeline-t - test_ffto_pgsql_stmt_portal-t Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2 months ago
Rene Cannao	eb487560ab	Fix mysql84+ pre-proxysql.bash: remove infra-default, fix infra paths - Remove references to non-existent infra-default/docker-compose-destroy.bash - Fix paths from test/infra/control/${INFRA}/ to test/infra/${INFRA}/ (infra directories live under test/infra/, not test/infra/control/) Affects: mysql84, mysql84-gr, mysql90-mysql93, mysql90-gr-mysql93-gr Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2 months ago
Rene Cannao	29fec9d9ff	Add hostgroup aliasing hook for basictests group The in-repo infra uses hostgroup pairs like 1300/1301, but the legacy proxysql-tester.py benchmark validates against hostgroups 0 and 1. This setup hook copies the first hostgroup pair into 0/1 and remaps users and query rules accordingly. Benchmark and change_user tests now pass locally. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2 months ago
Rene Cannao	0c58d1c645	Fix REPO_ROOT derivation: 4 levels up from test/tap/groups/<name>/ Scripts are at test/tap/groups/<name>/pre-proxysql.bash, need 4 parent traversals to reach repo root, not 3. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2 months ago
Rene Cannao	75d958f29d	Fix hardcoded /home/rene/proxysql paths in pre-proxysql.bash scripts Replace hardcoded absolute paths with dynamically derived REPO_ROOT so these scripts work on any machine (CI runners, other developers). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2 months ago
Rene Cannao	7949016f62	Register new FFTO E2E tests in TAP groups (#5516 ) Add the 6 new FFTO E2E TAP test binaries to groups.json so they are discovered and executed by the TAP test runner infrastructure. All new tests are assigned to the same groups as the existing FFTO tests: legacy-g4 and mysql84-g4. These groups provide the required infrastructure (running ProxySQL + MySQL backend with fast_forward user configuration). New entries: - test_ffto_mysql_bypass_recovery-t (bypass stickiness + isolation) - test_ffto_mysql_concurrent-t (10-thread concurrent sessions) - test_ffto_mysql_large_queries-t (large SQL text + rapid fire) - test_ffto_mysql_large_resultsets-t (10K+ rows + threshold tests) - test_ffto_mysql_mixed_protocol-t (COM_QUERY + COM_STMT_EXECUTE) - test_ffto_mysql_transactions-t (BEGIN/COMMIT/ROLLBACK + PS×20) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2 months ago
Rene Cannao	5d02ad5faf	Add basictests TAP group and sysbench to CI base image Part of the CI migration from jenkins-build-scripts to in-repo test/infra/ infrastructure (#5522). Adds: - test/tap/groups/basictests/ group (benchmark, chuser, failover tests) - sysbench to docker-base Dockerfile for benchmark tests Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	2 months ago
René Cannaò	4d2bcc303e	Merge remote-tracking branch 'origin/v3.0' into copilot/feature-load-restapi-routes-config # Conflicts: # test/tap/groups/groups.json	2 months ago
René Cannaò	cb8d0aa737	Merge pull request #5456 from sysown/copilot/add-ssl-tls-certificate-stats-table feat: Add stats_tls_certificates and stats_global tables for TLS certificate metadata and tracking	2 months ago
René Cannaò	da13ab59e5	Merge remote-tracking branch 'origin/v3.0' into copilot/add-ssl-tls-certificate-stats-table # Conflicts: # test/tap/groups/groups.json	2 months ago
René Cannaò	9781f5992d	Merge remote-tracking branch 'origin/v3.0' into copilot/feature-load-restapi-routes-config	2 months ago
René Cannaò	a616b34b95	Register test_tls_stats-t in groups.json (g4 groups)	2 months ago
René Cannaò	d1bb48e444	Register test_load_restapi_from_config_startup-t in groups.json (g4 groups)	2 months ago
René Cannaò	ceff98eeb1	Add README.md for TAP test groups directory Documents groups.json format (one entry per line, compact arrays), group naming conventions, directory structure, and infrastructure-free groups.	2 months ago
René Cannaò	4c4736c73d	Add SKIP_PROXYSQL support for infrastructure-free test groups Unit tests run standalone against libproxysql.a with stubs and need no ProxySQL instance or backend databases. This adds a SKIP_PROXYSQL flag that test groups can set in their env.sh to bypass all Docker infrastructure setup and run tests directly on the host. Changes: - ensure-infras.bash: source group env.sh early; exit immediately when SKIP_PROXYSQL=1 - run-tests-isolated.bash: source group env.sh early; skip ProxySQL and backend verification; run proxysql-tester.py directly on host instead of inside a Docker container - New test/tap/groups/unit-tests/env.sh with SKIP_PROXYSQL=1 - groups.json: add 18 unit test binaries to unit-tests-g1 group - Documentation: SKIP_PROXYSQL in env var tables and new section for infrastructure-free groups in test/infra/README.md	2 months ago
René Cannaò	abd1da861f	Merge pull request #5470 from sysown/private/multi-group-runner Multi-group TAP test runner and ClickHouse test fixes	2 months ago
Rene Cannao	60b5aff5c4	Fix tests not running after infrastructure setup in multi-group runner The heredoc in run-multi-group.bash was causing bash to exit prematurely after ensure-infras.bash completed, without executing the test commands. This was due to shell expansion issues with the heredoc structure. Changes: - Replace heredoc with temp file approach in run-multi-group.bash - Add pre-proxysql hook call to ensure-infras.bash for cluster setup - Create legacy/pre-proxysql.bash for cluster node startup - Update mysql84/pre-proxysql.bash with cluster startup calls - Fix coverage collection to work correctly in parallel execution	2 months ago
Rene Cannao	4dcabbc7ec	Move test_utf8mb4_as_ci-4841-t to mysql84-g1 group only This test requires MySQL 8.0+ for utf8mb4_0900_as_ci collation support. Removed from legacy-g4 and all g4 variants since they don't have MySQL 8.	2 months ago
Rene Cannao	47907c2ead	Move test_client_limit_error-t to todo-g1 group This test uses network namespaces which don't work in Docker containers. Moving to todo-g1 for later review.	2 months ago
Rene Cannao	758b8116c8	Fix reg_test_3223 RESTAPI test by adding TAP group setup hook Fix reg_test_3223-restapi_return_codes-t failure: 1. Created test/tap/groups/legacy/setup-infras.bash: - TAP group hook that copies RESTAPI scripts to ProxySQL data directory - Sets execute permissions on all scripts EXCEPT script_no_permissions - Keeps script_no_permissions without execute permissions for EACCES test 2. Modified test/tap/groups/legacy/env.sh: - Added REGULAR_INFRA_DATADIR=/var/lib/proxysql - Points to where scripts are mounted inside ProxySQL container This follows the architectural principle that test-specific setup belongs in TAP group hooks, not in the generic test runner or infrastructure scripts.	2 months ago
Rene Cannao	33aae3b1c2	Fix reg_test_3223 RESTAPI scripts path and add group env inheritance Fix reg_test_3223-restapi_return_codes-t failure by properly configuring REGULAR_INFRA_DATADIR in group environment files: 1. test/tap/groups/legacy/env.sh: Add REGULAR_INFRA_DATADIR pointing to test/tap/tests where reg_test_3223_scripts directory exists 2. test/tap/groups/mysql84/env.sh: Same REGULAR_INFRA_DATADIR setting 3. Create env.sh for feature-specific groups: - mysql-auto_increment_delay_multiplex=0 - mysql-multiplexing=false - mysql-query_digests=0 - mysql-query_digests_keep_comment=1 4. test/scripts/bin/proxysql-tester.py: Add fallback logic to load base group env.sh when subgroup env.sh doesn't exist (e.g., legacy-g1 falls back to legacy). This follows the same pattern as ensure-infras.bash. This follows the architectural principle that test-specific setup belongs in TAP group configuration, not in the generic test runner.	2 months ago
Rene Cannao	679d758e24	Fix ai-g1 env.sh to be POSIX sh compatible The Python tester sources env files using /bin/sh, but the previous implementation used bash-specific features (BASH_SOURCE[0] and source). Changed to inline the values from ai/env.sh for POSIX compatibility.	2 months ago
Rene Cannao	2f8e63cef2	Address code review feedback for AI TAP group migration Fixes from PR #5461 review: 1. Fix BASE_GROUP derivation inconsistency in run-tests-isolated.bash - Changed from bash pattern matching `${TAP_GROUP%%-g[0-9]}` to sed - Now uses same pattern as ensure-infras.bash: `sed -E "s/[-_]g[0-9]+.//"` - Removed redundant BASE_GROUP reassignment 2. Update outdated comments in seed files - seed-mysql.sql and seed-pgsql.sql now correctly reference setup-infras.bash 3. Change INSERT OR IGNORE to INSERT OR REPLACE in mcp-config.sql - Ensures credentials are updated on reruns for deterministic state 4. Add shellcheck directives to env.sh files - Added `# shellcheck shell=bash` to ai/env.sh and ai-g1/env.sh 5. Add explicit validation for derived infrastructure names - setup-infras.bash now validates DEFAULT_MYSQL_INFRA and DEFAULT_PGSQL_INFRA - Provides clear error messages if infras.lst is misconfigured 6. Add WORKSPACE export to README examples - All runnable examples now include `export WORKSPACE=$(pwd)` Test verification: - ai-g1 group infrastructure setup works correctly - legacy-g2 group tests still pass (backward compatibility verified)	2 months ago
Rene Cannao	0b9806c46e	Migrate AI TAP group to Unified CI Infrastructure pattern This commit migrates the ai TAP group from its legacy hook-based approach to the standard test/infra/ Unified CI Infrastructure pattern. Changes: - Added generic hook system to infrastructure scripts: * ensure-infras.bash: Executes group-specific setup-infras.bash hook * run-tests-isolated.bash: Executes group-specific pre-cleanup.bash hook - Migrated ai group infrastructure: * Created infras.lst with infra-mysql84 and docker-pgsql16-single * Renamed seed files: mysql-seed.sql -> seed-mysql.sql, pgsql-seed.sql -> seed-pgsql.sql * Deleted legacy docker-compose.yml, docker-compose-init.bash, docker-compose-destroy.bash * Deleted legacy hooks: pre-proxysql.bash, post-proxysql.bash - Added group-specific MCP configuration: * setup-infras.bash: Configures MCP targets and seeds test data * pre-cleanup.bash: Removes MCP configuration after tests * mcp-config.sql: SQL template for MCP setup * cleanup.sql: SQL template for MCP cleanup * seed-mysql.sql & seed-pgsql.sql: Test data for AI tests - Updated documentation in README.md with usage examples Architecture: - ai is the supergroup (defines infrastructure and shared config) - ai-g1, ai-g2, etc. are subgroups (define test sets in groups.json) - Group-specific hooks live in test/tap/groups/ai/ directory - Infrastructure scripts remain generic and delegate to group hooks	2 months ago
René Cannaò	1a0dea6da0	Merge pull request #5457 from sysown/copilot/fix-admin-credentials-logging Redact `admin-admin_credentials` from admin SET command logs	2 months ago
Rene Cannao	ffb610f4b8	Fix no-infra TAP group setup	2 months ago
Rene Cannao	4517edea78	Address review feedback and create no-infra TAP group - Fix retry loop in admin_set_credentials_logging-t.cpp to clear eofbit (CodeRabbit) - Fix spelling in lib/Admin_Handler.cpp (CodeRabbit) - Create new 'no-infra-g1' TAP group in groups.json - Add 'tests_no_infra' target to Makefiles to run tests without backends	2 months ago
René Cannaò	458ef01970	Merge pull request #5453 from sysown/v3.0-ci_84 [WIP] multiple changes related to CI	2 months ago
René Cannaò	8c4252814f	Merge branch 'v3.0-ci_84' into v3.0-ci_84_mysql84 Signed-off-by: René Cannaò <rene@proxysql.com>	2 months ago
Rahim Kanji	2765c3a9c7	Added mysql84-* group	2 months ago

1 2 3 4

165 Commits (v3.0_pgsql_sslkeylog_5281)