proxysql

Commit Graph

Author	SHA1	Message	Date
René Cannaò	93eabfc7bc	fix: address code review issues in PostgreSQL cluster sync - Fix fetch_and_store incrementing success counter on failure - Fix sqlite3_stmt leak in pull_pgsql_variables_from_peer (use RAII) - Convert get_peer_to_sync_pgsql_query_rules and _servers_v2 to unified framework delegates (fixes atomic data race on .load()) - Merge dual scan loops in set_checksums to respect enabled_check - Remove spurious "deprecated" label from new checksum_pgsql_variables - Replace sprintf with snprintf in pgsql diffs_before_sync getters - Fix indentation on cluster_pgsql_variables blocks - Fix TAP plan mismatch: replace MYSQL_QUERY with inline mysql_query - Fix trivially-true table accessibility assertions - Add mysql_close on failed connect in test - Move test_cluster_sync_pgsql-t to legacy-g5 group	2 months ago
René Cannaò	8a61232517	fix: decouple MySQL and PostgreSQL checksum/sync controls Two bugs in the PostgreSQL cluster sync admin variables: 1. Cross-protocol coupling (critical): Disabling a MySQL checksum flag (e.g. SET admin-checksum_mysql_query_rules='false') incorrectly zeroed out the corresponding PostgreSQL cluster sync threshold (cluster_pgsql_query_rules_diffs_before_sync). An operator disabling MySQL checksums would unexpectedly halt PostgreSQL cluster sync. Fix: MySQL checksum setters now only affect MySQL sync thresholds. The warning messages no longer mention pgsql side effects. 2. Single-gate for all pgsql modules (operational limitation): All four PostgreSQL cluster sync thresholds (cluster_pgsql_{query_rules,servers,users,variables}_diffs_before_sync) were gated by a single flag (checksum_pgsql_variables). This meant you could not independently control checksum computation per-module. By contrast, MySQL has per-module flags (checksum_mysql_query_rules, checksum_mysql_servers, checksum_mysql_users, checksum_mysql_variables). Fix: Add three new per-module admin variables: - admin-checksum_pgsql_query_rules - admin-checksum_pgsql_servers - admin-checksum_pgsql_users Each gates only its corresponding cluster_*_diffs_before_sync setter, mirroring the MySQL pattern. The existing admin-checksum_pgsql_variables now only gates cluster_pgsql_variables_diffs_before_sync. Disabling admin-checksum_pgsql_variables no longer cascades to zero out query_rules/servers/users sync thresholds. Files changed: - include/proxysql_admin.h: Add checksum_pgsql_query_rules, checksum_pgsql_servers, checksum_pgsql_users to checksum_variables struct. - lib/ProxySQL_Admin.cpp: Add variable names, constructor defaults, get/set handlers for new flags. Update diffs_before_sync guards to use per-module flags. Remove pgsql zeroing from MySQL checksum setters. Fix checksum_pgsql_variables setter to only affect pgsql variables sync threshold.	2 months ago
Rene Cannao	75ce817577	ci: add workflows for legacy-g1/g3/g5 and mysql84-g1 through g5 Add dedicated CI workflows for test groups that were missing. All use the generic ci-taptests-groups.yml reusable workflow on GH-Actions with the testgroup parameter set to the specific group.	2 months ago
Rene Cannao	38613e9acc	fix: handle v-prefix in GIT_VERSION when extracting CURVER The CURVER extraction regex assumed GIT_VERSION starts with digits, but git describe produces 'v3.0.8-66-gb24f9ca' with a 'v' prefix. When the Makefile is invoked in contexts where GIT_VERSION_NORM normalization is skipped (MAKELEVEL != 0), CURVER was empty and the $(error) validation halted the build. Add 'v?' to the sed pattern to handle both prefixed and unprefixed versions.	2 months ago
Rene Cannao	b24f9ca917	fix: add missing closing paren to Makefile $(error) call The previous fix removed the parenthesized text but didn't keep the closing ')' that terminates the $(error) function call.	2 months ago
Rene Cannao	edca171b2b	fix: escape parenthesis in Makefile error message that broke CI-builds The $(error ...) call on line 81 contained a literal ')' in "(expected format: X.Y.Z)" which make interprets as the closing delimiter of the $(error) function. This caused all docker-compose build targets (ubuntu22-dbg, ubuntu22-tap, etc.) to fail with "unterminated call to function 'error': missing ')'". This was the root cause of CI-builds failing since Release 3.0.8.	2 months ago
Rene Cannao	3f87479225	revert: remove unnecessary SYNC_TIMEOUT increase and legacy-g5 override The SYNC_TIMEOUT increase (10→30) and PROXYSQL_CLUSTER_NODES=1 override were based on wrong diagnosis. The actual root cause of the cluster isolation timeout was a GITVERSION mismatch in the binary from incremental builds — different .o files had different version strings, causing the cluster monitor to refuse syncing. With a clean build (as in CI), the default SYNC_TIMEOUT=10 and default cluster node count work fine.	2 months ago
Rene Cannao	94de63dbb7	fix: correct log message variable name for V2 modules and reduce cluster interference 1. Fix DIFFS_BEFORE_SYNC_FORMAT log message: mysql_servers_v2 and pgsql_servers_v2 modules logged non-existent variable names like 'admin-cluster_mysql_servers_v2_diffs_before_sync'. Added sync_var_suffix field to ChecksumModuleInfo so V2 modules use the correct variable name (e.g. 'mysql_servers'). 2. Increase SYNC_TIMEOUT from 10 to 30 seconds in test_cluster_sync-t to handle slower sync propagation with multiple cluster nodes. 3. Add legacy-g5 group env.sh with PROXYSQL_CLUSTER_NODES=1 to reduce cluster node interference during cluster isolation steps. These tests manage their own replicas and multi-node sync creates race conditions where removed proxysql_servers entries get re-added.	2 months ago
Rene Cannao	a7db1eb4a8	fix: correct 'Not syncing' log message variable name for V2 modules The DIFFS_BEFORE_SYNC_FORMAT log message used module_name directly to construct the admin variable name, producing incorrect names like 'admin-cluster_mysql_servers_v2_diffs_before_sync' (doesn't exist). The actual variable is 'admin-cluster_mysql_servers_diffs_before_sync'. Add sync_var_suffix field to ChecksumModuleInfo so V2 modules (mysql_servers_v2, pgsql_servers_v2) can specify the correct variable name suffix. This fixes test_cluster_sync-t log regex checks that verify sync-disable behavior.	2 months ago
Rene Cannao	1c5e03f354	fix: use dynamic MySQL host/port in test_cluster_sync_mysql_servers The test hardcoded 127.0.0.1:13306-13309 as MySQL backend addresses. In the Unified CI, MySQL backends run in separate containers reachable by hostname (e.g. mysql1.infra-mysql57:3306), not on localhost ports. Use cl.mysql_host and cl.mysql_port from environment variables instead of hardcoded addresses. Non-ONLINE servers use offset ports as dummy entries since they're only sync test data and never connected to. Results: 44/44 tests pass (was 4/44 with immediate replica shutdown).	2 months ago
Rene Cannao	1232948fcb	fix: resolve cross-container networking for cluster sync tests The cluster sync tests spawn ProxySQL replicas inside the test-runner container. In the Unified CI, the primary ProxySQL runs in a separate container. The primary could not reach replicas registered as 127.0.0.1 because that resolves to the primary's own loopback. Fix: - Add --hostname test-runner to test-runner container so it has a stable DNS name on the Docker network - Add get_cluster_visible_host() to both cluster sync tests that returns the container hostname (resolvable by other containers) - Use this hostname for replica connection opts so proxysql_servers registrations and stats_proxysql_servers_checksums lookups work cross-container Results: - test_cluster_sync-t: 397/399 pass (was 21/399) - test_cluster_sync_mysql_servers-t: replicas now start and connect correctly, but further investigation needed for cluster node interference causing premature SHUTDOWN	2 months ago
Rene Cannao	69437751fd	fix: prevent cluster sync tests from hanging on replica shutdown failure Both test_cluster_sync-t and test_cluster_sync_mysql_servers-t launch ProxySQL replicas via system() in threads. If PROXYSQL SHUTDOWN fails or the admin connection is NULL, the process never exits and thread.join() hangs forever. Fix: replace system() with fork()/exec() to track the child PID. Before join(), wait up to 5 seconds for the process to exit, then SIGKILL it as a fallback. This guarantees the test always terminates.	2 months ago
Rene Cannao	faa1054674	Merge remote-tracking branch 'origin/v3.0' into fix/postgresql-cluster-sync_2	2 months ago
Rene Cannao	7e94ff904e	test: remove unnecessary fresh_connection workaround from pgsql cluster sync test The test was opening a new admin connection for every replica query, working around a supposed visibility issue where persistent sessions could not see synced pgsql data. Live testing confirms both persistent and fresh connections see identical data on the replica for all three modules (servers, users, query rules). The workaround is unnecessary.	2 months ago
Rene Cannao	ccf797a8c4	ci: rewrite pgsql cluster sync workflow to use Unified CI infra Replace the old jenkins-build-scripts based CI-taptests-pgsql-cluster workflow with the Unified CI infrastructure (test/infra/). This aligns it with all other test workflows. Changes: - Use ensure-infras.bash + run-tests-isolated.bash instead of jenkins-build-scripts cluster_start/cluster_init - Build proxysql-ci-base Docker image in the workflow - Pass TAP_PGSQL_SYNC_REPLICA_PORT through to the test container so the full cluster sync validation runs (not just checksums) - Set TAP_PGSQL_SYNC_REPLICA_PORT=6042 by default in env-isolated when cluster nodes are active - Proper cleanup with destroy-infras.bash on failure	2 months ago
René Cannaò	21b3e238ee	Merge pull request #5579 from sysown/infra-mysql57-binlog CI: Fix `infra-mysql57-binlog` setup	2 months ago
Rene Cannao	f019e8ee66	ci: make test workflows wait for CI-builds before running All 24 test workflows were triggering on CI-trigger completion, which meant they could start before CI-builds finished producing the ubuntu22-tap artifact. Change them to trigger on CI-builds completion so test binaries are available when tests start. Unchanged: CI-builds (triggers on CI-trigger in_progress), CI-codeql (does its own build), CI-package-build (own trigger).	2 months ago
Rene Cannao	2588630faf	ci: add groups.json format linter and GitHub Actions workflow Add lint_groups_json.py that enforces the compact one-line-per-entry format documented in test/tap/groups/README.md: valid JSON, sorted keys, no multi-line arrays. The CI workflow triggers on push/PR when groups.json is modified.	2 months ago
Rene Cannao	f2ed9436ce	fix: reformat groups.json to compact one-line-per-entry style The merge resolution used json.dump with indent=4 which broke the required format (one entry per line, compact arrays). Reformat to match the convention documented in test/tap/groups/README.md.	2 months ago
Rene Cannao	822f4aaa6e	doc: remove stale pgsql cluster sync PR status document The pgsql_cluster_sync_pr5297_status.md was written during development and is now outdated — the TAP test passes fully with cluster sync validation including disk persistence.	2 months ago
Rene Cannao	47758a468d	merge: resolve groups.json conflict with v3.0 Take v3.0's groups.json as base (legacy-gN naming, mysql84-gN groups, new test entries) and add test_cluster_sync_pgsql-t from this branch. Drop duplicate pgsql_admin_metacmds-t (already exists as pgsql-admin_metacmds-t in v3.0).	2 months ago
Rene Cannao	8c1b401cb7	Release 3.0.8	2 months ago
Wazir Ahmed	fa772c40b0	CI: Fix `infra-mysql57-binlog` setup - Fix infra-mysql57-binlog setup - Rework `docker-orchestrator-post.bash` based on infra-mysql57 orchestrator script - Add `testuser` to ProxySQL infra config - Fix multi-group cleanup - Rename the `GROUPS` variable in `destroy-multi-group.bash` to avoid the Bash special variable collision - Stop tracking CI-generated artifacts - Remove `test_cluster_sync.cnf` and `test-scripts/deps/mysqlbinlog` from git index - Ignore `test-scripts/deps/` directory Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	2 months ago
René Cannaò	d7a26b79e9	Merge pull request #5577 from sysown/v3.0-ci0405 Fix CI test failures and workflow race conditions	2 months ago
Rene Cannao	78c8a3d5e3	Fail fast in purge-trigger setup if connection or stmt fails Address review feedback: instead of silently continuing when the purge-trigger reconnect/prepare/deallocate fails, return false immediately with a diagnostic message.	2 months ago
Rene Cannao	3296ed5a07	Moving test_cluster_sync_mysql_servers-t from legacy-g3 to legacy-g5	2 months ago
Rene Cannao	26c5a25724	Fix read_only test assuming default_hostgroup=0 The test configures mysql_servers with writer_hostgroup=0 and reader_hostgroup=1, but the CI infrastructure sets users' default_hostgroup to 1300. When the test opens a proxy connection and runs BEGIN, ProxySQL routes to hostgroup 1300 which has no servers, causing a 10s timeout. Fix by setting default_hostgroup=0 for the test user at the start of each scenario.	2 months ago
Rene Cannao	1261d18a95	Fix SHOW WARNINGS test to work with MySQL 5.7 CAST('abc' AS DOUBLE) is MySQL 8.0+ syntax and fails on the legacy test group which uses MySQL 5.7. Use CAST('abc' AS SIGNED) instead, which produces a truncation warning on all MySQL versions. Fixes regression from PR #5548.	2 months ago
Rene Cannao	9098992f82	Fix timing issue in PgSQL prepared statement purge test The purge verification in test 1 was checking the statement count immediately after creating statements. The purge mechanism has a 1-second minimum interval between attempts and only removes statements with ref_count_client == 0. Fix by closing the connection (dropping refcounts), waiting 3 seconds, then triggering a new refcount change to invoke the purge before checking.	2 months ago
Rene Cannao	73a561450d	Fix race condition in CI-legacy-* workflow triggers All 4 CI-legacy workflows used `types: [ in_progress ]` which fired them before CI-builds finished, causing cache-miss failures on the build artifacts. Change to `types: [ completed ]` and add success guard, matching the pattern used by CI-taptests-groups. Affected workflows: - CI-legacy-clickhouse-g1 - CI-legacy-g2 - CI-legacy-g2-genai - CI-legacy-g4	2 months ago
Rene Cannao	fe5a819051	Add 3 missing tests to groups.json Register newly compiled test binaries that were missing from groups.json, causing check_groups.py to block CI runs: - gtid_server_data_unit-t → unit-tests-g1 - test_ffto_mysql_errors-t → legacy-g4, mysql84-g4 - test_ffto_pgsql_error_stats-t → legacy-g4, mysql84-g4	2 months ago
René Cannaò	7510d8073a	Merge pull request #5553 from sysown/fix/5539-ffto-default-by-tier Disable FFTO by default in 3.1.x, keep enabled in 4.0.x	2 months ago
René Cannaò	ceb9690891	Merge commit from fork Fix COM_STMT_SEND_LONG_DATA short packet handling	2 months ago
René Cannaò	3659ba85fb	Merge branch 'v3.0' into advisory/com-stmt-send-long-data-fix-1 Signed-off-by: René Cannaò <rene@proxysql.com>	2 months ago
Rene Cannao	151169da9b	Fix COM_STMT_SEND_LONG_DATA short packet handling	2 months ago
René Cannaò	49b60b24a6	Merge commit from fork Fix unterminated HandshakeResponse username parsing	2 months ago
René Cannaò	b94c855d4c	Merge branch 'v3.0' into advisory-fix-1 Signed-off-by: René Cannaò <rene@proxysql.com>	2 months ago
René Cannaò	fa6ebcdcc9	Merge commit from fork Fix PROXY protocol v1 address overflow	2 months ago
René Cannaò	419cd7608b	Merge commit from fork Fix COM_STMT_CLOSE short packet handling	2 months ago
René Cannaò	7a574a537a	Merge commit from fork Validate COM_CHANGE_USER packet bounds	2 months ago
Rene Cannao	c9b30f8f56	fix: handle SHUNNED status in CLUSTER_QUERY_RUNTIME_PGSQL_SERVERS The CASE expression was missing a WHEN branch for SHUNNED, causing rows with that status to get NULL instead of a valid string. This corrupted runtime pgsql server checksums during cluster sync. Add WHEN 'SHUNNED' THEN 'ONLINE' (consistent with CLUSTER_QUERY_PGSQL_SERVERS_V2) and an ELSE fallback to prevent NULL for any unexpected status values.	2 months ago
Rene Cannao	cbc50ebf8a	ci: add dedicated CI workflow and groups.json entry for pgsql cluster sync test Register test_cluster_sync_pgsql-t in groups.json (default-g3) so it is picked up by CI-taptests-groups and passes check_groups.py validation. Add CI-taptests-pgsql-cluster.yml as a standalone workflow that runs only the PostgreSQL cluster sync TAP test, triggered after CI-trigger or via workflow_dispatch.	2 months ago
René Cannaò	63007cfa39	Merge pull request #5559 from sysown/v3.0-issue5556 Fix crash when COM_BINLOG_DUMP received with idle backends (#5556)	2 months ago
Rene Cannao	499896e44b	Address PR review feedback for test_binlog_dump_multi_backend_crash - Move wait_timeout mutation after hostgroup discovery skip check so early returns don't leave backend state modified (Copilot review) - Add precondition assertion verifying connections exist in both writer and reader hostgroups before sending COM_BINLOG_DUMP, using ConnOK pool stats (CodeRabbit review)	2 months ago
Rene Cannao	b2f4b4f9a0	Fix crash when COM_BINLOG_DUMP received with idle backends from other hostgroups (#5556 ) When a session has backend connections to multiple hostgroups and receives COM_BINLOG_DUMP, GPFC_Replication_SwitchToFastForward() transitions the session to FAST_FORWARD mode but only manages the active backend. Idle backends from other hostgroups remain in the session's mybes array with async_state_machine==ASYNC_IDLE. If MySQL's wait_timeout closes one of these idle connections, the resulting POLLIN event causes handler() to be called with ASYNC_IDLE, which has no case in the switch and hits assert(0). Fix: Clean up idle backends from other hostgroups during the FAST_FORWARD transition in GPFC_Replication_SwitchToFastForward(). These connections will never be used again, so they are destroyed immediately. Includes a regression test (test_binlog_dump_multi_backend_crash-t) that: - Sets up two hostgroups with query rules routing SELECTs to reader HG - Creates backend connections to both hostgroups on one session - Sends COM_BINLOG_DUMP to enter FAST_FORWARD mode - Waits for wait_timeout to close the idle backend - Verifies ProxySQL does not crash	2 months ago
Rene Cannao	61fc7a129c	Disable FFTO by default in 3.1.x, keep enabled in 4.0.x (#5539 ) Gate the ffto_enabled default on PROXYSQLGENAI so that: - 3.1.x (PROXYSQL31): FFTO compiled in but disabled by default - 4.0.x (PROXYSQLGENAI): FFTO enabled by default (unchanged) Users on 3.1.x can still opt in via SET mysql-ffto_enabled='true'.	2 months ago
René Cannaò	d0625dc243	Merge pull request #5549 from sysown/v3.0-CodeCov0325 Add 32 unit test files (2963 assertions) and CI workflow improvements	2 months ago
Rene Cannao	6d6015a203	fix: align MCP Bearer token test with case-insensitive implementation The test expected lowercase 'bearer' to be rejected, but the implementation uses strncasecmp() per RFC 7235 (auth-scheme is case-insensitive). Update test to expect true for lowercase prefix.	2 months ago
René Cannaò	2fcc5729ee	Merge pull request #5548 from yuji-hatakeyama/5306-fix_show_warnings_with_comment Fix #5306 - SHOW WARNINGS with inline comments incorrectly sets warning_count	2 months ago
Rene Cannao	8b0c7be827	Add fallback to raw query when digest_text is unavailable When query digests are disabled (digest_text is NULL), the SHOW WARNINGS detection would always return false, losing the original behavior for plain "SHOW WARNINGS" queries. Add an else branch that falls back to checking query.ptr, preserving the pre-fix behavior when digests are off.	2 months ago

1 2 3 4 5 ...

9324 Commits (93eabfc7bc265a274a2bf9ffe37fbbcafbfda374) All Branches Search

9324 Commits (93eabfc7bc265a274a2bf9ffe37fbbcafbfda374)

All Branches