proxysql

Commit Graph

Author	SHA1	Message	Date
Rahim Kanji	e7ee68dda5	Add SSL/TLS traffic decryption for PostgreSQL backend connections (#5281 ) Extends the existing NSS keylog-based SSL decryption to PostgreSQL backend connections by patching libpq with a global PQsetSSLKeyLogCallback() API. When admin-ssl_keylog_file is configured, TLS secrets from all PgSQL backend SSL handshakes are written to the keylog file in NSS Key Log Format, enabling traffic decryption with Wireshark/tshark. Approach: - Patch libpq with global callback following the PQsslKeyPassHook pattern - Single startup call covers all backend paths (regular, monitor, kill, harvester) - No new admin variables — existing admin-ssl_keylog_file covers PgSQL backends - Zero changes to ProxySQL PgSQL connection code Includes 9 integration tests (pgsql-ssl_keylog-t) and user guide updates.	2 weeks ago
Rene Cannao	b2f4b4f9a0	Fix crash when COM_BINLOG_DUMP received with idle backends from other hostgroups (#5556 ) When a session has backend connections to multiple hostgroups and receives COM_BINLOG_DUMP, GPFC_Replication_SwitchToFastForward() transitions the session to FAST_FORWARD mode but only manages the active backend. Idle backends from other hostgroups remain in the session's mybes array with async_state_machine==ASYNC_IDLE. If MySQL's wait_timeout closes one of these idle connections, the resulting POLLIN event causes handler() to be called with ASYNC_IDLE, which has no case in the switch and hits assert(0). Fix: Clean up idle backends from other hostgroups during the FAST_FORWARD transition in GPFC_Replication_SwitchToFastForward(). These connections will never be used again, so they are destroyed immediately. Includes a regression test (test_binlog_dump_multi_backend_crash-t) that: - Sets up two hostgroups with query rules routing SELECTs to reader HG - Creates backend connections to both hostgroups on one session - Sends COM_BINLOG_DUMP to enter FAST_FORWARD mode - Waits for wait_timeout to close the idle backend - Verifies ProxySQL does not crash	2 weeks ago
René Cannaò	d0625dc243	Merge pull request #5549 from sysown/v3.0-CodeCov0325 Add 32 unit test files (2963 assertions) and CI workflow improvements	2 weeks ago
René Cannaò	2fcc5729ee	Merge pull request #5548 from yuji-hatakeyama/5306-fix_show_warnings_with_comment Fix #5306 - SHOW WARNINGS with inline comments incorrectly sets warning_count	2 weeks ago
Rene Cannao	8b0c7be827	Add fallback to raw query when digest_text is unavailable When query digests are disabled (digest_text is NULL), the SHOW WARNINGS detection would always return false, losing the original behavior for plain "SHOW WARNINGS" queries. Add an else branch that falls back to checking query.ptr, preserving the pre-fix behavior when digests are off.	2 weeks ago
Rene Cannao	b40c77c48d	Revert "fix(#5306 ): SHOW WARNINGS with inline comments incorrectly sets warning_count" This reverts commit `049ea8ce84`.	2 weeks ago
Rene Cannao	049ea8ce84	fix(#5306 ): SHOW WARNINGS with inline comments incorrectly sets warning_count When SHOW WARNINGS queries contain inline comments (e.g., "SHOW /* comment */ WARNINGS"), ProxySQL incorrectly includes warning_count in the EOF packet because the detection used raw query text which still contains comments. Fix: use digest_text (comments stripped by the query processor) for SHOW WARNINGS detection, with fallback to raw query.ptr when digest_text is unavailable (digests disabled). This preserves the original behavior for both digest-enabled and digest-disabled modes. Reported by DBIx::Sunny (Perl) users who send annotated queries. Closes #5306	2 weeks ago
Rene Cannao	acaa100dc8	fix: address PR review feedback from coding agents Actionable fixes from CodeRabbit / agent reviews on PR #5549: 1. include/Stats_Tool_Handler.h: add #include <cstdint> — uint64_t used in parse_digest_filter() without direct include 2. lib/MCP_Endpoint.cpp: use strncasecmp() for Bearer token prefix check — RFC 7235 Section 2.1 says auth-scheme is case-insensitive. This is a real bug fix in production code. 3. test/tap/tests/unit/sqlite3db_unit-t.cpp: guard result->rows_count dereference with null check to avoid segfault if execute_statement returns null 4. test/tap/tests/unit/genai_mcp_endpoint_unit-t.cpp: remove memset() calls on MCP_Threads_Handler that leaked constructor's strdup("") allocations for auth fields 5. test/tap/groups/check_groups.py: use os.walk() instead of hardcoded directory list so new test subdirectories are discovered 6. test/tap/tests/unit/admin_disk_upgrade_unit-t.cpp: add comment documenting intentional calloc pattern (ProxySQL_Admin constructor has too many daemon deps for unit tests)	2 weeks ago
Yuji Hatakeyama	604c020857	Fix #5306 : Use digest_text to suppress warning_count for SHOW WARNINGS	3 weeks ago
Rene Cannao	99dcb5b160	fix: namespace stats_utils helpers to avoid symbol collision ProxySQL_Admin_Tests2.cpp defines its own parse_int_or_zero() in an anonymous namespace. Our declaration in Stats_Tool_Handler.h (included transitively via cpp.h) created an ambiguous overload error. Fix: wrap the 4 helper functions in namespace stats_utils in both the header and implementation. A using-directive within the .cpp keeps existing call sites working without qualification.	3 weeks ago
Rene Cannao	1d2f16b506	test: add 4 more unit test files with 198 assertions (Tier 1) Expand coverage for charset lookups, GenAI Query/LLM/MCP components. New test files (test/tap/tests/unit/): Core: - charset_find_unit-t (62 tests): all 4 proxysql_find_charset functions — edge cases for NULL returns, utf8mb3→utf8 translation, case-insensitive matching, collation preference GenAI/MCP (PROXYSQLGENAI=1): - genai_query_handler_unit-t (65 tests): validate_sql_identifier, escape_string_literal, strip_leading_comments, json_string/ json_int/json_double safe extraction, is_pgsql_protocol, is_runtime_online_status, uppercase_or_unknown - genai_llm_clients_unit-t (35 tests): is_retryable_error HTTP status codes (408/429/5xx) and CURL error codes, plus LLM_WriteCallback data accumulation - genai_mcp_endpoint_unit-t (36 tests): validate_bearer_token extraction/comparison, create_jsonrpc_response/error JSON-RPC 2.0 formatting Minimal production changes for testability: - include/Query_Tool_Handler.h: friend class + extern declarations for 8 pure helper functions - lib/Query_Tool_Handler.cpp: remove static from 8 file-local pure functions (no behavioral change) - include/MCP_Endpoint.h: friend class + new static validate_bearer_token() extracted from authenticate_request() - lib/MCP_Endpoint.cpp: extract token validation into static method - lib/LLM_Clients.cpp: remove static from is_retryable_error() and rename WriteCallback to LLM_WriteCallback (avoid collisions) Running totals: 15 test files, 718 assertions, all passing.	3 weeks ago
Rene Cannao	accc68c9dc	test: add 11 new unit test files with 520 assertions for lib/ coverage Add targeted unit tests for pure functions and isolated components across lib/, increasing test coverage for both core utilities and GenAI/MCP components. New test files (test/tap/tests/unit/): Core utilities: - pgsql_variables_validator_unit-t (75 tests): bool, float, string, memory size, search_path, and client_encoding validators - gen_utils_unit-t (28 tests): mywildcmp wildcard matching, escape_string_single_quotes, Proxy_file_regular - mysql_encode_unit-t (23 tests): mysql_encode_length, CPY3/CPY8, proxy_my_crypt XOR, write_encoded_length - proxysql_utils_unit-t (23 tests): hex/unhex, sql_escape, split_str, replace_str, generate_multi_rows_query, checksums - proxy_protocol_unit-t (22 tests): PROXY v1 header parsing, subnet validation, IPv4/IPv6 network matching - pgsql_error_helper_unit-t (139 tests): SQLSTATE code mapping, all 38 error class branches, severity classification, fill/reset - pgsql_txn_state_unit-t (54 tests): TxnCmdParser for BEGIN, COMMIT, ROLLBACK, SAVEPOINT, RELEASE, AND CHAIN variants - sqlite3db_unit-t (21 tests): execute, return_one_int, check_table_structure, build_table, row ops, locking GenAI/MCP (PROXYSQLGENAI=1): - genai_stats_parsing_unit-t (79 tests): parse_server_filter, parse_digest_filter, parse_ll_or_zero, get_interval_config, calculate_percentile - genai_fts_string_unit-t (32 tests): MySQL_FTS sanitize_name, escape_identifier, escape_sql, table name construction - genai_anomaly_unit-t (24 tests): normalize_query SQL normalization, check_sql_injection pattern detection Minimal production changes for GenAI testability: - include/Anomaly_Detector.h: add friend class for test access - include/Stats_Tool_Handler.h: declare 4 parsing helpers, move 2 pure computation methods (get_interval_config, calculate_percentile) to public - lib/Stats_Tool_Handler.cpp: remove static from 4 free helper functions (no behavioral change, only linkage visibility)	3 weeks ago
Rene Cannao	c6f442b4d1	feat(ffto): record PostgreSQL FF errors in stats_pgsql_errors	3 weeks ago
Rene Cannao	c3df4ee014	feat(ffto): record MySQL FF errors in stats_mysql_errors	3 weeks ago
Rene Cannao	fe701d29aa	feat(ffto): add pgsql_parse_error_response() helper with unit tests	3 weeks ago
Rene Cannao	1c559e96d8	feat(ffto): add mysql_parse_err_packet() helper with unit tests	3 weeks ago
Rene Cannao	7f5bbf0f19	Fix compilation error: add missing #include <cstdint> in MonitorHealthDecision.cpp The int64_t cast in can_unshun_server() requires <cstdint>, which was not included, causing build failure. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>	3 weeks ago
René Cannaò	4d2bcc303e	Merge remote-tracking branch 'origin/v3.0' into copilot/feature-load-restapi-routes-config # Conflicts: # test/tap/groups/groups.json	3 weeks ago
René Cannaò	da13ab59e5	Merge remote-tracking branch 'origin/v3.0' into copilot/add-ssl-tls-certificate-stats-table # Conflicts: # test/tap/groups/groups.json	3 weeks ago
René Cannaò	9781f5992d	Merge remote-tracking branch 'origin/v3.0' into copilot/feature-load-restapi-routes-config	3 weeks ago
René Cannaò	efedd93c92	Rename stats_global to stats_proxysql_global to avoid strstr substring collision stats_global is a substring of stats_mysql_global and stats_pgsql_global, causing the strstr-based dispatch to match incorrectly. Exclusion guards don't work either because a query referencing both tables would suppress the stats_proxysql_global refresh. Renaming to stats_proxysql_global follows the existing naming convention (stats_mysql_global, stats_pgsql_global) and eliminates the substring collision entirely.	3 weeks ago
René Cannaò	c3564af9cf	Fix timeout_ms validation and write path section name consistency 1. Validate timeout_ms against the CHECK constraint (100..100000000) and skip routes with invalid or missing values with an error message. Previously, omitting timeout_ms defaulted to 0, which silently failed the SQLite CHECK constraint and dropped the route. 2. Update Write_Restapi_to_configfile to emit 'restapi_routes:' instead of 'restapi:' so save-then-reload cycles use the preferred section name consistently.	3 weeks ago
René Cannaò	9850d9daeb	Fix stats_global substring collision and reload failure tracking 1. stats_global strstr match: add exclusion guards for stats_mysql_global and stats_pgsql_global to prevent spurious stats___global() calls and unnecessary ssl_mutex contention on every MySQL/PgSQL global stats query. 2. TLS reload failure tracking: set tls_last_load_ok=false and update tls_load_count/tls_last_load_timestamp on non-bootstrap reload failures. Previously, a failed PROXYSQL RELOAD TLS would leave stale SUCCESS status in stats_global.	3 weeks ago
René Cannaò	ea16ebb8ad	Merge branch 'v3.0-5473' into v3.0-5499 Signed-off-by: René Cannaò <rene@proxysql.com>	3 weeks ago
René Cannaò	d872a392a6	Merge branch 'v3.0-5473' into v3.0-5498 Signed-off-by: René Cannaò <rene@proxysql.com>	3 weeks ago
René Cannaò	8d7459e560	Merge branch 'v3.0-5473' into v3.0-5497 Signed-off-by: René Cannaò <rene@proxysql.com>	3 weeks ago
René Cannaò	9bccd2d1fa	Merge branch 'v3.0-5473' into v3.0-5495 Signed-off-by: René Cannaò <rene@proxysql.com>	3 weeks ago
René Cannaò	cffd85f0ff	Merge pull request #5511 from sysown/v3.0-5494 Phase 3.6: Backend variable sync decisions + unit tests	3 weeks ago
René Cannaò	f5ace60850	Address review on PgSQL error classifier (PR #5514 ) - Fix: 57014 (query_canceled) is now classified as REPORT_TO_CLIENT instead of FATAL (it's a normal cancellation, not a server crash) - Fix docstring: crash_shutdown is 57P02 (class 57), not class 58; class 58 is system/I/O errors - Add test for 57014 exception (+1 test, plan 25→26)	3 weeks ago
René Cannaò	b51f50b96b	Address review: fix NULL asymmetry in backend sync decisions (PR #5511 ) - BackendSyncDecision.cpp: treat asymmetric NULLs as mismatches (client=NULL + backend="bob" → SYNC_USER, not SYNC_NONE) - Tests: replace weak null assertions (a >= 0 always true) with specific checks for SYNC_USER/SYNC_SCHEMA on asymmetric NULLs, both-null → no sync, schema asymmetric null (+2 tests, plan 15→17)	3 weeks ago
René Cannaò	c40eed9405	Merge branch 'v3.0-5473' into v3.0-5492 Signed-off-by: René Cannaò <rene@proxysql.com>	3 weeks ago
René Cannaò	f07cd83d16	Extract MySQL protocol utilities for FFTO unit testing (#5499 ) New: include/MySQLProtocolUtils.h, lib/MySQLProtocolUtils.cpp Functions: - mysql_read_lenenc_int(): reads MySQL length-encoded integers from raw buffers (mirrors read_lenenc_int in MySQLFFTO.cpp) - mysql_build_packet(): constructs MySQL wire-format packets (3-byte length + seq_id + payload) for crafting test data	3 weeks ago
René Cannaò	e82bcb391b	Extract PgSQL error classification logic (Phase 3.10, #5498 ) New: include/PgSQLErrorClassifier.h, lib/PgSQLErrorClassifier.cpp Functions: - classify_pgsql_error(): classifies by SQLSTATE class — connection (08), transaction rollback (40), resources (53) = retryable; operator intervention (57), system error (58) = fatal; all others (syntax 42, constraints 23, data 22) = report to client - pgsql_can_retry_error(): checks retry conditions (retries left, not in transaction — PgSQL transactions are atomic)	3 weeks ago
René Cannaò	78400743ff	Extract PgSQL monitor health decisions (Phase 3.9, #5497 ) New: include/PgSQLMonitorDecision.h, lib/PgSQLMonitorDecision.cpp Functions: - pgsql_should_shun_on_ping_failure(): threshold-based shunning (simpler than MySQL — always aggressive with kill_all) - pgsql_should_offline_for_readonly(): read-only server in writer hostgroup should go OFFLINE_SOFT Unshun recovery is already covered by MonitorHealthDecision.h can_unshun_server() (same logic for both protocols).	3 weeks ago
René Cannaò	ead2331fb9	Extract backend variable sync decisions (Phase 3.6, #5494 ) New: include/BackendSyncDecision.h, lib/BackendSyncDecision.cpp determine_backend_sync_actions() returns bitmask of needed sync: - SYNC_USER: username mismatch → CHANGE USER required - SYNC_SCHEMA: schema mismatch → USE <db> required (skipped if SYNC_USER set, since CHANGE USER handles schema) - SYNC_AUTOCOMMIT: autocommit state mismatch Covers both MySQL (7 verify functions) and PgSQL (2 verify functions) at the decision level — the core comparisons are the same.	3 weeks ago
René Cannaò	5c227a3bc9	Extract MySQL error classification logic (Phase 3.7, #5495 ) New: include/MySQLErrorClassifier.h, lib/MySQLErrorClassifier.cpp Functions: - classify_mysql_error(): classifies error codes as retryable (1047 WSREP, 1053 shutdown) or fatal, checking retry conditions - can_retry_on_new_connection(): checks if offline server retry is possible given connection state (reusable, no txn, no transfer)	3 weeks ago
René Cannaò	5148989a72	Extract PgSQL CommandComplete tag parser for unit testing (#5499 ) New files: - include/PgSQLCommandComplete.h: PgSQLCommandResult struct + parse_pgsql_command_complete() declaration - lib/PgSQLCommandComplete.cpp: implementation mirroring extract_pg_rows_affected() from PgSQLFFTO.cpp Parses PostgreSQL CommandComplete tags (INSERT/UPDATE/DELETE/SELECT/ FETCH/MOVE/COPY/MERGE) to extract row counts. Pure function with no FFTO or session dependencies.	3 weeks ago
René Cannaò	7b06515d7f	Merge branch 'v3.0-5473' into v3.0-5496 Signed-off-by: René Cannaò <rene@proxysql.com>	3 weeks ago
René Cannaò	3f0281651e	Extract transaction state tracking logic (Phase 3.8, #5496 ) New files: - include/TransactionState.h: 2 pure function declarations - lib/TransactionState.cpp: implementations Functions: - update_transaction_persistent_hostgroup(): mirrors session logic for locking/unlocking HG on transaction start/end - is_transaction_timed_out(): checks if transaction exceeded max time Logic identical for MySQL and PgSQL (shared via Base_Session).	3 weeks ago
René Cannaò	aaa02bb400	Extract hostgroup routing decision logic (Phase 3.5, #5493 ) New files: - include/HostgroupRouting.h: HostgroupRoutingDecision struct + resolve_hostgroup_routing() declaration - lib/HostgroupRouting.cpp: implementation mirroring the routing block in get_pkts_from_client() (~lines 5340-5377) Logic covers: QP destination override, transaction affinity, hostgroup lock acquisition/enforcement, error on HG mismatch. Identical for both MySQL and PgSQL protocols.	3 weeks ago
René Cannaò	7e49c41975	Address review feedback on server selection (PR #5508 ) - Fix truncation: use uint64_t modulo instead of unsigned int cast when computing target from total_weight (prevents skewed selection when total_weight > UINT_MAX) - Remove unused #include <cstdlib> - Add docstring note about max_latency_us=0 semantics: in production this means "use thread default", this extraction treats it as "no limit" — callers should resolve defaults before populating the struct - Remove trailing "fix" comment from test	3 weeks ago
René Cannaò	dba1294585	Extract server selection algorithm into pure functions (Phase 3.4, #5492 ) New files: - include/ServerSelection.h: ServerCandidate struct (lightweight, no MySrvC dependency) + is_candidate_eligible() + select_server_from_candidates() - lib/ServerSelection.cpp: implementations ServerCandidate decouples selection logic from connection pool: contains only weight, status, connections, latency, and replication lag — no MySQL_Connection pointers or pool objects. Selection algorithm: 1. Filter candidates by eligibility (ONLINE, below max_connections, within latency/lag thresholds) 2. Weighted random selection using seeded LCG (deterministic for testing, avoids polluting global srand() state) Uses its own ServerSelectionStatus enum to avoid pulling in proxysql_structs.h and its entire dependency chain.	3 weeks ago
René Cannaò	99980ca196	Address review feedback on monitor health (PR #5507 ) - Fix integer overflow: cast shun_recovery_time_sec to int64_t before multiplying by 1000 to prevent overflow for large values (>24 days) - Add docstring note about production lag==-2 override path (issue #959) not covered by this simplified extraction - Remove trailing "fix plan" comment from test	3 weeks ago
René Cannaò	59654e48a3	Extract monitor health decision logic into pure functions (Phase 3.3, #5491 ) New files: - include/MonitorHealthDecision.h: declarations for 4 pure functions - lib/MonitorHealthDecision.cpp: implementations Extracted functions: - should_shun_on_connect_errors(): mirrors MySrvC::connect_error() threshold logic (min(shun_on_failures, connect_retries+1)) - can_unshun_server(): mirrors MyHGC recovery loop with time check, connect_timeout_max cap, min 1s floor, and kill_all drain check - should_shun_on_replication_lag(): mirrors MySQL_HostGroups_Manager lag check with consecutive count threshold - can_recover_from_replication_lag(): mirrors lag recovery check All functions are pure (no global state, no I/O) and added to libproxysql.a via lib/Makefile.	3 weeks ago
copilot-swe-agent[bot]	54a2af325f	refactor: extract query rule matching helper Co-authored-by: renecannao <3645227+renecannao@users.noreply.github.com> Agent-Logs-Url: https://github.com/sysown/proxysql/sessions/15475838-0841-46d6-9afd-ebdc5847a3ae	3 weeks ago
René Cannaò	37ce0525bb	Fix connection pool extraction: separate header, proper unit tests Fixes the Copilot-generated PR #5500 production code and test file: Production fixes: - Create standalone ConnectionPoolDecision.h header with the struct and function declarations (no circular dependency issues) - Include ConnectionPoolDecision.h in MySrvConnList.cpp and PgSQL_HostGroups_Manager.cpp to resolve compilation errors - Replace inline declarations in Base_HostGroups_Manager.h with #include "ConnectionPoolDecision.h" Test fixes: - Remove misplaced test from test/tap/tests/ (tested reimplemented copies of functions instead of the real code) - Add proper unit test at test/tap/tests/unit/connection_pool_unit-t.cpp using the test harness (test_globals.h, test_init.h) - 22 test cases covering eviction, throttling, pool state decisions, warming, and edge cases - Register in unit test Makefile	3 weeks ago
copilot-swe-agent[bot]	5e8b1e3e5a	Extract connection pool decision logic into pure functions (MySQL + PgSQL) Co-authored-by: renecannao <3645227+renecannao@users.noreply.github.com> Agent-Logs-Url: https://github.com/sysown/proxysql/sessions/89d35f54-3313-48d4-8252-70e3c1f3b036	3 weeks ago
Rene Cannao	2e552b8ae8	fix(tls): improve SSL/TLS certificate tracking and fix memory leaks - lib/ProxySQL_GloVars.cpp: Free tls_cert_file, tls_ca_file, and tls_key_file in ProxySQL_GlobalVariables destructor. - src/proxy_tls.cpp: Added std::lock_guard during initial bootstrap of TLS variables to prevent race conditions. - src/proxy_tls.cpp: Ensure TLS file paths are updated in GloVars during PROXYSQL RELOAD TLS to keep stats table consistent with runtime configuration.	3 weeks ago
copilot-swe-agent[bot]	bbcc67b339	fix: harden restapi config inserts Co-authored-by: renecannao <3645227+renecannao@users.noreply.github.com> Agent-Logs-Url: https://github.com/sysown/proxysql/sessions/40d309f0-bd3d-4616-9cd6-fa7d8f1cb355	3 weeks ago
copilot-swe-agent[bot]	04c5e58e6b	fix: escape restapi config values Co-authored-by: renecannao <3645227+renecannao@users.noreply.github.com> Agent-Logs-Url: https://github.com/sysown/proxysql/sessions/40d309f0-bd3d-4616-9cd6-fa7d8f1cb355	3 weeks ago

1 2 3 4 5 ...

5091 Commits (v3.0_pgsql_sslkeylog_5281)