proxysql

Commit Graph

Author	SHA1	Message	Date
Rene Cannao	031f6fa1f5	fix(build): adapt callers of dns_cache counters to std::atomic The previous commit promoted MySQL_Monitor::dns_cache_{queried, lookup_success,record_updated} to std::atomic<unsigned long long> but missed several call sites that pass them through implicit copy: * lib/MySQL_Thread.cpp sprintf("%llu", GloMyMon->dns_cache_) — varargs cannot copy std::atomic. lib/PgSQL_Thread.cpp sprintf("%llu", GloPgMon->dns_cache_) — same, for the PgSQL counters. lib/MySQL_Monitor.cpp p_update_counter(..., GloMyMon->dns_cache_) — works via implicit operator T(), but explicit .load() is clearer and matches the rest of the file. Promote PgSQL_Monitor::dns_cache_ to std::atomic for the same race that motivated the MySQL side (resolver workers increment, stats readers and p_update_metrics read from another thread), and add .load() at every read site so we don't depend on implicit conversion. Verified with a local 'make -j32 debug' build in the integration worktree.	1 month ago
René Cannaò	af4aa17346	Merge pull request #5807 from sysown/fix/5790-mariadb-collation-255 fix(core): validate_charset must replace collation 255 on MariaDB backends (#5790)	1 month ago
Rene Cannao	3fd7aae059	fix(core): wrap MON_GALERA case body in braces (compile fix) The previous commit introduced local variable declarations with initializers (`const char *sv = …`, `bool is_mariadb = …`, etc.) directly inside `case MON_GALERA:` of the `switch` in `init_async()` at lib/MySQL_Monitor.cpp:742. The sibling `case MON_CLOSE_CONNECTION:` / `MON_CONNECT:` / `MON_AWS_AURORA:` labels then jump past these initializations, which GCC rejects: MySQL_Monitor.cpp:779:14: error: jump to case label MySQL_Monitor.cpp:781:14: error: jump to case label MySQL_Monitor.cpp:783:14: error: jump to case label Wrap the `#else` body of the MON_GALERA case in `{ ... }` so the declarations live in their own scope and no sibling case can jump over them. No behavior change. The async path at lib/MySQL_Monitor.cpp:2299 was already inside its own `{ ... }` block (added with the original code, not by this patch), so it compiled fine and needs no change. The `MySQL_Session.cpp` edits live inside `if / else if` chains with their own braces, also unaffected.	1 month ago
Rene Cannao	f2b53ea6e3	Address findings from independent diff review A subagent code review on top of the CodeRabbit + Gemini bot reviews found five additional issues that the bots missed. All five are fixed here. 1. LOAD PGSQL SERVERS TO RUNTIME was waking MySQL's resolver, not PgSQL's. PgSQL_HostGroups_Manager::commit() inherited a MySQL_Monitor::trigger_dns_cache_update() call from a copy of the MySQL HGM, so after a LOAD pgsql hostnames had to wait for the refresh interval (default 60 s) before becoming resolvable — defeating the warm-cache-before-traffic intent that motivated issue #5768. Route to PgSQL_Monitor::trigger_dns_cache_update(). Verified directly: with refresh_interval=60000 ms, inserting a pgsql_servers row + LOAD now produces record_updated=1 within ~4 s instead of waiting a full minute. 2. DNSResolverWorker could deadlock shutdown. monitor_dns_resolver_thread isn't noexcept (vector growth, set_thread_name, etc. can throw); if it threw, the std::promise on DNS_Resolve_Data was never satisfied, the producer's future::get() blocked forever, the resolver loop never reached the shutdown check, and pthread_join in main hung. Wrap the call in try/catch and force-satisfy the promise with a failure result on any uncaught exit. 3. Test step 3 wasn't actually exercising the connect-path lookup it claimed. hammer_proxy() routes through the proxy front-end, which in standard test infra resolves through the default hostgroup — typically configured with an IP literal (see e.g. test/infra/docker-pgsql16-single/conf/proxysql/config.sql). IP literals bypass dns_lookup via validate_ip(), so the dns_cache_queried bumps observed came from the resolver loop alone, not from the connect path. Step 3 would have passed even if PgSQL_Connection::connect_start_DNS_lookup() was wired up wrong. Install a pgsql_query_rules row from inside the test that routes testuser to hostgroup 999, so the proxy actually calls into the cache on client connects. Clean up the rule in the test exit path. 4. Use-after-move in the _dns_cache_update debug log. Both PgSQL_Monitor::_dns_cache_update and MySQL_Monitor::_dns_cache_update logged debug_iplisttostring(ip_address) after std::move'ing ip_address into add_if_not_exist(). The moved-from vector is in a "valid but unspecified" state — typically empty — so the log printed "IP:[]" instead of the IPs. Capture the string before the move. Same bug also fixed in the bookkeeper expired-record requeue path inside PgSQL_Monitor::monitor_dns_cache. 5. monitor_dns_cache had inconsistent indentation inside the `if (hostnames.empty()) { ... } else { ... }` block introduced when I wrapped the original goto-based control flow. The braces were correct but the body sat at the wrong tab depth, making the scope visually misleading and prone to mis-edits. Re-indented. Findings deliberately skipped from the review (rationale in commit body): - "thread_local shared_ptr<DNS_Cache> keeps stale cache alive across GloPgMon recreate" — pre-existing MySQL pattern; GloPgMon is never destroyed mid-process today. - "PgSQL_monitor_dns_cache_pthread can deref GloPgMon during a tight shutdown" — shutdown ordering in main.cpp sets GloPgMon->shutdown before joining the thread and deletes GloPgMon only afterwards, matching the MySQL pattern. - "cache only seeded on ASYNC_CONNECT_SUCCESSFUL" — matches MySQL; pre-existing scope choice. - "validate_ip rejects bracketed IPv6" — pre-existing in MySQL; no current configuration path produces bracketed IPv6 in pgsql_servers. - "Test relies on outbound DNS for example.com" — matches the upstream MySQL test_dns_cache test which uses google.com / yahoo.com / amazon.com. Established convention. - "ai_family hard-coded to AF_UNSPEC for PgSQL" — already commented in code; follow-up if/when pgsql-resolution_family is added.	1 month ago
Rene Cannao	81bcc802da	fix(core): apply MariaDB / MySQL 9.x detection to sibling version checks The same mental defect fixed in lib/MySQL_Variables.cpp:validate_charset() (commit before this one) exists at four other call sites in core. All four use a first-character or 3-byte string-prefix comparison against server_version and consequently misclassify either MariaDB or MySQL 9.x (or both). Two are latent (MariaDB happens to land on the still-working branch); the two in MySQL_Monitor.cpp are an active break on MySQL 9.x Galera/PXC, because `INFORMATION_SCHEMA.GLOBAL_STATUS` was removed in MySQL 8.4 and MySQL 9.x falls into the else branch that targets it. Audited sites and fixes: lib/MySQL_Session.cpp:2607-2615 -- tx_isolation / tx_read_only Pre-fix: `strncmp(sv,"8",1)==0` chose `transaction_isolation` / `transaction_read_only`; everything else got `tx_isolation` / `tx_read_only`. MySQL 9.x fell to the else branch, but `tx_isolation` was removed in MySQL 8.4; on MySQL 9.x the SET fails. MariaDB also fell to the else branch -- coincidentally correct, since MariaDB keeps `tx_isolation` across all versions. Post-fix: use the modern name only for non-MariaDB MySQL >= 8.0 (any major), so MySQL 8.0/8.4/9.x all get `transaction_isolation` and MariaDB 10/11 keeps `tx_isolation`. lib/MySQL_Monitor.cpp:749 (sync MON_GALERA setup) lib/MySQL_Monitor.cpp:2297 (async MON_GALERA dispatch) Pre-fix: `strncmp(sv,"5.7",3)==0 \|\| strncmp(sv,"8",1)==0` picked performance_schema.global_status; everything else used INFORMATION_SCHEMA.GLOBAL_STATUS. MySQL 9.x fell to the else branch and queries INFORMATION_SCHEMA.GLOBAL_STATUS, which was removed in MySQL 8.4 -- monitor query fails on MySQL 9.x Galera/PXC. MariaDB Galera correctly fell to the else branch and uses information_schema, which is where MariaDB keeps GLOBAL_STATUS. Post-fix: use performance_schema for non-MariaDB MySQL >= 5.7 (so MySQL 5.7, 8.0, 8.4, 9.x all share that path); MariaDB Galera and legacy MySQL/PXC < 5.7 retain the information_schema path. Pattern used at all sites (matching the existing fix in validate_charset): const char sv = ...->mysql->server_version; bool is_mariadb = (sv != NULL && strstr(sv, "MariaDB") != NULL); int sv_major = (sv != NULL) ? atoi(sv) : 0; No dedicated test added for the Monitor / SET TRANSACTION paths in this commit: the Monitor change is exercised by any mariadb10-galera or mysql{8,9}x- CI group with Galera/PXC backends, and the transaction_isolation change by SET-isolation-level coverage in existing TAP groups. The validate_charset regression test added in the prior commit on this branch still covers the original #5790 path.	1 month ago
Rene Cannao	a2fff3725f	Extract DNS cache into shared DNS_Cache module DNS_Cache, DNS_Cache_Record, DNS_Resolve_Data, the resolver worker, and the validate_ip / get_connected_peer_ip_from_socket / debug_iplisttostring helpers used to live inside MySQL_Monitor.{hpp,cpp}. Move them to a new DNS_Cache.{hpp,cpp} so the same plumbing can back an independent PgSQL_Monitor cache. DNS_Cache no longer references GloMyMon directly; counters are wired in per-instance via set_counters(). monitor_dns_resolver_thread no longer reads mysql_thread___resolution_family; the desired ai_family is now a field on DNS_Resolve_Data set by the caller. MySQL_Monitor wires both up in its constructor and resolver loop respectively; behavior is identical. No change to the MySQL DNS cache semantics.	1 month ago
Wazir Ahmed	77cf813276	aurora: Update `AWS_Aurora_Hosts_resultset` when servers are purged Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	2 months ago
Wazir Ahmed	8c6e85f343	aurora: Fix monitor race condition and implement autopurge - Closes #5546 and #5547 Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	2 months ago
Rene Cannao	96c686dafa	MySQL_Monitor: GR thread ignores cached ping state on first iteration Why: mysql_server_ping_log can carry stale "unpingable" entries from before monitor_GR_thread_HG was (re)started (e.g. a previous test left those hostnames marked bad). With the cache filter applied on the very first cycle, find_resp_srvs() returns empty, the writer HG stays empty for a full healthcheck_interval, and GR-based tests flake on warm-up. Probe all configured hosts on the first iteration so subsequent cycles see a fresh ping_log. The filter resumes from iteration two onward.	2 months ago
Rene Cannao	fe89811324	fix(lib): eliminate remaining lib compiler warnings - Fix snprintf truncation in MySQL_Monitor ConsumerThread (%.15s for 16-byte buffer) - Mark unused safe_update_peer_info in ProxySQL_Cluster with __attribute__((unused)) - Remove unused start_time in GenAI_Thread::generate_embeddings - Remove unused now_iso() in Discovery_Schema	3 months ago
Rene Cannao	424a3664ae	Use thread-local storage for resolution_family variable Replace direct GloMTH->get_variable_string() call in monitor_dns_resolver_thread() with the proper mysql_thread___ thread-local variable pattern: - Add __thread char* mysql_thread___resolution_family in proxysql_structs.h - Add REFRESH_VARIABLE_CHAR(resolution_family) in refresh_variables() - Free the thread-local variable in MySQL_Thread destructor - Read mysql_thread___resolution_family directly in the monitor thread instead of taking a lock via get_variable_string()	3 months ago
copilot-swe-agent[bot]	bc843a1a25	test: cover mysql resolution family helper cases Agent-Logs-Url: https://github.com/sysown/proxysql/sessions/7d0ebed6-886c-4e06-add5-d74b44eced4a Co-authored-by: renecannao <3645227+renecannao@users.noreply.github.com>	3 months ago
copilot-swe-agent[bot]	4a93861c6c	feat: add mysql resolution_family setting Agent-Logs-Url: https://github.com/sysown/proxysql/sessions/7d0ebed6-886c-4e06-add5-d74b44eced4a Co-authored-by: renecannao <3645227+renecannao@users.noreply.github.com>	3 months ago
René Cannaò	5de836a4c1	Merge pull request #5373 from sysown/v3.0_improve_mysql_monitoring_5256 Improve MySQL Monitor connection management	4 months ago
Rene Cannao	85952373d9	Fix uninitialized mondb pointer in MySQL_Monitor_State_Data constructor Add mondb = NULL initialization to prevent Valgrind errors when monitor threads access the pointer before it's properly set.	4 months ago
Rene Cannao	200fdecb8d	Apply AI agent review fixes to PR #5374 Critical Fixes: - Fix NULL pointer dereferences in MySQL_Logger.cpp: * Add null checks for myds->myconn->parent in log_audit_entry() * Fix cleanup code in log_request() and log_audit_entry() * Change sprintf to snprintf for buffer safety - Remove redundant pthread_rwlock_init in debug.cpp: * filters_rwlock already initialized with PTHREAD_RWLOCK_INITIALIZER * Re-initialization caused undefined behavior per POSIX Major Improvements: - Add bounded wait helper wait_for_glo_mth() in proxysql_utils.h: * Replaces indefinite while loops with 10s timeout * Applied to 26 locations across 4 files Build System: - Require C++17 in Makefile (remove C++11 fallback): * Code uses C++17 structured bindings extensively * Cleanest solution vs adding fallback guards to 27+ locations Compilation fixes: - Add unistd.h include for usleep - Remove conflicting GloMyMon extern declaration Files changed: - Makefile - include/proxysql_utils.h - lib/MySQL_Logger.cpp - lib/debug.cpp - lib/MySQL_Monitor.cpp - lib/ProxySQL_Admin.cpp - lib/ClickHouse_Server.cpp - src/SQLite3_Server.cpp	5 months ago
Rahim Kanji	1cccfb3220	Fix connection cleanup in failure paths	5 months ago
Rene Cannao	3c85c593cf	Add GloMTH initialization wait to all monitor and server threads Fix race condition where threads were accessing GloMTH before it was fully initialized, causing Valgrind errors in get_variable_int() and related functions. Added wait loop pattern (while GloMTH==NULL { usleep(50000); }) followed by usleep(100000) to ensure GloMTH is fully initialized before use: In lib/MySQL_Monitor.cpp: - monitor_connect(), monitor_ping(), monitor_read_only() - monitor_group_replication(), monitor_galera(), monitor_replication_lag() - monitor_AWS_Aurora_thread_HG(), monitor_aws_aurora() - init_mysql_thread_struct() In src/SQLite3_Server.cpp: - SQLite3_Server_session_handler In lib/ProxySQL_Admin.cpp: - admin_session_handler In lib/ClickHouse_Server.cpp: - ClickHouse_Server_session_handler The pattern ensures threads wait for GloMTH initialization during startup, then check for NULL during shutdown to exit cleanly.	5 months ago
Rene Cannao	aa53d990ed	Add GloMTH initialization wait to all monitor threads Fix race condition where monitor threads were accessing GloMTH before it was fully initialized, causing Valgrind errors in get_variable_int(). Added wait loop pattern (while GloMTH==NULL { usleep(50000); }) followed by usleep(100000) to: - monitor_ping_thread - monitor_read_only_thread - monitor_replication_lag_thread - monitor_group_replication_thread (also moved check before MySQL_Thread creation) - monitor_galera_thread (also moved check before MySQL_Thread creation) This matches the existing pattern in monitor_connect_thread.	5 months ago
Rene Cannao	f4cd34be5c	Fix race condition in monitor_connect_thread Add wait loop for GloMTH initialization before accessing variables. Other monitor threads (MonitorPing, MonitorReadOnly, etc.) already had this pattern, but monitor_connect_thread was missing it. This fixes Valgrind errors about uninitialized values in get_variable_uint16() when strcasecmp reads from memory that hasn't been fully initialized yet.	5 months ago
Rahim Kanji	9eac5f6420	Improve async ping mmsd ownership tracking and pool validation	5 months ago
Rahim Kanji	bed851471a	Fix MySQL Monitor assertion failure in DEBUG builds Introduce destroy_mysql_connection() helper to centralize the pattern of unregistering, closing, and nullifying MySQL connections in monitor threads.	5 months ago
Rene Cannao	f779a6059a	refactor: migrate remaining prepare_v2 SIMPLE cases in MySQL_Monitor Migrated the following functions: - populate_monitor_mysql_server_aws_aurora_log() - populate_monitor_mysql_server_aws_aurora_check_status() - monitor_ping_process_ready_task_thread() - monitor_read_only_process_ready_tasks() (with if/else branches for replication_lag) All statements are now automatically managed by RAII smart pointers, eliminating manual finalize calls and potential memory leaks.	5 months ago
Rene Cannao	bdba0b44a0	refactor: migrate all prepare_v2 SIMPLE cases to RAII API in MySQL_Monitor Migrated the following functions: - update_monitor_mysql_servers() - update_monitor_proxysql_servers() - monitor_connect_thread() - monitor_ping_thread() - monitor_read_only_thread() - monitor_replication_lag_thread() - monitor_connect() cleanup - monitor_ping() cleanup - monitor_read_only() cleanup - monitor_replication_lag() cleanup - populate_monitor_mysql_server_group_replication_log() - populate_monitor_mysql_server_galera_log() All statements are now automatically managed by RAII smart pointers, eliminating manual finalize calls and potential memory leaks.	5 months ago
Rene Cannao	df1c4e7928	fix: medium priority warnings - write-strings and unused variables 1. Fix write-strings warnings (12 occurrences): - MySQL_HostGroups_Manager.cpp: Change char* to const char* for src_status and dst_status variables - MySQL_Monitor.cpp: Change ConsumerThread constructor parameter from char thread_name[16] to const char *thread_name 2. Fix unused-but-set-variable warnings (6 occurrences) by wrapping PROXYSQLGENAI-specific variables in #ifdef blocks: - stats_mcp_query_tools_counters - stats_mcp_query_tools_counters_reset - stats_mcp_query_digest - stats_mcp_query_digest_reset - stats_mcp_query_rules - runtime_mcp_query_rules These variables were assigned values but appeared unused when PROXYSQLGENAI was not defined. Now they are only defined and assigned when PROXYSQLGENAI is enabled. This eliminates 18 compiler warnings: 12 write-strings and 6 unused-but-set-variable warnings. Found during compiler warning analysis with 'make debug'.	5 months ago
René Cannaò	27f13ca18d	Merge pull request #5096 from Gonlo2/fix-close-ssl-con Avoid send close in ssl connections	5 months ago
Rahim Kanji	01a5b23b22	refactor connection pool put_connection to use mmsd and improve debug handling * Change MySQL_Monitor_Connection_Pool::put_connection signature to accept MySQL_Monitor_State_Data* instead of raw MYSQL/port. Centralize access to mysql and port via mmsd, reducing parameter mismatch and misuse. * Improve DEBUG bookkeeping: ensure connections are properly unregistered from the global debug registry with clearer assertions and logs. * Add consistent proxy_debug messages for connection register/unregister events. * Simplify server lookup/creation logic when returning connections to the pool. * Fix ordering of error handling to always unregister before closing connections. * Minor cleanup: remove unused labels/variables and modernize casts. * This refactor improves correctness, debuggability, and safety of monitor connection lifecycle management.	6 months ago
René Cannaò	0f7ff1f374	Merge branch 'v3.0' into v3.0_pgsql-query-digest-gen-5253 Signed-off-by: René Cannaò <rene@proxysql.com>	6 months ago
Rahim Kanji	5e75264bb3	Updated TAP test	6 months ago
Rahim Kanji	24e02e95d5	Changing monitor ping poll() timeout to 10ms	8 months ago
Rahim Kanji	6a80c3f288	Implemented explicit task memory ownership management in Monitor_Poll.	8 months ago
Rahim Kanji	50fae0b02a	Current check on file descriptors (fd) is not reliable or necessary	8 months ago
Rahim Kanji	1c3c42959c	Improve ping accuracy - Introduced batching for ping task dispatch (default: 30 servers per batch) - Moved mysql_server_ping_log writes to a thread pool	8 months ago
Juan Manuel Fernández García-Minguillán	8554742131	Avoid send close in ssl connections	10 months ago
René Cannaò	7490af1783	Group Replication support for MySQL 8.4 and 9.x Closes #4954	1 year ago
René Cannaò	f9575b7ee9	Addressing minor issue on Readyset Monitoring - Renamed ReadySet to Readyset - fixed incorrect status variable - added missing newlines	1 year ago
René Cannaò	d641ca89be	First POC for integration with ReadySet Monitor is able to detect if a backend is a ReadySet server, and it enables special monitoring. The special monitoring hacks replication lag checks: a ReadySet server is "monitored for replication lag" , but it has a special query and a special handler. The query for check is "SHOW READYSET STATUS" , and the `Status` line is processed: * Online: the backend is configured as ONLINE * Maintenance* : the backend is configured as OFFLINE_SOFT * anything else, or failed check: SHUNNED A new monitor table is also added: `readyset_status_log` . It has a similar structure of `mysql_server_replication_lag_log` , but instead of storing `repl_lag` (replication lag) the full output of `SHOW READYSET STATUS` is saved as a JSON (that can be queried using `JSON_EXTRACT()`	1 year ago
Javier Jaramago Fernández	7955dd21a8	Fix MySQL 8.4 replication lag monitoring - Closes #4727 Added support for new syntax to simulator.	1 year ago
René Cannaò	b53b5b24cc	Merge pull request #4810 from sysown/v3.0-SetThreadName_optional Make set_thread_name() optional	1 year ago
René Cannaò	2d797a55ba	Merge pull request #4809 from sysown/v3.0-fix_ssl_params_leak Fix memory leak on `ssl_params` fetch in Monitor - Port of #4795	1 year ago
René Cannaò	c2f1871adf	Make set_thread_name() optional	1 year ago
Javier Jaramago Fernández	ac3e75244c	Fix leak of SSL caches for auxiliary threads Whenever a thread creates MySQL connections using SSL params the thread-local caches are populated. These thread-local resources need to be free on thread exit via 'mysql_thread_end', otherwise these objects will be leak. This is not relevant for the fixed size thread-pools, but it's for workloads making use of temporary/auxiliary threads.	1 year ago
Javier Jaramago Fernández	9a08a12204	Fix memory leak on 'ssl_params' fetch in Monitor	1 year ago
Rahim Kanji	bd6674c383	* Introduced a 5% jitter to cache_ttl to increase the likelihood of distributing DNS resolution across multiple intervals, reducing the chance of simultaneous expiry times. * Added a small delay after queuing DNS records, allowing DNS resolver thread time to process as many hostnames as possible before assessing the need for additional helper threads. This helps prevent 'DNS resolver queue too big' warnings in the logs.	2 years ago
René Cannaò	04abd43a57	Initial commit with PostgreSQL support	2 years ago
René Cannaò	2726c27710	Merge pull request #4662 from sysown/v2.7_dns_cache_fixes DNS record will be removed if MySQL connection fails or times out for a specific hostname [v2.7]	2 years ago
Rahim Kanji	c5bc9ed2c6	DNS Cache Fixes * If a MySQL connection fails or times out for a specific hostname, the corresponding DNS record will be removed from DNS Cache. This ensures that outdated DNS records are not retained. (Rest of the DNS resolution process remains unchanged) * DNS cache functionality will automatically be disabled when ProxySQL is run with the -M or --no-monitor option.	2 years ago
René Cannaò	cf9d5fe21a	Merge pull request #4574 from sysown/set-thread-names Set thread names	2 years ago
Niels Vogell	89a3247959	Adding a null-check on a MySQL query result and avoiding redundant string initialization	2 years ago
Andriy Utkin	af2e223349	ConsumerThread: parametrize thread name	2 years ago

1 2 3 4 5 ...

441 Commits (2001c429f830ba2d93fe6ea90715afff65eec08a)