proxysql

Commit Graph

Author	SHA1	Message	Date
René Cannaò	8b2b29918a	Merge pull request #5291 from sysown/v3.0-misc260111 [skip-ci] Remove deprecated read_only_action implementations from MySQL and PgSQL HostGroups managers	4 months ago
René Cannaò	b1e37b3387	Merge branch 'v3.0' into v3.0_pgsql-use-ssl-issue-5284 Signed-off-by: René Cannaò <rene@proxysql.com>	4 months ago
René Cannaò	c6ed5b96cd	Merge pull request #5282 from sysown/v3.0_bind-format-issue-5273-fix Fix malformed Bind packets when client provides a single parameter format	4 months ago
Rene Cannao	4aedacd83b	[skip-ci] Remove deprecated read_only_action implementations from MySQL and PgSQL HostGroups managers The old read_only_action() implementations were marked for deletion after 2025-07-14. These were replaced with new implementation that doesn't depend on the admin table. This change removes the deprecated code paths to clean up the codebase.	4 months ago
René Cannaò	9c3f6b0aa5	Merge pull request #5286 from sysown/v3.0-poll_doc [skip-ci] Add comprehensive ProxySQL_Poll usage documentation throughout codebase	4 months ago
Rene Cannao	0699c7ba11	Add comprehensive ProxySQL_Poll usage documentation throughout codebase Enhance ProxySQL_Poll class documentation with detailed usage patterns: - lib/ProxySQL_Poll.cpp: Enhanced file-level documentation with architecture overview, template specialization, memory management, and event processing pipeline explanations - lib/MySQL_Thread.cpp: Added usage documentation for listener registration, removal patterns, client session setup, and main poll loop - lib/PgSQL_Thread.cpp: Added equivalent PostgreSQL usage documentation mirroring MySQL patterns with protocol-specific details - lib/mysql_data_stream.cpp: Documented cleanup, receive activity tracking, and send activity tracking patterns - lib/PgSQL_Data_Stream.cpp: Documented equivalent PostgreSQL data stream patterns for cleanup and activity tracking All documentation is placed directly where code is used, avoiding specific line numbers for better maintainability. Includes comprehensive explanations of when, why, and how ProxySQL_Poll methods are used throughout ProxySQL's event-driven architecture. [skip-ci]	4 months ago
René Cannaò	e6cbdcad7f	Merge pull request #5276 from sysown/v3.0_fork Fix critical bugs in close_all_non_term_fd() for fork/exec safety	4 months ago
Rahim Kanji	860657f8fa	use_ssl value from pgsql_users is properly assigned to the session	4 months ago
Rahim Kanji	556b1023c4	Removed change_user_auth_switch flag	4 months ago
René Cannaò	ef872c7f48	Merge pull request #5280 from sysown/v3.0-timezone-parser-fix Fix timezone parsing to support 3-component IANA names and hyphens	4 months ago
René Cannaò	591e1bca6b	Merge pull request #5277 from sysown/v3.0_5272 Fix #5272: Add mysql-select_version_forwarding variable for SELECT VERSION()	4 months ago
Rahim Kanji	5a7e7b30e7	Fix extended query Bind handling when a single parameter format is provided PostgreSQL allows a Bind message to specify a single parameter format (num_param_formats = 1), which applies to all parameters. libpq, however, always expects a format entry per parameter and previously sent uninitialized values for the remaining parameters when only one format was specified. This caused ProxySQL to forward malformed Bind packets to backend. ProxySQL now detects this case and propagates the single provided parameter format to all parameters, matching PostgreSQL semantics.	4 months ago
Rene Cannao	6e9abed581	Fix timezone parsing to support 3-component IANA names and hyphens This commit fixes a parsing error in the MySQL SET statement parser that occurred when processing `SET time_zone` statements with: 1. Three-component IANA timezone names (e.g., America/Argentina/Buenos_Aires) 2. Timezone names containing hyphens (e.g., America/Port-au-Prince) Previously, the regex pattern `(?:\w+/\w+)` only matched 2-component timezone names and did not support hyphens. This caused parsing errors logged as: "[ERROR] Unable to parse query. If correct, report it as a bug: SET time_zone=\"America/Argentina/Buenos_Aires\";" When multiplexing is enabled, this bug causes timestamps to be incorrectly written to the database. Changes: - Updated timezone regex from `(?:\w+/\w+)` to `(?:[\w-]+(?:/[\w-]+){1,2})` - Supports 2-3 components: Area/Location or Area/Country/Location - Supports hyphens in component names (e.g., Port-au-Prince) - Added comprehensive Doxygen documentation for timezone parsing - Extended TAP test cases with new timezone formats Note: Bare words like 'SYSTEM' and 'UTC' were already supported via other patterns in the parser (vp2 pattern for word matching). Fixes: #4993 Related: gemini-code-assist review comments	4 months ago
Rene Cannao	8c90bda52a	Address gemini-code-assist review comments for SSL keylog documentation This commit addresses all review comments from gemini-code-assist on PR #5279: 1. Fixed FLUSH LOGS documentation - clarified that file is reopened for appending, not truncating, and updated the note about preserving contents 2. Fixed callback documentation - clarified that the callback attaches to all frontend connections, not just admin connections 3. Updated security warning - focused on passive eavesdropping and offline decryption as the primary threats 4. Fixed typo: proxyql_ip -> proxysql_ip in tcpdump example 5. Removed misleading @see HPKP link - HPKP is unrelated to NSS Key Log Format and is a deprecated feature 6. Updated NSS Key Log Format URL to use official MDN link instead of unofficial mirror 7. Fixed buffer size comment to accurately reflect 256-byte buffer and 254-byte line length validation 8. Clarified fputs comment to emphasize the read lock's role in allowing concurrent writes from multiple threads	4 months ago
Rene Cannao	b39e193f4f	Fix critical issues in close_all_non_term_fd() per code review This commit addresses critical issues identified in PR #5276 by gemini-code-assist's code review, which could undermine the goal of being allocation-free and cause hangs or silent failures. Bug 1: Vector Passed by Value (Critical) ------------------------------------------ The function took std::vector<int> excludeFDs by value, causing heap allocation during the copy operation. This undermines the PR's goal of avoiding heap allocations after fork() to prevent deadlocks in multi-threaded programs. Fix: Change to pass by const reference to avoid heap allocation. void close_all_non_term_fd(const std::vector<int>& excludeFDs) Bug 2: Infinite Loop Risk (Critical) ------------------------------------ The loop used unsigned int for the variable while comparing against rlim_t (unsigned long long). If rlim_cur exceeded UINT_MAX, this would create an infinite loop. Fix: Use rlim_t type for the loop variable and cap at INT_MAX. for (rlim_t fd_rlim = 3; fd_rlim < nlimit.rlim_cur && fd_rlim <= INT_MAX; fd_rlim++) Bug 3: close_range() Detection Logic (High) ------------------------------------------ The original detection logic had two problems: 1. Executed close_range syscall twice on first successful call 2. Incorrectly cached availability on transient failures (EINTR), leaving file descriptors open without fallback Fix: Reordered logic to only cache on success, allow retry on transient failures. Only cache as "not available" on ENOSYS. For other errors (EBADF, EINVAL, etc.), don't cache - might be transient. Files Modified -------------- - include/proxysql_utils.h - lib/proxysql_utils.cpp	4 months ago
Rene Cannao	fc73ec1c50	Code review improvements: Add enum and refactor SELECT VERSION() handling - Add SelectVersionForwardingMode enum to replace magic numbers (0,1,2,3) - Refactor modes 2 and 3 to eliminate code duplication - Improve code readability and maintainability Addresses feedback from gemini-code-assist on PR #5277	4 months ago
Rene Cannao	442635b721	Add comprehensive documentation for SSL/TLS key logging feature This commit adds extensive documentation for the ssl_keylog_file feature (introduced in PR #4236), which enables TLS key logging for debugging encrypted traffic. ## Background The ssl_keylog_file variable (exposed as admin-ssl_keylog_file in SQL interface) allows ProxySQL to write TLS secrets to a file in NSS Key Log Format. These secrets can be used by tools like Wireshark and tshark to decrypt and analyze TLS traffic for debugging purposes. ## Changes ### Inline Documentation (Code) 1. include/proxysql_sslkeylog.h (+96 lines) - File-level documentation explaining the module purpose and security - Doxygen comments for all 5 public APIs - Thread-safety annotations - Parameter descriptions and return values 2. lib/proxysql_sslkeylog.cpp (+136 lines) - Implementation-level documentation - Algorithm explanations (double-checked locking, thread safety) - Reference to NSS Key Log Format specification 3. include/proxysql_admin.h (+19 lines) - Variable documentation for ssl_keylog_file - Path handling rules (absolute vs relative) - Security implications ### Developer Documentation (doc/ssl_keylog/ssl_keylog_developer_guide.md) Target audience: Developers working on ProxySQL codebase Contents: - Variable naming convention (SQL vs config file vs internal) - Architecture diagrams - Thread safety model (pthread rwlock) - NSS Key Log Format specification - Complete API reference for all public functions - Integration points in the codebase - Security considerations and code review checklist - Testing procedures ### User Documentation (doc/ssl_keylog/ssl_keylog_user_guide.md) Target audience: End users and system administrators Contents: - What is SSL key logging and when to use it - Variable naming: admin-ssl_keylog_file (SQL) vs ssl_keylog_file (config) - Step-by-step enable/disable instructions - Path resolution (absolute vs relative) - Log rotation procedures - Production workflow: tcpdump capture → offline analysis - Wireshark (GUI) integration tutorial - tshark (command-line) usage examples - Troubleshooting common issues - Security best practices - Quick reference card ## Key Features Documented 1. Variable Naming Convention - SQL interface: SET admin-ssl_keylog_file = '/path'; - Config file: ssl_keylog_file='/path' (in admin_variables section) - Internal code: ssl_keylog_file 2. Production Workflow - Capture traffic with tcpdump (no GUI on production server) - Transfer pcap + keylog to analysis system - Analyze offline with Wireshark (GUI) or tshark (CLI) 3. tshark Examples - Command-line analysis of encrypted traffic - Filter examples for debugging TLS issues - JSON export for automated analysis ## Security Notes The documentation emphasizes that: - Key log files contain cryptographic secrets that decrypt ALL TLS traffic - Access must be restricted (permissions 0600) - Only enable for debugging, never in production - Securely delete old key log files ## Files Modified - include/proxysql_admin.h - include/proxysql_sslkeylog.h - lib/proxysql_sslkeylog.cpp ## Files Added - doc/ssl_keylog/ssl_keylog_developer_guide.md - doc/ssl_keylog/ssl_keylog_user_guide.md	4 months ago
Rene Cannao	366164ab26	Fix #5272 : Add mysql-select_version_forwarding variable for SELECT VERSION() Since ProxySQL 3.0.4, SELECT VERSION() queries were intercepted and returned ProxySQL's mysql-server_version variable instead of proxying to backends. This broke SQLAlchemy for MariaDB which expects "MariaDB" in the version string. This commit adds a new variable `mysql-select_version_forwarding` with 4 modes: - 0 = never: Always return ProxySQL's version (3.0.4+ behavior) - 1 = always: Always proxy to backend (3.0.3 behavior) - 2 = smart (fallback to 0): Try backend connection, else ProxySQL version - 3 = smart (fallback to 1): Try backend connection, else proxy (default) The implementation includes: - New global variable mysql_thread___select_version_forwarding - New function get_backend_version_for_hostgroup() to peek at backend connection versions without removing them from the pool - Modified SELECT VERSION() handler to support all 4 modes - ProxySQL backend detection to avoid recursion Mode 3 (default) ensures SQLAlchemy always gets the real MariaDB version string while maintaining fast response when connections are available.	4 months ago
Rene Cannao	2448b12a56	Fix critical bugs in close_all_non_term_fd() for fork/exec safety This commit fixes two critical bugs in close_all_non_term_fd() that caused undefined behavior and potential deadlocks when called after fork() before execve() in multi-threaded programs. Bug 1: Self-Referential Directory FD Closure ---------------------------------------------- When iterating through /proc/self/fd, opendir() creates a file descriptor for the directory stream. This fd appears in the enumeration while we're iterating, and if we close it, readdir() operates on a corrupted DIR* stream, causing undefined behavior, crashes, or missed file descriptors. Fix: Use dirfd() to obtain the directory's fd and explicitly skip closing it. Bug 2: Heap Allocation After fork() in Multi-Threaded Programs ---------------------------------------------------------------- In multi-threaded programs, when fork() is called while other threads hold malloc locks, the child process inherits a "frozen" state where those locks remain locked (the owning threads don't exist in the child). Any heap allocation (malloc/free/new/delete) in the child before execve() can deadlock. The original code used: - std::stol(std::string(dir->d_name)) - creates a temporary std::string - std::find() - may allocate internally Fix: Replace with heap-allocation-free alternatives: - atoi(dir->d_name) instead of std::stol(std::string(...)) - Simple C loops instead of std::find() Additional Improvements ----------------------- 1. Added close_range() syscall support (Linux 5.9+) with runtime detection - O(1) atomic operation, most efficient method - Only used when excludeFDs is empty (closes all fds >= 3) - Falls back to /proc/self/fd iteration when excludeFDs is non-empty 2. Added extensive doxygen documentation covering: - Security implications (preventing fd leaks to child processes) - Resource management (preventing fd exhaustion) - Deadlock prevention in multi-threaded fork() contexts - Implementation details (three strategies: close_range, /proc/self/fd, rlimit) - fork() safety design considerations - Example usage and portability notes 3. Added required includes: dirent.h, sys/syscall.h, linux/close_range.h Workflow Safety --------------- The function is now safe to use in the common fork() -> close_all_non_term_fd() -> execve() workflow, even in multi-threaded programs. Files Modified -------------- - lib/proxysql_utils.cpp	4 months ago
René Cannaò	af23187865	Merge pull request #5270 from sysown/v3.0_restapi_improvement Use parameterized prepared statements in REST API for safer SQL execution	5 months ago
Rahim Kanji	79df69332d	Refactor find_script() to use parameterized prepared statements (via execute_prepared()) for safer SQL execution.	5 months ago
Rahim Kanji	14aef13827	Add method to execute already prepared SQLite3 statements, supporting both non-SELECT and SELECT queries.	5 months ago
Rahim Kanji	01a5b23b22	refactor connection pool put_connection to use mmsd and improve debug handling * Change MySQL_Monitor_Connection_Pool::put_connection signature to accept MySQL_Monitor_State_Data* instead of raw MYSQL/port. Centralize access to mysql and port via mmsd, reducing parameter mismatch and misuse. * Improve DEBUG bookkeeping: ensure connections are properly unregistered from the global debug registry with clearer assertions and logs. * Add consistent proxy_debug messages for connection register/unregister events. * Simplify server lookup/creation logic when returning connections to the pool. * Fix ordering of error handling to always unregister before closing connections. * Minor cleanup: remove unused labels/variables and modernize casts. * This refactor improves correctness, debuggability, and safety of monitor connection lifecycle management.	5 months ago
René Cannaò	faa64a570d	Merge pull request #5259 from sysown/v3.0_mysql_monitor_cur_cmd_cmnt_fix Fix: Make cur_cmd_cmnt thread-safe	5 months ago
Javier Jaramago Fernández	5c8a32a0b0	Merge branch 'v3.0' of github.com:sysown/proxysql into v3.0-handle_unexp_ping	5 months ago
Rahim Kanji	91e20648f2	Fixed an issue where cur_cmd_cmnt was shared across threads	5 months ago
Javier Jaramago Fernández	6fea828e86	Improve logging in unexpected COM_PING packet handling Logging messages now include 'client address', 'session status' and 'data stream status'. Client address is also logged when OK packets are dispatched, this should help tracking if a client has received the expected packets or not.	5 months ago
René Cannaò	88edaac61b	Merge pull request #5258 from sysown/misc251219 Documentation additions and bug fix for vacuum_stats()	5 months ago
René Cannaò	0f7ff1f374	Merge branch 'v3.0' into v3.0_pgsql-query-digest-gen-5253 Signed-off-by: René Cannaò <rene@proxysql.com>	5 months ago
René Cannaò	2667540fcc	Merge pull request #5237 from sysown/v3.0_pgsql-monitor-sslsupport-5205 Add SSL support for backend connections in PGSQL monitor	5 months ago
Javier Jaramago Fernández	d0e88599ee	Add special handling for unexpected COM_PING packets Implements a workaround for the handling of unexpected 'COM_PING' packets received during query processing, while a resultset is yet being streamed to the client. Received 'COM_PING' packets are queued in the form of a counter. This counter is later used to sent the corresponding number of 'OK' packets to the client after 'MySQL_Session' has finished processing the current query.	5 months ago
Rene Cannao	efe0d4fe61	Add extensive doxygen documentation for vacuum_stats and stats_pgsql_stat_activity This commit documents: 1. The vacuum_stats() function's purpose, behavior, and the reason why stats_pgsql_stat_activity is excluded from bulk deletion operations 2. The fact that stats_pgsql_stat_activity is a SQL VIEW (not a table) and attempting DELETE on it would cause SQLite error: "cannot modify stats_pgsql_stat_activity because it is a view" The documentation explains: - Why TRUNCATE stats_mysql_query_digest triggers vacuum_stats(true) - Why both MySQL and PostgreSQL tables are cleared regardless of protocol - How the view is automatically cleared via its underlying table stats_pgsql_processlist - The importance of keeping the view excluded from deletion lists	5 months ago
Rahim Kanji	5e75264bb3	Updated TAP test	5 months ago
Rahim Kanji	5b3805ad7a	Refactored comment handling * Removed is_cmd (/!) handling Proper handling of Keep Comments and First comment extraction * Proper handling for nested comments	5 months ago
Rahim Kanji	e70fcbf021	* Add dedicated handling for double-quoted PostgreSQL identifiers * Added crash payload testing * Fixed unterminated comments handling	5 months ago
René Cannaò	6ee087c73c	Merge pull request #5250 from sysown/v3.0-issue5248 Fix cache_empty_result=0 not caching non-empty resultsets (issue #5248)	5 months ago
Rahim Kanji	fd53642f12	Added pgsql-query_digests_stages_test-t to groups.json	5 months ago
Rahim Kanji	39728b2dc8	Add missing pgsql_tokenizer.cpp	5 months ago
Rahim Kanji	42864e8867	Improved Tokenizer for PostgreSQL - Added `process_pg_typecast()` to handle PostgreSQL type cast syntax (::) - Recognizes type casts in various contexts: 'value'::type, column::type, etc. - Added `process_array_literal()` for PostgreSQL array processing - Handles both ARRAY[] constructor and {} literal syntax - Processes multi-dimensional arrays and nested array structures - Added `process_literal_prefix_type()` for PostgreSQL prefixed literals - Processes E'' escape string constants with backslash escapes - Handles U&'' Unicode string literals with optional UESCAPE clauses - Supports x'' hex string literals and b'' bit string literals - Manages B'' bit strings and bytea literals (\\xDEADBEEF format) - Added `process_replace_boolean()` for boolean literal replacement - Replaces TRUE and FALSE literals with parameter placeholders - Maintains case-insensitive matching (true, True, TRUE, etc.) - Preserves boolean context in expressions and WHERE clauses	5 months ago
Rene Cannao	2987242d4f	Fix cache_empty_result=0 not caching non-empty resultsets (issue #5248 ) The `cache_empty_result` field in query rules has three possible values: • -1: Use global setting (`query_cache_stores_empty_result`) • 0: Do NOT cache empty resultsets, but cache non-empty resultsets • 1: Always cache resultsets (both empty and non-empty) Previously, when `cache_empty_result` was set to 0, nothing was cached at all, even for non-empty resultsets. This prevented users from disabling caching for empty resultsets while still allowing caching of non-empty resultsets on a per-rule basis. Changes: 1. Modified caching logic in MySQL_Session.cpp and PgSQL_Session.cpp to add the condition `(qpo->cache_empty_result == 0 && MyRS->num_rows)` (MySQL) and `(qpo->cache_empty_result == 0 && num_rows)` (PgSQL) to allow caching when cache_empty_result=0 AND result has rows. 2. Added comprehensive Doxygen documentation in query_processor.h explaining the semantics of cache_empty_result values. 3. Updated Query_Processor.cpp with inline comments explaining the three possible values. Now when cache_empty_result is set to 0: - Empty resultsets (0 rows) are NOT cached - Non-empty resultsets (>0 rows) ARE cached - This matches the intended per-rule behavior described in issue #5248. Fixes: https://github.com/sysown/proxysql/issues/5248	5 months ago
René Cannaò	5a314d2364	Merge pull request #4889 from sysown/v3.0_get_server_version Added handling of SELECT @@version and SELECT VERSION() without backend	5 months ago
René Cannaò	8cf3e59ead	Merge pull request #5247 from sysown/v3.0-issue5246 Fix: Automatic prefix stripping for mysql_variables, pgsql_variables, and admin_variables config parsing	5 months ago
Rene Cannao	0b2bc1bf22	Fix SQL injection vulnerability in Read_Global_Variables_from_configfile Replace sprintf-based SQL query construction with prepared statements using bound parameters to prevent SQL injection attacks. This addresses the security issue identified in PR #5247 review. Changes: - Use SQLite prepared statement with placeholders ?1, ?2 - Bind variable names and values securely using proxy_sqlite3_bind_text - Use ASSERT_SQLITE_OK for error handling as per ProxySQL conventions - Remove malloc/sprintf vulnerable code pattern - Add necessary includes for SQLite functions and ASSERT_SQLITE_OK macro Security: SQL injection could have occurred if configuration variable names or values contained malicious quotes. Prepared statements eliminate this risk.	5 months ago
Rahim Kanji	fae283cf7e	Add SSL and non-SSL connection OK metrics for PostgreSQL monitor connections Adds two new metrics, ssl_connections_OK and non_ssl_connections_OK, to improve visibility into PostgreSQL monitor connection status.	5 months ago
Rene Cannao	6c97d3d244	Add extensive Doxygen documentation for ProxySQL_Config and Read_Global_Variables_from_configfile This commit adds detailed Doxygen documentation for: 1. The ProxySQL_Config class - describes its role in configuration management 2. The Read_Global_Variables_from_configfile() method - documents its behavior, parameters, return value, and the automatic prefix stripping feature The documentation explains the automatic prefix stripping behavior that handles cases where users mistakenly include module prefix (e.g., "mysql-") in variable names within configuration files.	5 months ago
Rene Cannao	7ebdf561ca	Fix automatic prefix stripping to work with libconfig lookup The previous implementation stripped the prefix before calling group.lookupValue(), which would fail because the config file contains the prefixed name (e.g., "mysql-log_unhealthy_connections"). The lookup must use the original name from the config file. This commit moves the prefix stripping logic to after the value lookup but before constructing the SQL query, ensuring both: 1. The correct value is retrieved from the config using the original prefixed name 2. The variable is stored in the database with a single prefix Also includes a test to verify the fix works for mysql_variables, pgsql_variables, and admin_variables sections.	5 months ago
Rene Cannao	b4683569d6	Add automatic prefix stripping for mysql_variables, pgsql_variables, and admin_variables config parsing When users mistakenly include the module prefix (e.g., mysql-log_unhealthy_connections) in the mysql_variables section, the variable gets stored with a double prefix (e.g., mysql-mysql-log_unhealthy_connections). This fix automatically strips the prefix if present, ensuring variables are stored correctly. The same logic applies to pgsql_variables (pgsql-) and admin_variables (admin-). Fixes #5246	5 months ago
Rene Cannao	ec1247f2a9	Add Doxygen docs for MySQL_Data_Stream::check_data_flow()	5 months ago
Rene Cannao	4044a40794	Skip bidirectional data check for permanent fast-forward sessions Allow permanent fast-forward sessions (SESSION_FORWARD_TYPE_PERMANENT) to continue processing when bidirectional data flow is detected, instead of treating it as a fatal error. This prevents unnecessary session termination in these specific cases while maintaining the original strict validation for all other session types.	5 months ago
Rahim Kanji	f507903743	Added nested comments support for PostgreSQL	5 months ago

1 2 3 4 5 ...

4592 Commits (3.0.6)