proxysql

Commit Graph

Author	SHA1	Message	Date
Rene Cannao	6b46b26931	chore: implement PR review feedback for release tier system - Makefile: normalized GIT_VERSION by stripping 'v' prefix before arithmetic. - Makefile: corrected .PHONY declaration for build_deps_debug_default. - Admin Handler: properly guarded TSDB commands to return error when disabled. - REST API: removed redundant nested guards inside tsdb_resource class. - Statistics: implemented rigorous input validation for set_variable. - Statistics: added return value checks and rollback logic for TSDB migrations. - Statistics: implemented bounded concurrency (batching) for backend probes. - FFTO: implemented cumulative buffer limit enforcement in MySQL and PgSQL. - FFTO: ensured initial packet capture in PgSQL_Session by early initialization. - FFTO: improved reassembly buffer compaction logic. - Headers: default-initialized ffto_bypassed to false. - Code cleanup: merged redundant PROXYSQLFFTO guards.	1 month ago
Rene Cannao	c2f678a9de	feat: implement release tiers and feature guards for v3.0, v3.1, and v4.0 This commit introduces a structured release tier system controlled by environment variables, allowing three distinct ProxySQL versions to be built from the same codebase while maintaining a clear upgrade path. Key Changes: - Tier 1 (v3.0.x Stable): Default build including bug fixes and core enhancements. - Tier 2 (v3.1.x Innovative): Enabled via PROXYSQL31=1. Adds FFTO and TSDB. - Tier 3 (v4.0.x AI/MCP): Enabled via PROXYSQLGENAI=1. Adds the AI stack. Implementation Details: - Makefile: Implemented dynamic versioning logic that increments the minor version for v3.1 and the major version for v4.0. Added tier documentation. - Feature Guards: Introduced PROXYSQLFFTO and PROXYSQLTSDB guards across ProxySQL_Admin, ProxySQL_Statistics, REST API, and HTTP server. - FFTO: Guarded Fast Forward Traffic Observer in MySQL and PgSQL sessions, including thread-local variables and configuration parameters. - TSDB: Guarded the entire Time Series Database subsystem, including SQLite table definitions, background loops, and REST API endpoints. - Build Fixes: * Fixed 'proxy_debug_func' linkage by adding extern "C" and moving the definition outside of DEBUG guards to ensure non-debug builds link correctly. * Corrected misplaced/nested DEBUG and PROXYSQLFFTO guards. * Verified build stability for all tiers (unset, PROXYSQL31=1, PROXYSQLGENAI=1).	1 month ago
René Cannaò	16f8837e0d	Merge branch 'v3.0' into v3.0-pgsql_monitor_repl_lag Signed-off-by: René Cannaò <rene@proxysql.com>	1 month ago
Rene Cannao	c4fc60bdb0	fix: address security and logic issues in PostgreSQL replication lag monitoring - Fix SQL injection in pt-heartbeat query concatenation.\n- Fix copy-paste errors between mysql_thread_ and pgsql_thread_ variables.\n- Restore monitor_replication_lag_group_by_host logic.	1 month ago
Javier Jaramago Fernández	ff729a657e	chore: disable 'pt-heartbeat' in 'PgSQL_Threads_Handler::get_variable' This isn't for functional purposes, just for consistency in reporting the motivation for disabling the feature in the codebase.	1 month ago
Javier Jaramago Fernández	a4ea3b2174	Merge branch 'v3.0' of github.com:sysown/proxysql into v3.0-pgsql_monitor_repl_lag	1 month ago
Javier Jaramago Fernández	4103d69979	feat: disable 'pt-heartbeat' support for PostgreSQL The feature has been disabled until 'pt-heartbeat' support for PostgreSQL is fixed. See https://perconadev.atlassian.net/browse/PT-2030.	1 month ago
Rene Cannao	acef17a8c1	Merge branch 'v3.0-ff_inspect' into v3.0-unified-large-pr # Conflicts: # test/tap/tests/Makefile	2 months ago
Rene Cannao	e2ebc0c449	Merge branch 'v4.0-mcp-stats2' into v3.0-unified-large-pr	2 months ago
Rene Cannao	5519080ac2	mcp stats: address review findings and remove --genai CLI flag	2 months ago
Javier Jaramago Fernández	ba5af979ff	feat-wip: Add replication lag support for PostgreSQL monitoring	2 months ago
Rene Cannao	1c25787b88	Introduce {mysql,pgsql}-query_processor_first_comment_parsing variable (#5384 ) This universal fix allows controlling when the first comment of a query is processed relative to the query rules. By setting this variable to 1, ProxySQL-specific annotations (like GTID) can be parsed immediately, allowing subsequent query rules to strip them from the query string. This prevents prepared statement cache bloat and improves backend statement reuse when annotations with unique literals are used.	2 months ago
Rene Cannao	1258a1f2f2	Address PR reviews: improve performance, safety, and robustness of FFTO	2 months ago
Rene Cannao	5d786520b0	Merge remote-tracking branch 'origin/v3.0' into merge/v3.0-into-v3.0-5243 # Conflicts: # include/PgSQL_Logger.hpp # lib/Makefile # lib/PgSQL_Logger.cpp # lib/ProxySQL_Admin.cpp # test/tap/groups/groups.json	2 months ago
Rene Cannao	e762f91cda	MCP stats: add in-memory processlist filtering and PgSQL concurrency stress TAP Replace direct stats-schema reads in MCP show_processlist with in-memory processlist access via SQL3_Processlist query options. Add processlist query option types in proxysql_admin interfaces and implement filtering, sorting, and pagination post-processing in both MySQL and PgSQL thread implementations. Introduce configurable MCP cap mcp_stats_show_processlist_max_rows (default 200, max 1000), wire it through MCP variable loading, and expose it in default config. Expand Stats_Tool_Handler processlist handling to parse filter/sort arguments, enforce cap metadata, and return consistent MCP payload fields from in-memory snapshots. Add extensive doxygen documentation in the touched headers and source blocks to describe API contracts, filtering behavior, and runtime constraints. Add new TAP test mcp_pgsql_concurrency_stress-t that generates sustained concurrent PgSQL traffic (simple reads, read/write table workload, randomized pg_sleep) while polling MCP show_processlist and show_queries in parallel with filter and ordering assertions. Validation performed locally: mcp_pgsql_concurrency_stress-t (31 assertions) and mcp_show_queries_topk-t (12 assertions).	2 months ago
Rene Cannao	acbd7d0df3	Implement PostgreSQL eventslog circular buffer and sink pipeline	2 months ago
Rene Cannao	2a46c239be	Add PostgreSQL advanced eventslog schema and variable scaffolding	2 months ago
Rene Cannao	3a3be5a559	Implement Fast Forward Traffic Observer (FFTO) for MySQL and PostgreSQL - Added TrafficObserver base interface - Implemented MySQLFFTO and PgSQLFFTO classes for protocol-aware observation - Integrated FFTO hooks into MySQL and PostgreSQL session handlers in FAST_FORWARD state - Added configuration variables: mysql-ffto_enabled, mysql-ffto_max_buffer_size, pgsql-ffto_enabled, pgsql-ffto_max_buffer_size - Implemented query digest extraction and reporting parity with internal stats - Added session-level bypass when packet size exceeds buffer threshold - Included architectural documentation in doc/ffto_design.md	2 months ago
Rene Cannao	caf324f911	logger: harden buffered logging flush/rotation semantics This commit addresses three correctness regressions introduced by thread-local log buffering in PR #5364 (query logging performance): 1) stale logfile pointer/UAF race during concurrent rotate/close 2) stale logfile-open state after close 3) non-global flush behavior in admin/format-switch paths What was fixed - Make `flush_and_rotate()` consume the current logfile pointer under lock - Signature changed from `std::fstream` to `std::fstream&` - Prevents dereferencing a stale stream pointer captured before lock acquisition while another thread rotates/closes the file - Updated declaration/definition and all call sites - Add explicit synchronization for cross-thread buffer draining - Added `LogBufferThreadContext::buffer_lock` - Any path that appends or flushes a thread buffer now locks this mutex - Guarantees force-flush from admin/config paths cannot race with worker-thread appends on the same context - Restore global forced flush semantics where required - Extended `MySQL_Logger::flush` and `PgSQL_Logger::flush` to `flush(bool force = false)` - `force=false`: preserves existing low-overhead worker-loop behavior (per-thread timeout-based flush) - `force=true`: snapshots all known thread contexts and drains both events/audit buffers regardless of timeout - `flush_log()` now calls `flush(true)` before file rotation, so admin flush and format-switch operations no longer miss pending thread buffers - Avoid unintended rotation during forced draining - In `force=true` path, flush uses `rotate_fn=nullptr` - Drains buffered payload into the current file first - `flush_log()` then performs one controlled rotate/open step - Fix logfile state tracking after close - `events_close_log_unlocked()` and `audit_close_log_unlocked()` now set `logfile_open=false` when the stream is closed - Prevents write paths from treating a closed stream as open - Remove per-thread-context dependency during metadata header write - `events_open_log_unlocked()` now uses a local `LogBuffer` for metadata emission in format=1 instead of reusing a thread context buffer - Keeps open/rotate path independent from worker context lifecycle - Keep callers consistent and non-duplicative - Since `flush_log()` now force-drains internally, removed redundant explicit `flush()` calls from: - MySQL/PgSQL `eventslog_format` switch handlers - `ProxySQL_Admin::flush_logs` Behavioral outcome - No stale stream pointer use when close/rotate interleaves with flush - No false-positive logfile-open state after close - `FLUSH LOGS` and eventslog format switch now drain all known thread buffers before rotating, preventing dropped/misplaced buffered records Validation - Built modified objects directly - Ran full debug build with GENAI enabled: make clean && export PROXYSQLGENAI=1 && make debug -j24 Build completed successfully.	2 months ago
René Cannaò	568fe1524e	Merge pull request #5364 from mevishalr/feature/query-logging-improvements Optimize query logging performance (#5243)	2 months ago
Rene Cannao	cc53ddca23	Merge branch 'v3.0-openssl-fix' into v3.0-test0213	2 months ago
Rene Cannao	a69b925932	Minor typo fixes	2 months ago
vramesha	15a4ed66e9	Optimize query logging performance (#5243 ) Detailed Changes: - Introduced LogBuffer and LogBufferThreadContext to implement per-thread buffering, reducing lock contention. - Replaced localtime() with localtime_r() for improved performance and thread safety during timestamp generation. - Implemented configurable sampling for event logs to reduce I/O overhead. - Added global configuration variables to control flush size, flush interval, and sampling rate for event and audit logs.	2 months ago
Rene Cannao	778e01174c	Add protocol labels to Thread Handler metrics and enable PostgreSQL metrics This commit extends PR #5069 by adding protocol labels to distinguish MySQL and PostgreSQL metrics in the Thread Handler modules. Changes: - Add { "protocol", "mysql" } labels to 65 MySQL_Thread.cpp metrics - Add { "protocol", "pgsql" } labels to 60 PgSQL_Thread.cpp metrics - Enable GloPTH (PostgreSQL threads handler) metrics export - Document Query Cache limitation (template base class collision) This resolves Issue #5068 by ensuring PostgreSQL prometheus metrics are properly exported with protocol labels, distinguishing them from MySQL metrics. Related: #5068, #5069	2 months ago
René Cannaò	fcaa904135	fix: crash on macOS/FreeBSD when running PostgreSQL queries Fix NULL pointer dereference in CopyCmdMatcher that caused ProxySQL to crash immediately when running any PostgreSQL query on macOS and FreeBSD systems. Root cause: copy_cmd_matcher was guarded by #ifdef IDLE_THREADS, but IDLE_THREADS is explicitly disabled on macOS (__APPLE__) and FreeBSD in proxy_defines.h. This caused the pointer to be NULL when accessed. The fix removes #ifdef IDLE_THREADS guards from copy_cmd_matcher since COPY command detection is a fundamental PostgreSQL feature for fast- forward mode and has no dependency on idle threads. Affected systems: macOS, FreeBSD (where IDLE_THREADS is disabled) Fixes: #5359	2 months ago
René Cannaò	d43ae6e121	Surgical fixes for macOS compatibility: headers, types, and Makefile linking	3 months ago
Rene Cannao	0699c7ba11	Add comprehensive ProxySQL_Poll usage documentation throughout codebase Enhance ProxySQL_Poll class documentation with detailed usage patterns: - lib/ProxySQL_Poll.cpp: Enhanced file-level documentation with architecture overview, template specialization, memory management, and event processing pipeline explanations - lib/MySQL_Thread.cpp: Added usage documentation for listener registration, removal patterns, client session setup, and main poll loop - lib/PgSQL_Thread.cpp: Added equivalent PostgreSQL usage documentation mirroring MySQL patterns with protocol-specific details - lib/mysql_data_stream.cpp: Documented cleanup, receive activity tracking, and send activity tracking patterns - lib/PgSQL_Data_Stream.cpp: Documented equivalent PostgreSQL data stream patterns for cleanup and activity tracking All documentation is placed directly where code is used, avoiding specific line numbers for better maintainability. Includes comprehensive explanations of when, why, and how ProxySQL_Poll methods are used throughout ProxySQL's event-driven architecture. [skip-ci]	3 months ago
Javier Jaramago Fernández	5c8a32a0b0	Merge branch 'v3.0' of github.com:sysown/proxysql into v3.0-handle_unexp_ping	4 months ago
Javier Jaramago Fernández	6fea828e86	Improve logging in unexpected COM_PING packet handling Logging messages now include 'client address', 'session status' and 'data stream status'. Client address is also logged when OK packets are dispatched, this should help tracking if a client has received the expected packets or not.	4 months ago
René Cannaò	2667540fcc	Merge pull request #5237 from sysown/v3.0_pgsql-monitor-sslsupport-5205 Add SSL support for backend connections in PGSQL monitor	4 months ago
Rahim Kanji	fae283cf7e	Add SSL and non-SSL connection OK metrics for PostgreSQL monitor connections Adds two new metrics, ssl_connections_OK and non_ssl_connections_OK, to improve visibility into PostgreSQL monitor connection status.	4 months ago
René Cannaò	3c4e09fec0	Merge pull request #5225 from sysown/v3.0_refactor_prepared_statement_cache_design_5211 Refactored Prepared-Statement Cache Design (Lock-Free Hot Path) - Part 2	4 months ago
Rahim Kanji	9c0e14a5d1	Replace rand() with lock-free Xoshiro128++ PRNG	4 months ago
Rahim Kanji	1a48aadf5e	Code cleanup	4 months ago
René Cannaò	8be91071ef	Merge pull request #5190 from sysown/v3.0_default_interface_global_var_5189 Ensure default mysql-interfaces=0.0.0.0:6033 and pgsql-interfaces=0.0.0.0:6133 are set	5 months ago
Rahim Kanji	48928ce04e	Ensure default mysql-interfaces=0.0.0.0:6033 and pgsql-interfaces=0.0.0.0:6133 are set	5 months ago
Rahim Kanji	86c17f04d7	Replace single sleep with looped delay for simulated watchdog test	5 months ago
Wazir Ahmed	5219d4be93	processlist: Prevent crash due to SQLITE_TOOBIG - Add new mysql/pgsql variable `processlist_max_query_length`. - Min: 1K - Max: 32M - Default: 2M - Truncate current query based on the configuration before inserting into `stats_*_processlist` tables. - Refactor/fix code related to other processlist configurations. 1. `session_idle_show_processlist` value was not updated in `ProxySQL_Admin.variables`. 2. Pass processlist config as an argument to `MySQL_Threads_Handler::SQL3_Processlist` instead of using thread-local variables. Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	6 months ago
René Cannaò	abafaccbc0	Merge pull request #5156 from sysown/v3.0_pgsql_stat_activity_issue_4682 Add backend_pid and backend_state to stats_pgsql_processlist and create stats_pgsql_stat_activity view	6 months ago
Rahim Kanji	f3ea179384	Add backend PID/state to stats_pgsql_processlist and create pg_stat_activity-style view - Added `backend_pid` and `backend_state` columns to `stats_pgsql_processlist` to display PostgreSQL backend process ID and connection state. - Created `stats_pgsql_stat_activity` view on top of `stats_pgsql_processlist` with column aliases matching PostgreSQL's `pg_stat_activity` for consistency.	6 months ago
Rahim Kanji	4d91452fc5	Implement backend query cancellation for PostgreSQL #5136 Add support for PostgreSQL query cancellation and backend termination features to allow clients to cancel long-running queries and terminate connections through the standard PostgreSQL protocol. Features implemented: - Intercept pg_backend_pid() queries and return ProxySQL session thread ID - Intercept pg_terminate_backend() to terminate client connections asynchronously - Intercept pg_cancel_backend() to cancel queries on backend connections - Support Cancel Request protocol via separate connection with PID and secret key validation - Return BackendKeyData message on successful authentication with session thread ID and unique cancel secret key This enables clients to use standard PostgreSQL cancellation mechanisms (pg_cancel_backend, pg_terminate_backend, and Cancel Request protocol) while ProxySQL maintains proper session isolation and maps client requests to appropriate backend connections.	6 months ago
Rahim Kanji	a89291c9a9	Added query logging support for PostgreSQL	7 months ago
Rahim Kanji	8d8577b0ad	Ensure PgSQL_Threads_Handler and PgSQL_HostGroups_Manager objects are released when ProxySQL shuts down or restarts.	7 months ago
Rahim Kanji	ce94a7080a	Merge branch v3.0_session_param_reset_connection into v3.0_extended_query_protocol	8 months ago
Rahim Kanji	50243ef283	Corrected RESET/DEFAULT logic * Now will actually reset value to server default (represented by nullptr value), if that parameter is not set in startup parameter. * Default parameter values (pgsql_default_*) will now be set only for critical parameters.	8 months ago
Rahim Kanji	344ba25777	Add Watchdog PROXYSQLTEST covering MySQL and PgSQL threads	9 months ago
Rahim Kanji	3619fa5d21	Added pgsql-max_stmts_per_connection variable	9 months ago
Rahim Kanji	bc87d0d901	* For a Describe Portal message, peek at the next message in the extended query frame. If it is an Execute message, indicating that client is sending a Bind/Describe/Execute sequence skip processing the Describe Portal. This is because libpq automatically includes a Describe Portal during execution, and sending it again would be redundant. * Removed server_capabilities from PgSQL modules	9 months ago
Rahim Kanji	87ad15b01c	* Removed autocommit_on_hostgroup from PostgreSQL modules * Introduced an error generator method to reduce code duplication * Added several comments for clarity * Addressed edge cases related to libpq limitations	9 months ago
Rahim Kanji	48ba491cb7	Add support for unnamed Describe Portal and improve extended query handling - Always send Describe Portal to the backend - Enforce check for unnamed portals in Bind, Describe, Close, and Execute; raise errors for named portals - Improve error handling for extended query flow - Introduce Extended_Query_Info struct to unify extended query-related members and reduce redundancy	9 months ago

1 2 3

114 Commits (2fcc5729eec032a0e3d95a72fbdc08e7903940cc)