proxysql

Commit Graph

Author	SHA1	Message	Date
Wazir Ahmed	8d94919b22	fix: preserve EOF capability in enforced mode - Keep `CLIENT_DEPRECATE_EOF` enabled during frontend handshake when `session_track_variables=ENFORCED`. - Avoid unsafe mysql_error() calls in session-track TAP tests. Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	1 month ago
Wazir Ahmed	885712c679	Merge branch 'v3.0' into session-track-system-variable	1 month ago
Rene Cannao	266a10617f	feat: declare mysql_thread___zstd_compression_level thread-local and struct member	1 month ago
Rene Cannao	7e1a78fd6b	lint: fix reserved identifiers and include guards - Rename __HGM_TABLES_SIZE -> HGM_TABLES_SIZE_ (Base/MySQL/PgSQL HGM) - Rename __TYPE_SIZE -> TYPE_SIZE_ (Base/MySQL/PgSQL HGM) - Fix include guard defines: __PROXYSQL_* -> PROXYSQL_* - Fix include guard comments: __CLASS_* -> PROXYSQL_*_H (29 files) - Add uint8_t underlying type to small enums	2 months ago
Rene Cannao	9bdfb8b1cd	lint: optimize enum types and fix reserved identifiers in Query_Cache, admin, thread, debug performance-enum-size fixes: - p_qc_counter, p_qc_gauge: add uint8_t (7 and 2 values) - p_admin_counter, p_admin_gauge, p_admin_dyn_counter, p_admin_dyn_gauge: add uint8_t - p_debug_dyn_counter: add uint8_t (2 values) bugprone-reserved-identifier fixes: - Rename __size -> SIZE_ in all metric enums - Update std::array size references across multiple files - Update prometheus_helpers.h template parameter names and comments Files changed: - Query_Cache.h - proxysql_admin.h - MySQL_Thread.h - proxysql_debug.h - prometheus_helpers.h (template changes) - debug.cpp (usage update)	2 months ago
Rene Cannao	4810f1f05b	lint: fix reserved identifier in include guards Replace __CLASS_* and ____CLASS_* patterns with PROXYSQL_*_H to comply with the C++ standard where identifiers starting with __ are reserved in the global namespace. Also fixes: - ____CLASS_STANDARD_MYSQL_THREAD_H -> PROXYSQL_STANDARD_MYSQL_THREAD_H - ____CLASS_STANDARD_PGSQL_THREAD_H -> PROXYSQL_STANDARD_PGSQL_THREAD_H - __CLASS_PROXYSQL_POLL -> PROXYSQL_POLL_H (added _H for consistency)	2 months ago
Rene Cannao	17225cf2a8	lint: fix bugprone-macro-parentheses in 6 header files - MySQL_Thread.h: wrap negative hostgroup constants in parens - PgSQL_Thread.h: wrap negative hostgroup constants in parens - proxysql_admin.h: wrap expression multipliers in parens - proxysql_debug.h: wrap macro args in ASSERT_SQLITE* macros - proxysql_macros.h: wrap macro args in strdup_null, free_null, itostr, Copy4B - proxysql_structs.h: wrap macro args in IS_PGTRACKED_VAR_OPTION_* macros	2 months ago
copilot-swe-agent[bot]	4a93861c6c	feat: add mysql resolution_family setting Agent-Logs-Url: https://github.com/sysown/proxysql/sessions/7d0ebed6-886c-4e06-add5-d74b44eced4a Co-authored-by: renecannao <3645227+renecannao@users.noreply.github.com>	2 months ago
Rene Cannao	3458cbd28b	fix: address zstd review feedback	2 months ago
Rene Cannao	c2f678a9de	feat: implement release tiers and feature guards for v3.0, v3.1, and v4.0 This commit introduces a structured release tier system controlled by environment variables, allowing three distinct ProxySQL versions to be built from the same codebase while maintaining a clear upgrade path. Key Changes: - Tier 1 (v3.0.x Stable): Default build including bug fixes and core enhancements. - Tier 2 (v3.1.x Innovative): Enabled via PROXYSQL31=1. Adds FFTO and TSDB. - Tier 3 (v4.0.x AI/MCP): Enabled via PROXYSQLGENAI=1. Adds the AI stack. Implementation Details: - Makefile: Implemented dynamic versioning logic that increments the minor version for v3.1 and the major version for v4.0. Added tier documentation. - Feature Guards: Introduced PROXYSQLFFTO and PROXYSQLTSDB guards across ProxySQL_Admin, ProxySQL_Statistics, REST API, and HTTP server. - FFTO: Guarded Fast Forward Traffic Observer in MySQL and PgSQL sessions, including thread-local variables and configuration parameters. - TSDB: Guarded the entire Time Series Database subsystem, including SQLite table definitions, background loops, and REST API endpoints. - Build Fixes: * Fixed 'proxy_debug_func' linkage by adding extern "C" and moving the definition outside of DEBUG guards to ensure non-debug builds link correctly. * Corrected misplaced/nested DEBUG and PROXYSQLFFTO guards. * Verified build stability for all tiers (unset, PROXYSQL31=1, PROXYSQLGENAI=1).	3 months ago
René Cannaò	80f9c1a079	Merge branch 'v3.0' into session-track-system-variable Signed-off-by: René Cannaò <rene@proxysql.com>	3 months ago
Rene Cannao	acef17a8c1	Merge branch 'v3.0-ff_inspect' into v3.0-unified-large-pr # Conflicts: # test/tap/tests/Makefile	3 months ago
Rene Cannao	e2ebc0c449	Merge branch 'v4.0-mcp-stats2' into v3.0-unified-large-pr	3 months ago
Rene Cannao	1c25787b88	Introduce {mysql,pgsql}-query_processor_first_comment_parsing variable (#5384 ) This universal fix allows controlling when the first comment of a query is processed relative to the query rules. By setting this variable to 1, ProxySQL-specific annotations (like GTID) can be parsed immediately, allowing subsequent query rules to strip them from the query string. This prevents prepared statement cache bloat and improves backend statement reuse when annotations with unique literals are used.	3 months ago
Rene Cannao	1258a1f2f2	Address PR reviews: improve performance, safety, and robustness of FFTO	3 months ago
Rene Cannao	e762f91cda	MCP stats: add in-memory processlist filtering and PgSQL concurrency stress TAP Replace direct stats-schema reads in MCP show_processlist with in-memory processlist access via SQL3_Processlist query options. Add processlist query option types in proxysql_admin interfaces and implement filtering, sorting, and pagination post-processing in both MySQL and PgSQL thread implementations. Introduce configurable MCP cap mcp_stats_show_processlist_max_rows (default 200, max 1000), wire it through MCP variable loading, and expose it in default config. Expand Stats_Tool_Handler processlist handling to parse filter/sort arguments, enforce cap metadata, and return consistent MCP payload fields from in-memory snapshots. Add extensive doxygen documentation in the touched headers and source blocks to describe API contracts, filtering behavior, and runtime constraints. Add new TAP test mcp_pgsql_concurrency_stress-t that generates sustained concurrent PgSQL traffic (simple reads, read/write table workload, randomized pg_sleep) while polling MCP show_processlist and show_queries in parallel with filter and ordering assertions. Validation performed locally: mcp_pgsql_concurrency_stress-t (31 assertions) and mcp_show_queries_topk-t (12 assertions).	3 months ago
Rene Cannao	3a3be5a559	Implement Fast Forward Traffic Observer (FFTO) for MySQL and PostgreSQL - Added TrafficObserver base interface - Implemented MySQLFFTO and PgSQLFFTO classes for protocol-aware observation - Integrated FFTO hooks into MySQL and PostgreSQL session handlers in FAST_FORWARD state - Added configuration variables: mysql-ffto_enabled, mysql-ffto_max_buffer_size, pgsql-ffto_enabled, pgsql-ffto_max_buffer_size - Implemented query digest extraction and reporting parity with internal stats - Added session-level bypass when packet size exceeds buffer threshold - Included architectural documentation in doc/ffto_design.md	3 months ago
vramesha	15a4ed66e9	Optimize query logging performance (#5243 ) Detailed Changes: - Introduced LogBuffer and LogBufferThreadContext to implement per-thread buffering, reducing lock contention. - Replaced localtime() with localtime_r() for improved performance and thread safety during timestamp generation. - Implemented configurable sampling for event logs to reduce I/O overhead. - Added global configuration variables to control flush size, flush interval, and sampling rate for event and audit logs.	3 months ago
René Cannaò	75a62f334d	Merge branch 'v4.0' into v3.1-vec Signed-off-by: René Cannaò <rene@proxysql.com>	4 months ago
Rene Cannao	52a70b0b09	feat: Implement AI-based Anomaly Detection for ProxySQL Phase 3: Anomaly Detection Implementation This commit implements a comprehensive multi-stage anomaly detection system for real-time SQL query security analysis. Core Detection Methods: 1. SQL Injection Pattern Detection (lib/Anomaly_Detector.cpp) - Regex-based detection of 11 SQL injection patterns - Suspicious keyword detection (11 patterns) - Covers: tautologies, union-based, comment-based, stacked queries 2. Query Normalization (lib/Anomaly_Detector.cpp:normalize_query) - Converts to lowercase - Removes SQL comments - Replaces string/numeric literals with placeholders - Normalizes whitespace 3. Rate Limiting (lib/Anomaly_Detector.cpp:check_rate_limiting) - Per user/host query rate tracking - Configurable time windows (3600s default) - Auto-block on threshold exceeded - Prevents DoS and brute force attacks 4. Statistical Anomaly Detection (lib/Anomaly_Detector.cpp:check_statistical_anomaly) - Z-score based outlier detection - Abnormal execution time detection (>5s) - Large result set detection (>10000 rows) - Behavioral profiling per user 5. Embedding-based Similarity (lib/Anomaly_Detector.cpp:check_embedding_similarity) - Placeholder for vector similarity search - Framework for sqlite-vec integration - Detects novel attack variations Query Flow Integration: - Added `detect_ai_anomaly()` to MySQL_Session (line 3626) - Integrated after libinjection SQLi detection (line 5150) - Blocks queries when risk threshold exceeded (default: 0.70) - Sends error response with anomaly details Status Variables Added: - `ai_detected_anomalies`: Total anomalies detected - `ai_blocked_queries`: Total queries blocked - Available via: `SELECT * FROM stats_mysql_global` Configuration (defaults): - `enabled`: true - `risk_threshold`: 70 (0-100) - `similarity_threshold`: 85 (0-100) - `rate_limit`: 100 queries/hour - `auto_block`: true - `log_only`: false Detection Pipeline: ``` Query → SQLi Check → AI Anomaly Check → [Block if needed] → Execute (libinjection) (Multi-stage) ``` Files Modified: - include/MySQL_Session.h: Added detect_ai_anomaly() declaration - include/MySQL_Thread.h: Added AI status variables - lib/Anomaly_Detector.cpp: Full implementation (700+ lines) - lib/MySQL_Session.cpp: Integration and query flow - lib/MySQL_Thread.cpp: Status variable definitions Next Steps: - Add unit tests for each detection method - Add integration tests with sample attacks - Add user and developer documentation Related: Phase 1-2 (NL2SQL foundation and testing) Related: Phase 4 (Vector storage for embeddings)	4 months ago
René Cannaò	f82bcdff38	Merge branch 'v3.0' into session-track-system-variable Signed-off-by: René Cannaò <rene@proxysql.com>	4 months ago
Rene Cannao	366164ab26	Fix #5272 : Add mysql-select_version_forwarding variable for SELECT VERSION() Since ProxySQL 3.0.4, SELECT VERSION() queries were intercepted and returned ProxySQL's mysql-server_version variable instead of proxying to backends. This broke SQLAlchemy for MariaDB which expects "MariaDB" in the version string. This commit adds a new variable `mysql-select_version_forwarding` with 4 modes: - 0 = never: Always return ProxySQL's version (3.0.4+ behavior) - 1 = always: Always proxy to backend (3.0.3 behavior) - 2 = smart (fallback to 0): Try backend connection, else ProxySQL version - 3 = smart (fallback to 1): Try backend connection, else proxy (default) The implementation includes: - New global variable mysql_thread___select_version_forwarding - New function get_backend_version_for_hostgroup() to peek at backend connection versions without removing them from the pool - Modified SELECT VERSION() handler to support all 4 modes - ProxySQL backend detection to avoid recursion Mode 3 (default) ensures SQLAlchemy always gets the real MariaDB version string while maintaining fast response when connections are available.	5 months ago
Wazir Ahmed	f0ac86c317	Enable session tracking based on server capabilities Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	5 months ago
Rene Cannao	36442e4cd7	docs: Add comprehensive Doxygen documentation for session variable tracking Add detailed architectural documentation for PR 5166 session variable tracking: - High-level architecture overview in session_track_variables struct explaining the 3-phase workflow - Detailed documentation for handler_rc0_Process_Variables explaining the core processing workflow - Technical implementation details for MySQL_Connection::get_variables protocol interface - Configuration logic documentation for handler_again___verify_backend_session_track_variables - Added inline comments explaining why session tracking is needed and performance considerations This documentation provides a complete understanding of how MySQL session variable tracking integrates with ProxySQL's existing state machine and leverages MySQL's native session tracking capabilities.	5 months ago
Wazir Ahmed	4b9c083209	MySQL: Track session-specific system variables changes - Add new mysql variable `mysql-session_track_variables`. - Configure `session_track_system_variables` and `session_track_state_change` on backend connections if the mysql variable is enabled. - Utilize notifications from backend servers to capture system variable changes that cannot be handled by `MySQL_Set_Stmt_Parser` - Update both client and server variable maps based on backend responses. - TAP test to verify this patch. Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	5 months ago
Javier Jaramago Fernández	d0e88599ee	Add special handling for unexpected COM_PING packets Implements a workaround for the handling of unexpected 'COM_PING' packets received during query processing, while a resultset is yet being streamed to the client. Received 'COM_PING' packets are queued in the form of a counter. This counter is later used to sent the corresponding number of 'OK' packets to the client after 'MySQL_Session' has finished processing the current query.	5 months ago
Rene Cannao	5a7b22181f	Fix metrics collection for wait_timeout counters The get_status_variable() function was only scanning worker threads but ignoring auxiliary threads (idle threads) where timeout terminations are detected. This caused the timeout termination counter to show incorrect/zero values. - Added idle thread scanning to both overloaded versions of get_status_variable() function - Now properly collects metrics from both worker and idle threads - Fixes the issue where proxysql_mysql_timeout_terminated_connections_total showed zero despite actual timeout terminations Resolves the metrics reading issue identified in the previous commits.	6 months ago
Rene Cannao	dc4694d656	Refactor idle session scanning and improve test precision Code improvements: - Extract SESS_TO_SCAN_idle_thread constant to header file for better maintainability - Replace magic number 128 with named constant in idle_thread_to_kill_idle_sessions() - Improve code readability and consistency in session scanning logic Test enhancements: - Add mysql-poll_timeout configuration for more precise timeout testing - Reduce test sleep times to 13 seconds for faster test execution - Add diagnostic messages to clearly show timeout configurations in test output - Ensure tests properly validate timeout enforcement with precise timing The changes improve code maintainability and make tests more reliable and faster while maintaining accurate timeout validation.	6 months ago
Rene Cannao	44aa606caa	Implement fast forward grace close feature to prevent data loss Problem: In fast forward mode, ProxySQL forwards packets directly from client to backend without buffering them. If the backend connection closes unexpectedly (e.g., due to server crash, network failure, or other issues), ProxySQL immediately closes the client session. This can result in data loss because the client may have sent additional data that hasn't been fully transmitted yet, as ProxySQL does not wait for the output buffers to drain. Solution: Implement a configurable grace period for session closure in fast forward mode. When the backend closes unexpectedly, instead of closing the session immediately, ProxySQL waits for a configurable timeout (fast_forward_grace_close_ms, default 5000ms) to allow any pending client output data to be sent. During this grace period: - If the client output buffers become empty, the session closes gracefully. - If the timeout expires, the session closes anyway to prevent indefinite hanging. Changes: - Added global variable mysql_thread___fast_forward_grace_close_ms (0-3600000ms) - Added session flags: backend_closed_in_fast_forward, fast_forward_grace_start_time - Added data stream flag: defer_close_due_to_fast_forward - Modified MySQL_Data_Stream::read_from_net() to detect backend EOF and initiate grace close if client buffers are not empty - Modified MySQL_Session::handler() FAST_FORWARD case to implement grace close logic with timeout and buffer checks - Added extensive inline documentation explaining the feature and its mechanics This prevents data loss in fast forward scenarios while maintaining bounded session lifetime.	7 months ago
Lisandro Pin	6cb522283d	Implement a ignore_min_gtid_annotation global variable for ProxySQL When true, all `min_gtid` query annotations are ignored; see https://proxysql.com/documentation/query-annotations/ for details. This is useful on ProxySQL setups with multiple layers, where some layers mandate GTID-based routing while others don't.	7 months ago
Wazir Ahmed	5219d4be93	processlist: Prevent crash due to SQLITE_TOOBIG - Add new mysql/pgsql variable `processlist_max_query_length`. - Min: 1K - Max: 32M - Default: 2M - Truncate current query based on the configuration before inserting into `stats_*_processlist` tables. - Refactor/fix code related to other processlist configurations. 1. `session_idle_show_processlist` value was not updated in `ProxySQL_Admin.variables`. 2. Pass processlist config as an argument to `MySQL_Threads_Handler::SQL3_Processlist` instead of using thread-local variables. Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	7 months ago
René Cannaò	9737e0630a	Add variable mysql-eventslog_stmt_parameters #4923 Variable mysql-eventslog_stmt_parameters configures the logging of parameters in prepared statements: - 0 : logging disabled - 1 : logging enabled	1 year ago
René Cannaò	2551d350e9	Port to 3.0 of in-memory query logging See #4713 for further details	1 year ago
Rahim Kanji	f9232ae50d	Merge remote-tracking branch 'Master/v3.0' into v3.0_session_parameter_tracking	1 year ago
Rahim Kanji	35b2dd593d	Separate implementation of SET statement parser for MySQL and PostgreSQL	1 year ago
René Cannaò	7ef5993c35	Revert "Enforce 'CLIENT_DEPRECATE_EOF' capability match for conn-pool" This reverts commit `50ce205d21`.	1 year ago
Javier Jaramago Fernández	50ce205d21	Enforce 'CLIENT_DEPRECATE_EOF' capability match for conn-pool Add new config option ('mysql-connpool_match_client_deprecate_eof') that allows to control whether to enforce a match between fronted and backend connections for 'CLIENT_DEPRECATE_EOF' capability. By default, this option is 'true', allowing smooth transitions from regular sessions to 'fast_forward'.	1 year ago
Rahim Kanji	b4e9060f71	Modified Set Parser	1 year ago
Yashwant Sahu	a0db6c3a1f	1. Changed name to protocol_compression_level. 2. Added TAP test	2 years ago
Yashwant Sahu	a846447a22	Added MySQL compression level	2 years ago
René Cannaò	cbf4628ea3	Merge remote-tracking branch 'proxysql27/v2.7' into v2.x_postgres_merge27_20240917	2 years ago
Rene Cannao	d85430b709	Implementation of PROXY protocol V1 This commit introduces: - class ProxyProtocolInfo() , that: - performs parsing - validates subnet - run automated tests in DEBUG build - variable mysql-proxy_protocol_networks . Accepted values: - empty string: disables PROXY protocol - '*' : allows connections from any IP - comma separated list of subnets - automated testing in DEBUG build during start - export of PROXY protocol information in internal session, using PROXY_V1 - a TAP test to verify various connections	2 years ago
Miro Stauder	565848c4a0	use local includes - prometheus	2 years ago
Miro Stauder	8394fd444e	use local includes - prometheus	2 years ago
René Cannaò	2d7806c325	Moved run_SetAllSession_ToProcess0 to Base_Thread	2 years ago
René Cannaò	483b8335d3	More functions moved to Base_Thread * move_session_to_idle_mysql_sessions * find_session_idx_in_mysql_sessions * ProcessAllMyDS_BeforePoll	2 years ago
René Cannaò	a881936cad	More functions moved to Base_Thread * ProcessAllMyDS_AfterPoll * read_one_byte_from_pipe * tune_timeout_for_myds_needs_pause * tune_timeout_for_session_needs_pause * configure_pollout * set_backend_to_be_skipped_if_frontend_is_slow	2 years ago
René Cannaò	db6cb93fb9	Moved ProcessAllSessions_SortingSessions to Base_Thread	2 years ago
René Cannaò	8389f76413	More functions moved to Base_Thread() * check_timing_out_session() * check_for_invalid_fd()	2 years ago
René Cannaò	c9326564c0	Minor changes to try to optimize compiling	2 years ago

1 2 3 4 5 ...

358 Commits (ea3cc7a32c492a715c9850fa7e9671d1548f80ca)