proxysql

Commit Graph

Author	SHA1	Message	Date
Rene Cannao	c2f678a9de	feat: implement release tiers and feature guards for v3.0, v3.1, and v4.0 This commit introduces a structured release tier system controlled by environment variables, allowing three distinct ProxySQL versions to be built from the same codebase while maintaining a clear upgrade path. Key Changes: - Tier 1 (v3.0.x Stable): Default build including bug fixes and core enhancements. - Tier 2 (v3.1.x Innovative): Enabled via PROXYSQL31=1. Adds FFTO and TSDB. - Tier 3 (v4.0.x AI/MCP): Enabled via PROXYSQLGENAI=1. Adds the AI stack. Implementation Details: - Makefile: Implemented dynamic versioning logic that increments the minor version for v3.1 and the major version for v4.0. Added tier documentation. - Feature Guards: Introduced PROXYSQLFFTO and PROXYSQLTSDB guards across ProxySQL_Admin, ProxySQL_Statistics, REST API, and HTTP server. - FFTO: Guarded Fast Forward Traffic Observer in MySQL and PgSQL sessions, including thread-local variables and configuration parameters. - TSDB: Guarded the entire Time Series Database subsystem, including SQLite table definitions, background loops, and REST API endpoints. - Build Fixes: * Fixed 'proxy_debug_func' linkage by adding extern "C" and moving the definition outside of DEBUG guards to ensure non-debug builds link correctly. * Corrected misplaced/nested DEBUG and PROXYSQLFFTO guards. * Verified build stability for all tiers (unset, PROXYSQL31=1, PROXYSQLGENAI=1).	2 months ago
René Cannaò	16f8837e0d	Merge branch 'v3.0' into v3.0-pgsql_monitor_repl_lag Signed-off-by: René Cannaò <rene@proxysql.com>	2 months ago
Javier Jaramago Fernández	a4ea3b2174	Merge branch 'v3.0' of github.com:sysown/proxysql into v3.0-pgsql_monitor_repl_lag	3 months ago
Rene Cannao	acef17a8c1	Merge branch 'v3.0-ff_inspect' into v3.0-unified-large-pr # Conflicts: # test/tap/tests/Makefile	3 months ago
Javier Jaramago Fernández	ba5af979ff	feat-wip: Add replication lag support for PostgreSQL monitoring	3 months ago
Rene Cannao	1c25787b88	Introduce {mysql,pgsql}-query_processor_first_comment_parsing variable (#5384 ) This universal fix allows controlling when the first comment of a query is processed relative to the query rules. By setting this variable to 1, ProxySQL-specific annotations (like GTID) can be parsed immediately, allowing subsequent query rules to strip them from the query string. This prevents prepared statement cache bloat and improves backend statement reuse when annotations with unique literals are used.	3 months ago
Rene Cannao	5d786520b0	Merge remote-tracking branch 'origin/v3.0' into merge/v3.0-into-v3.0-5243 # Conflicts: # include/PgSQL_Logger.hpp # lib/Makefile # lib/PgSQL_Logger.cpp # lib/ProxySQL_Admin.cpp # test/tap/groups/groups.json	3 months ago
Rene Cannao	2a46c239be	Add PostgreSQL advanced eventslog schema and variable scaffolding	3 months ago
Rene Cannao	3a3be5a559	Implement Fast Forward Traffic Observer (FFTO) for MySQL and PostgreSQL - Added TrafficObserver base interface - Implemented MySQLFFTO and PgSQLFFTO classes for protocol-aware observation - Integrated FFTO hooks into MySQL and PostgreSQL session handlers in FAST_FORWARD state - Added configuration variables: mysql-ffto_enabled, mysql-ffto_max_buffer_size, pgsql-ffto_enabled, pgsql-ffto_max_buffer_size - Implemented query digest extraction and reporting parity with internal stats - Added session-level bypass when packet size exceeds buffer threshold - Included architectural documentation in doc/ffto_design.md	3 months ago
vramesha	15a4ed66e9	Optimize query logging performance (#5243 ) Detailed Changes: - Introduced LogBuffer and LogBufferThreadContext to implement per-thread buffering, reducing lock contention. - Replaced localtime() with localtime_r() for improved performance and thread safety during timestamp generation. - Implemented configurable sampling for event logs to reduce I/O overhead. - Added global configuration variables to control flush size, flush interval, and sampling rate for event and audit logs.	3 months ago
René Cannaò	75a62f334d	Merge branch 'v4.0' into v3.1-vec Signed-off-by: René Cannaò <rene@proxysql.com>	4 months ago
Rene Cannao	d9346fe64d	feat: Add AI features manager foundation - Add AI_Features_Manager coordinator class - Add AI_Vector_Storage interface (stub) - Add Anomaly_Detector class (stub for Phase 3) - Update includes and main initialization	4 months ago
Rene Cannao	253591d262	Add experimental GenAI EMBED: query support for MySQL This commit adds experimental support for generating embeddings directly from MySQL queries using a special EMBED: syntax. Changes: - Add MYDS_INTERNAL_GENAI to MySQL_DS_type enum for GenAI connections - Add handler___status_WAITING_CLIENT_DATA___STATE_SLEEP___MYSQL_COM_QUERY___genai_embedding() - Implement EMBED: query detection and JSON parsing for document arrays - Build CSV resultset with embeddings (1 row per document, 1 column) - Add myconn NULL check in MySQL_Thread for INTERNAL_GENAI type - Add "debug_genai" name to debug module array - Remove HAVE_LIBCURL checks (libcurl is always statically linked) - Use static curl header: "curl/curl.h" instead of <curl/curl.h> - Remove curl_global_cleanup() from GenAI module (should only be in main()) Query format: EMBED: ["doc1", "doc2", ...] Result format: 1 row per document, 1 column with CSV embeddings Error handling uses MySQL ERR_Packet instead of resultsets.	4 months ago
Rene Cannao	960704066d	Implement real GenAI module with embedding and rerank support Header changes (include/GenAI_Thread.h): - Add GenAI_EmbeddingResult, GenAI_RerankResult, GenAI_RerankResultArray structs - Add GenAI_Document, GenAI_Request structures for internal queue - Add 5 configuration variables: genai_threads, genai_embedding_uri, genai_rerank_uri, genai_embedding_timeout_ms, genai_rerank_timeout_ms - Add status variables: threads_initialized, active_requests, completed_requests, failed_requests - Add public API methods: embed_documents(), rerank_documents() - Add client management: register_client(), unregister_client() - Add threading components: worker threads, listener thread, epoll Implementation changes (lib/GenAI_Thread.cpp): - Implement move constructors/destructors for result structures - Initialize default values for variables (threads=4, embedding port 8013, rerank port 8012, timeout 30s) - Implement get_variable/set_variable with validation for all 5 variables - Implement call_llama_batch_embedding() using libcurl - Implement call_llama_rerank() using libcurl - Implement embed_documents() public API (single or batch) - Implement rerank_documents() public API with top_n parameter - Implement register_client() for socket pair integration - Implement listener_loop() and worker_loop() for async processing - Add proper error handling and status tracking Debug integration (include/proxysql_structs.h): - Add PROXY_DEBUG_GENAI to debug_module enum	4 months ago
Rene Cannao	366164ab26	Fix #5272 : Add mysql-select_version_forwarding variable for SELECT VERSION() Since ProxySQL 3.0.4, SELECT VERSION() queries were intercepted and returned ProxySQL's mysql-server_version variable instead of proxying to backends. This broke SQLAlchemy for MariaDB which expects "MariaDB" in the version string. This commit adds a new variable `mysql-select_version_forwarding` with 4 modes: - 0 = never: Always return ProxySQL's version (3.0.4+ behavior) - 1 = always: Always proxy to backend (3.0.3 behavior) - 2 = smart (fallback to 0): Try backend connection, else ProxySQL version - 3 = smart (fallback to 1): Try backend connection, else proxy (default) The implementation includes: - New global variable mysql_thread___select_version_forwarding - New function get_backend_version_for_hostgroup() to peek at backend connection versions without removing them from the pool - Modified SELECT VERSION() handler to support all 4 modes - ProxySQL backend detection to avoid recursion Mode 3 (default) ensures SQLAlchemy always gets the real MariaDB version string while maintaining fast response when connections are available.	4 months ago
Wazir Ahmed	0e537ddb3b	Fix typo in 'EXCLUDE_TRACKING_VARIABLES' macro Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	5 months ago
René Cannaò	3c4e09fec0	Merge pull request #5225 from sysown/v3.0_refactor_prepared_statement_cache_design_5211 Refactored Prepared-Statement Cache Design (Lock-Free Hot Path) - Part 2	5 months ago
Rahim Kanji	1a48aadf5e	Code cleanup	6 months ago
Rene Cannao	44aa606caa	Implement fast forward grace close feature to prevent data loss Problem: In fast forward mode, ProxySQL forwards packets directly from client to backend without buffering them. If the backend connection closes unexpectedly (e.g., due to server crash, network failure, or other issues), ProxySQL immediately closes the client session. This can result in data loss because the client may have sent additional data that hasn't been fully transmitted yet, as ProxySQL does not wait for the output buffers to drain. Solution: Implement a configurable grace period for session closure in fast forward mode. When the backend closes unexpectedly, instead of closing the session immediately, ProxySQL waits for a configurable timeout (fast_forward_grace_close_ms, default 5000ms) to allow any pending client output data to be sent. During this grace period: - If the client output buffers become empty, the session closes gracefully. - If the timeout expires, the session closes anyway to prevent indefinite hanging. Changes: - Added global variable mysql_thread___fast_forward_grace_close_ms (0-3600000ms) - Added session flags: backend_closed_in_fast_forward, fast_forward_grace_start_time - Added data stream flag: defer_close_due_to_fast_forward - Modified MySQL_Data_Stream::read_from_net() to detect backend EOF and initiate grace close if client buffers are not empty - Modified MySQL_Session::handler() FAST_FORWARD case to implement grace close logic with timeout and buffer checks - Added extensive inline documentation explaining the feature and its mechanics This prevents data loss in fast forward scenarios while maintaining bounded session lifetime.	6 months ago
Lisandro Pin	6cb522283d	Implement a ignore_min_gtid_annotation global variable for ProxySQL When true, all `min_gtid` query annotations are ignored; see https://proxysql.com/documentation/query-annotations/ for details. This is useful on ProxySQL setups with multiple layers, where some layers mandate GTID-based routing while others don't.	7 months ago
Wazir Ahmed	5219d4be93	processlist: Prevent crash due to SQLITE_TOOBIG - Add new mysql/pgsql variable `processlist_max_query_length`. - Min: 1K - Max: 32M - Default: 2M - Truncate current query based on the configuration before inserting into `stats_*_processlist` tables. - Refactor/fix code related to other processlist configurations. 1. `session_idle_show_processlist` value was not updated in `ProxySQL_Admin.variables`. 2. Pass processlist config as an argument to `MySQL_Threads_Handler::SQL3_Processlist` instead of using thread-local variables. Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	7 months ago
René Cannaò	01862a247a	Merge pull request #5149 from sysown/v3.0_track_search_path_param_5148 Add search_path parameter to tracked ProxySQL variables list - PgSQL	7 months ago
René Cannaò	413b0121a1	Merge pull request #5146 from sysown/v3.0_parameter_status_case_sensitive_issue_5145 Fix incorrect ParameterStatus key casing for DateStyle, TimeZone, and IntervalStyle	7 months ago
Rahim Kanji	b005c2dd8c	Add tracking for PostgreSQL search_path parameter	7 months ago
Rahim Kanji	03b63f6857	Fix ParameterStatus capitalization for DateStyle, TimeZone, and IntervalStyle These parameters use capitalized names in PostgreSQL for historical reasons. ProxySQL now sends them using canonical capitalization to ensure client compatibility.	7 months ago
Rahim Kanji	4d91452fc5	Implement backend query cancellation for PostgreSQL #5136 Add support for PostgreSQL query cancellation and backend termination features to allow clients to cancel long-running queries and terminate connections through the standard PostgreSQL protocol. Features implemented: - Intercept pg_backend_pid() queries and return ProxySQL session thread ID - Intercept pg_terminate_backend() to terminate client connections asynchronously - Intercept pg_cancel_backend() to cancel queries on backend connections - Support Cancel Request protocol via separate connection with PID and secret key validation - Return BackendKeyData message on successful authentication with session thread ID and unique cancel secret key This enables clients to use standard PostgreSQL cancellation mechanisms (pg_cancel_backend, pg_terminate_backend, and Cancel Request protocol) while ProxySQL maintains proper session isolation and maps client requests to appropriate backend connections.	8 months ago
Rahim Kanji	23a764eaac	Preserve implicit transactions with pipeline + FLUSH #5118 Previously, each extended-query block was terminated with a SYNC, which caused implicit transactions to commit prematurely. As a result, earlier write operations (INSERT/UPDATE/DELETE) could not be rolled back if a later statement in the same sequence failed. This change switches to libpq pipeline mode and replaces intermediate SYNC messages with FLUSH, ensuring that all client query frames execute as part of the same implicit transaction. A final SYNC is still issued to resynchronize the connection and make it safe for reuse in the pool.	8 months ago
Rahim Kanji	031d362aab	Add STMT prefix to ASYC DESCRIBE status	8 months ago
Rahim Kanji	7d5fe54230	Refactor: unify result handling for Parse and Describe Previously, Parse and Describe each had their own query result handling paths. This duplicated a lot of logic and also failed to handle some cases correctly—for example, Notice messages returned by the server during extended-protocol queries. Keeping these separate paths would be hard to maintain and prone to bugs. The simple-query result handling is already mature, optimized, and covers all the necessary cases. Reusing it for Parse and Describe makes behavior consistent across simple and extended query flows, while also reducing duplicate code.	8 months ago
Rahim Kanji	4d195adc89	Renamed Multiplex status from MYSQL to PGSQL Renamed status to ssl_status Removed STATE_SLEEP_MULTI_PACKET from PgSQL session handler	9 months ago
Rahim Kanji	ce94a7080a	Merge branch v3.0_session_param_reset_connection into v3.0_extended_query_protocol	10 months ago
Rahim Kanji	50243ef283	Corrected RESET/DEFAULT logic * Now will actually reset value to server default (represented by nullptr value), if that parameter is not set in startup parameter. * Default parameter values (pgsql_default_*) will now be set only for critical parameters.	10 months ago
Rahim Kanji	3619fa5d21	Added pgsql-max_stmts_per_connection variable	10 months ago
Rahim Kanji	48ba491cb7	Add support for unnamed Describe Portal and improve extended query handling - Always send Describe Portal to the backend - Enforce check for unnamed portals in Bind, Describe, Close, and Execute; raise errors for named portals - Improve error handling for extended query flow - Introduce Extended_Query_Info struct to unify extended query-related members and reduce redundancy	10 months ago
Rahim Kanji	7cde1dcad6	Added comments	10 months ago
Rahim Kanji	d5fc1fefff	Renamed pending_query to extended_query_frame Improved extended query frame handling	11 months ago
Rahim Kanji	a741598a8d	Added Describe message handling	11 months ago
René Cannaò	2c26a42897	Merge pull request #4819 from sysown/v3.0-ff_conns_eof_caps_match Enforce `CLIENT_DEPRECATE_EOF` capability match for `fast-forward` sessions	1 year ago
René Cannaò	edc84ea14b	Merge pull request #4922 from sysown/v3.0-4861_fixes1 Further enhancement in prepared statement logging #4861	1 year ago
René Cannaò	216b31feae	Added new event_type PROXYSQL_METADATA When creating an event log in binary format, a metadata packet is written. The metadata is in JSON format. It currently only provide the ProxySQL version. For illustration purposes, tool eventlog_reader_to_json.cpp supports it too. This commit includes also some reformatting	1 year ago
Rahim Kanji	2f9bb83dda	Added Abort (alias of Rollback)	1 year ago
René Cannaò	9737e0630a	Add variable mysql-eventslog_stmt_parameters #4923 Variable mysql-eventslog_stmt_parameters configures the logging of parameters in prepared statements: - 0 : logging disabled - 1 : logging enabled	1 year ago
Javier Jaramago Fernández	811874bad1	Merge branch 'v3.0' of github.com:sysown/proxysql into v3.0-ff_conns_eof_caps_match	1 year ago
René Cannaò	dd58ea0b0a	Merge pull request #4908 from sysown/v3.0-4727 Fix MySQL 8.4 replication lag monitoring - Closes #4727	1 year ago
René Cannaò	6df3d6b84b	Merge pull request #4899 from sysown/v3.0_refactor_connection_info_param Refactor PostgreSQL Connection-Level Parameters Handling	1 year ago
Javier Jaramago Fernández	7955dd21a8	Fix MySQL 8.4 replication lag monitoring - Closes #4727 Added support for new syntax to simulator.	1 year ago
Rahim Kanji	c2f2ae5aa4	* Updated code to use pgsql_tracked_variables[idx] consistently, removing direct usage of pgsql_tracked_variables[idx].idx. This change improves consistency and reduces potential human error. * Fixed some warning	1 year ago
Rahim Kanji	db442ba8cf	Treat client_encoding as normal server parameter/variable. Optimized RESET ALL command handling. Few optimizations. Code cleanup	1 year ago
René Cannaò	d3f06c537f	Merge branch 'v3.0' into v3.0-ff_conns_eof_caps_match	1 year ago
René Cannaò	2551d350e9	Port to 3.0 of in-memory query logging See #4713 for further details	1 year ago

1 2 3 4 5 ...

413 Commits (v3.0_pgsql_sslkeylog_5281)