proxysql

Commit Graph

Author	SHA1	Message	Date
copilot-swe-agent[bot]	e274862857	feat: introduce stats_global table and move TLS metrics out of stats_mysql_global; add TAP tests - Add STATS_SQLITE_TABLE_GLOBAL definition (stats_global table for non-MySQL/PgSQL metrics) - Register stats_global in Admin_Bootstrap.cpp - Add stats___global() implementation and declaration; document with Doxygen comment - Remove TLS_* variables from stats___mysql_global() - they were misplaced there - Move all TLS tracking metrics to stats___global() under ssl_mutex - Wire up stats_global query detection and refresh in GenericRefreshStatistics() - Add TAP test test_tls_stats-t.cpp: - Verifies stats_global contains all 6 TLS tracking variables - Checks value ranges and validity of each TLS variable - Verifies stats_tls_certificates has 2 rows (server + ca) with correct fields - Verifies TLS_Load_Count increments and TLS_Last_Load_Timestamp increases after PROXYSQL RELOAD TLS - Confirms TLS variables are absent from stats_mysql_global Co-authored-by: renecannao <3645227+renecannao@users.noreply.github.com>	3 weeks ago
copilot-swe-agent[bot]	4f88740b20	feat: add stats_tls_certificates table and TLS tracking metrics - Add STATS_SQLITE_TABLE_TLS_CERTIFICATES definition with cert_type, file_path, subject_cn, issuer_cn, serial_number, not_before, not_after, days_until_expiry, sha256_fingerprint, and loaded_at columns - Add TLS tracking fields to GloVars.global: tls_load_count, tls_last_load_timestamp, tls_last_load_ok, tls_cert_file, tls_ca_file, tls_key_file - Update ProxySQL_create_or_load_TLS() to populate tracking fields on success - Register stats_tls_certificates table in Admin_Bootstrap - Add stats___tls_certificates() function that reads certs from disk at query time, extracts metadata (CN, serial, validity dates, SHA-256 fingerprint, days_until_expiry) and populates the table - Add TLS variables to stats___mysql_global(): TLS_Load_Count, TLS_Last_Load_Timestamp, TLS_Last_Load_Result, TLS_Server_Cert_File, TLS_CA_Cert_File, TLS_Key_File - Wire up query detection and refresh in GenericRefreshStatistics() Co-authored-by: renecannao <3645227+renecannao@users.noreply.github.com>	3 weeks ago
Rahim Kanji	2046d92b70	Merge remote-tracking branch 'v3.0' into v3.0_pgsql_meta_command_describe_table	1 month ago
Rahim Kanji	ca45405942	Merge remote-tracking branch 'Master/v3.0-moveInfra' into v3.0_pgsql-pipeline-set-reset-discard-fix-5432	1 month ago
Rahim Kanji	8249fead6b	Fix options parameter parsing to reject unescaped spaces The parse_options() function now properly validates the options connection parameter to detect and reject unescaped spaces in values, matching PostgreSQL's behavior.	1 month ago
Rene Cannao	6b9918fdcc	chore: refine guard structures and fix indentation - Fixed indentation for TSDB methods in include/proxysql_admin.h. - Refined PROXYSQLTSDB guard structure in lib/Admin_FlushVariables.cpp. - Cleaned up indentation and redundant code in lib/ProxySQL_RESTAPI_Server.cpp. - Ensured consistent formatting across modified files.	1 month ago
Rene Cannao	bc524c7b27	chore: remove temporary snippet file	1 month ago
Rene Cannao	6b46b26931	chore: implement PR review feedback for release tier system - Makefile: normalized GIT_VERSION by stripping 'v' prefix before arithmetic. - Makefile: corrected .PHONY declaration for build_deps_debug_default. - Admin Handler: properly guarded TSDB commands to return error when disabled. - REST API: removed redundant nested guards inside tsdb_resource class. - Statistics: implemented rigorous input validation for set_variable. - Statistics: added return value checks and rollback logic for TSDB migrations. - Statistics: implemented bounded concurrency (batching) for backend probes. - FFTO: implemented cumulative buffer limit enforcement in MySQL and PgSQL. - FFTO: ensured initial packet capture in PgSQL_Session by early initialization. - FFTO: improved reassembly buffer compaction logic. - Headers: default-initialized ffto_bypassed to false. - Code cleanup: merged redundant PROXYSQLFFTO guards.	1 month ago
Rene Cannao	c2f678a9de	feat: implement release tiers and feature guards for v3.0, v3.1, and v4.0 This commit introduces a structured release tier system controlled by environment variables, allowing three distinct ProxySQL versions to be built from the same codebase while maintaining a clear upgrade path. Key Changes: - Tier 1 (v3.0.x Stable): Default build including bug fixes and core enhancements. - Tier 2 (v3.1.x Innovative): Enabled via PROXYSQL31=1. Adds FFTO and TSDB. - Tier 3 (v4.0.x AI/MCP): Enabled via PROXYSQLGENAI=1. Adds the AI stack. Implementation Details: - Makefile: Implemented dynamic versioning logic that increments the minor version for v3.1 and the major version for v4.0. Added tier documentation. - Feature Guards: Introduced PROXYSQLFFTO and PROXYSQLTSDB guards across ProxySQL_Admin, ProxySQL_Statistics, REST API, and HTTP server. - FFTO: Guarded Fast Forward Traffic Observer in MySQL and PgSQL sessions, including thread-local variables and configuration parameters. - TSDB: Guarded the entire Time Series Database subsystem, including SQLite table definitions, background loops, and REST API endpoints. - Build Fixes: * Fixed 'proxy_debug_func' linkage by adding extern "C" and moving the definition outside of DEBUG guards to ensure non-debug builds link correctly. * Corrected misplaced/nested DEBUG and PROXYSQLFFTO guards. * Verified build stability for all tiers (unset, PROXYSQL31=1, PROXYSQLGENAI=1).	1 month ago
Rahim Kanji	e84433159b	Merge remote-tracking branch 'v3.0' into v3.0_pgsql-pipeline-set-reset-discard-fix-5432	1 month ago
René Cannaò	16f8837e0d	Merge branch 'v3.0' into v3.0-pgsql_monitor_repl_lag Signed-off-by: René Cannaò <rene@proxysql.com>	1 month ago
Rahim Kanji	aca91c3bd2	Optimize PgSQL_Protocol::generate_ok_packet to use direct buffer writes Replace format-specifier based PG_pkt helper methods (write_CommandComplete, write_ParameterStatus, write_ReadyForQuery) with direct buffer writes using put_char(), put_uint32(), and put_string().	1 month ago
Rahim Kanji	742ff05ecc	Fix PostgreSQL transaction state management for pipeline mode Move transaction state tracking from RequestEnd to connection handler to enable proper binary pipeline mode support: - Call handle_transaction_state() at ASYNC_QUERY_END/ASYNC_STMT_EXECUTE_END in PgSQL_Connection instead of RequestEnd - Add reset_state() calls in 7 locations to prevent stale state: * Session reset * Connection detach (finishQuery) * Error handlers (4 paths) * Base session housekeeping - Refactor destructor to use reset_state() Fixes transaction state tracking in pipeline mode where connections can be returned to pool mid-transaction.	1 month ago
Wazir Ahmed	5421a382ab	fix: Load TSDB variables from disk during startup - TSDB variables stored on disk were not being loaded into runtime during startup. Add the missing `flush_tsdb_variables___database_to_runtime()` and invoke it during init. - Inline the existing load/save wrappers in the header as thin delegations to the underlying flush functions Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	1 month ago
Wazir Ahmed	1684d30725	fix: Refresh Prometheus metrics before TSDB sampling - `tsdb_sampler_loop()` calls `registry->Collect()` without refreshing module metrics, causing stale metric values to be stored in the TSDB. - Added `update_modules_metrics()` call before `Collect()` in the sampler. Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	1 month ago
Javier Jaramago Fernández	a4ea3b2174	Merge branch 'v3.0' of github.com:sysown/proxysql into v3.0-pgsql_monitor_repl_lag	1 month ago
Rahim Kanji	1ffb4f13ac	Fix PostgreSQL resync error detection Add flush() call in resync_start() for immediate socket error detection instead of deferring to event loop	1 month ago
Wazir Ahmed	daa0b2037e	fix: Use consistent value representation in MCP query rules `mcp_query_rules` and `runtime_mcp_query_rules` uses different representations for the same fields, causing the web UI to always report config differences even after loading to runtime. - Use `-1` sentinel for nullable int fields (`flagOUT`, `timeout_ms`, `log`) so `NULL` values from main are preserved when written to runtime, following the `MySQL QP_rule_t` pattern. - Serialize `re_modifiers` bitmask back to `VARCHAR` (`1→"CASELESS"`) to match the format used in main table - Update `evaluate_mcp_query_rules()` to handle tri-state log field Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	1 month ago
Rahim Kanji	176362c77d	Addressed feedback from AI review	2 months ago
Rahim Kanji	a82d653c0b	Handle late COPY protocol messages after session exits fast_forward mode When a COPY FROM STDIN operation encounters an error, the session switches back to normal mode. However, the client may have already pipelined CopyData('d'), CopyDone('c'), or CopyFail('f') messages that are still in the input queue. Previously, these messages fell through to the default case, generating a spurious "Feature not supported" error. This change adds explicit handling to discard these messages when session_fast_forward == SESSION_FORWARD_TYPE_NONE, preventing the race condition from causing errors. The client does not expect a response for these messages in this scenario.	2 months ago
Rene Cannao	70647fb426	fix: Remove unused legacy mcp-mysql_* variables These variables were deprecated in favor of profile-based routing but were never removed from the code. They existed in the struct definition, getters, and setters but were not consumed anywhere - Query_Tool_Handler uses target_auth_map via mcp_auth_profiles and mcp_target_profiles. Removed: - mcp-mysql_hosts, mcp-mysql_ports, mcp-mysql_user, mcp-mysql_password, mcp-mysql_schema from MCP_Thread.h struct - Variable names from mcp_thread_variables_names array - Initialization, cleanup, getters, and setters in MCP_Thread.cpp - Documentation in doc/MCP/VARIABLES.md	2 months ago
Rene Cannao	e53d2c76f9	Address PR #5410 review findings across FFTO, sessions, TAP, and docs Apply actionable bot review feedback from PR #5410 without behavior drift:\n- fix MySQL/PgSQL FFTO digest normalization and max-length hashing\n- add early FFTO packet/message size guards for bypass safety\n- restore MySQL session GTID reset and harden GenAI async error ownership\n- add null-check for stripped KILL query normalization path\n- call FFTO on_close() before observer reset/bypass in session flows\n- fix processlist query-options default pagination limit semantics\n- fix TAP noise_failures locking race and noise-aware plan count\n- extend wait_get_enpoint_ready with optional basic-auth passthrough\n- update MCP architecture docs from observe->stats naming\n- clean corrupted character in pgsql query-cache test comment\n\nValidation run:\n- git diff --check\n- make -C test/tap/tap tap.o utils_mariadb.o utils_mysql57.o utils_mysql8.o -j4\n- make -C lib obj/MySQLFFTO.oo obj/PgSQLFFTO.oo obj/MySQL_Session.oo obj/PgSQL_Session.oo obj/Query_Processor.oo -j4	2 months ago
Rene Cannao	79c0b383e8	integration: fix pre-CI issues in unified branch	2 months ago
Rene Cannao	acef17a8c1	Merge branch 'v3.0-ff_inspect' into v3.0-unified-large-pr # Conflicts: # test/tap/tests/Makefile	2 months ago
Rene Cannao	e2ebc0c449	Merge branch 'v4.0-mcp-stats2' into v3.0-unified-large-pr	2 months ago
Rene Cannao	90462f6d5a	ffto: harden MySQL/PgSQL observer state handling and align tests/docs Address outstanding review findings for FFTO on v3.0-ff_inspect and tighten protocol-state correctness for both engines. MySQL FFTO - Restrict on_close() reporting to true in-flight states and always clear query tracking state after close. - Add explicit active-query cleanup helpers and invoke them on state transitions to IDLE. - Preserve accounting on mid-resultset server ERR packets by reporting current query in READING_COLUMNS/READING_ROWS before reset. - Keep prepared-statement lifecycle cleanup robust (pending prepare cleared on prepare completion paths). MySQL session integration - Extract duplicated FAST_FORWARD client FFTO feed logic into observe_ffto_client_packet() and reuse it from all call sites. PostgreSQL FFTO - Replace regex-based CommandComplete parsing with lightweight token parsing, including NUL/whitespace trimming and strict numeric validation. - Add queued tracking for pipelined extended-protocol executes so query text and response attribution stay aligned under Parse/Bind/Execute pipelining. - Distinguish finalize semantics (execute-finalize on CommandComplete vs sync-finalize on ReadyForQuery) and centralize finalize/activation helpers. - Add frontend Close ('C') handling to evict statement/portal mappings. - Harden client/server message parsing with additional length checks. - Extend affected-row command tag coverage to COPY and MERGE. TAP tests - Stabilize test plans for failure paths by replacing early returns with a fail-and-skip-remaining flow and shared cleanup labels. - Ensure both MySQL and PgSQL FFTO tests preserve planned assertion counts under setup/prepare/execute failures. Documentation - Align FFT0 design doc state/response descriptions with current implementation (ReadyForQuery handling, pipelined queueing, supported PG command tags). - Fix wording/typo issues in protocol section. Validation performed - make -C lib -j4 - make -C test/tap/tests test_ffto_mysql-t test_ffto_pgsql-t -j4 Runtime execution of the two TAP binaries remains environment-dependent (admin endpoint connectivity required).	2 months ago
Rene Cannao	5519080ac2	mcp stats: address review findings and remove --genai CLI flag	2 months ago
Javier Jaramago Fernández	ba5af979ff	feat-wip: Add replication lag support for PostgreSQL monitoring	2 months ago
Rene Cannao	1c25787b88	Introduce {mysql,pgsql}-query_processor_first_comment_parsing variable (#5384 ) This universal fix allows controlling when the first comment of a query is processed relative to the query rules. By setting this variable to 1, ProxySQL-specific annotations (like GTID) can be parsed immediately, allowing subsequent query rules to strip them from the query string. This prevents prepared statement cache bloat and improves backend statement reuse when annotations with unique literals are used.	2 months ago
Rene Cannao	1258a1f2f2	Address PR reviews: improve performance, safety, and robustness of FFTO	2 months ago
Rene Cannao	5d786520b0	Merge remote-tracking branch 'origin/v3.0' into merge/v3.0-into-v3.0-5243 # Conflicts: # include/PgSQL_Logger.hpp # lib/Makefile # lib/PgSQL_Logger.cpp # lib/ProxySQL_Admin.cpp # test/tap/groups/groups.json	2 months ago
Rene Cannao	5d45d17456	mcp stats: split connection metrics from debug free-pool snapshots Refactor MCP stats connection tools so operational pool metrics remain lightweight while debug-level free-connection details are exposed through a dedicated gated tool. Changes in this commit: - Updated stats tool catalog and dispatch to add show_free_connections and keep show_connections focused on per-server pool metrics only. - Removed free-connection payload from show_connections and added explicit compatibility error when callers still pass detail=true, with guidance to use show_free_connections. - Implemented show_free_connections using in-memory hostgroup manager snapshots (MySQL and PgSQL) with hostgroup/server filtering and summary counters. - Added MCP runtime variable mcp-stats_enable_debug_tools (default false), including variable registration, getter/setter handling, and config default in proxysql.cfg. - Added extensive doxygen comments across modified code paths to document behavior, rationale, filters, and output contracts. - Added TAP coverage in mcp_show_connections_commands_inmemory-t for: show_commands baseline, show_connections aggregate-only contract, debug tool gating behavior, and enabled-path validation for show_free_connections on both MySQL and PgSQL. - Registered the new TAP test in test groups.	2 months ago
Rene Cannao	e762f91cda	MCP stats: add in-memory processlist filtering and PgSQL concurrency stress TAP Replace direct stats-schema reads in MCP show_processlist with in-memory processlist access via SQL3_Processlist query options. Add processlist query option types in proxysql_admin interfaces and implement filtering, sorting, and pagination post-processing in both MySQL and PgSQL thread implementations. Introduce configurable MCP cap mcp_stats_show_processlist_max_rows (default 200, max 1000), wire it through MCP variable loading, and expose it in default config. Expand Stats_Tool_Handler processlist handling to parse filter/sort arguments, enforce cap metadata, and return consistent MCP payload fields from in-memory snapshots. Add extensive doxygen documentation in the touched headers and source blocks to describe API contracts, filtering behavior, and runtime constraints. Add new TAP test mcp_pgsql_concurrency_stress-t that generates sustained concurrent PgSQL traffic (simple reads, read/write table workload, randomized pg_sleep) while polling MCP show_processlist and show_queries in parallel with filter and ordering assertions. Validation performed locally: mcp_pgsql_concurrency_stress-t (31 assertions) and mcp_show_queries_topk-t (12 assertions).	2 months ago
René Cannaò	9c6945fae8	Merge branch 'v3.0' into v3.0-tsdb-feature Signed-off-by: René Cannaò <rene@proxysql.com>	2 months ago
Rene Cannao	9a630a949b	Add extended Doxygen documentation for FFTO classes	2 months ago
Rene Cannao	40aff577a9	Complete FFTO implementation with verified metrics and robust error handling - Finalized MySQL and PostgreSQL protocol state machines. - Implemented accurate affected_rows and rows_sent capture. - Added defensive null checks to prevent early-session crashes. - Enhanced TAP tests with result-set draining and success verification. - Verified 100% pass rate for MySQL CRUD, binary protocol, and memory bypass.	2 months ago
Rene Cannao	70b61a86e3	MCP stats: implement in-memory show_queries Top-K path Replace stats table reads in show_queries with Query Processor in-memory aggregation to avoid stale/empty admin-db snapshots.\n\nIntroduce Top-K digest filtering primitives and wire them into Stats_Tool_Handler so filtering/sorting/limit are executed against live digest memory. Add runtime MCP variable mcp-stats_show_queries_max_rows with validation, defaulting, and configuration plumbing, plus cap metadata in the tool response.\n\nEnforce configurable hard cap semantics while preserving deterministic ordering and pagination behavior for MCP clients. Add internal DEBUG validator path (PROXYSQLTEST 56) in admin tests to stress generated digest data and validate the Top-K/filter pipeline against in-memory state.\n\nKeep the validator callable from startup/testing workflows without depending on MCP linkage. Add TAP coverage in test/tap/tests/mcp_show_queries_topk-t.cpp for:\n- MCP endpoint setup and reachability\n- digest generation + optional DEBUG internal validator\n- payload validation and cap metadata assertions\n- descending count order validation\n- match_digest_text filter verification\n\nMake payload parsing backward-compatible in TAP (direct tool result object and legacy wrapped success/result payloads).	2 months ago
Rene Cannao	c126e63a23	mcp stats: refresh stats tables under admin mutex before direct reads Problem MCP stats tools were querying stats.* tables directly through admindb. Unlike admin-session SQL, that path bypassed GenericRefreshStatistics() and could return stale or empty data for runtime-populated tables. What changed - Updated Stats_Tool_Handler::execute_admin_query() to mirror admin-session semantics for stats reads: - acquire GloAdmin->sql_query_global_mutex - optionally invoke ProxySQL_Admin::GenericRefreshStatistics(sql, ..., false) - execute admindb statement - release sql_query_global_mutex - Added strict input validation and explicit lock/unlock error reporting. - Added refresh_before_query parameter (default true) to avoid duplicate refresh passes for secondary count queries. - Switched secondary COUNT(*) calls in show_processlist/show_queries/ show_errors/show_query_rules to refresh_before_query=false. Documentation - Expanded Doxygen in include/Stats_Tool_Handler.h for execute_admin_query() to document locking, refresh behavior, and performance tradeoff. - Added detailed Doxygen above the execute_admin_query() implementation in lib/Stats_Tool_Handler.cpp. Testing - Added TAP integration test test/tap/tests/mcp_stats_refresh-t.cpp. - The test injects a synthetic stale marker row into stats.stats_mysql_global, calls /mcp/stats show_status, and verifies the marker disappears after refresh-before-read. - Added helper-level Doxygen in the test for setup/parsing behavior. Build verification - Compiled lib object: make -C lib obj/Stats_Tool_Handler.oo - Compiled TAP target: make -C test/tap/tests mcp_stats_refresh-t - Direct runtime execution of the TAP test in this workspace could not fully run because no local admin listener was available (connection to 127.0.0.1:6032 failed).	2 months ago
Rene Cannao	f3d4153e7d	Final FFTO implementation and verification fixes - Resolved signature mismatch in report_query_stats - Fixed missing type declarations for Query_Processor in FFTO classes - Verified Metric Parity for affected_rows and rows_sent - Confirmed full coverage in TAP tests for text/binary protocols and bypass logic - Verified clean build with make debug	2 months ago
Rene Cannao	33fce1ca35	MCP stats: address review findings and add detailed docs	2 months ago
René Cannaò	4a6809b60f	Merge branch 'v3.0' into v3.0-tsdb-feature Signed-off-by: René Cannaò <rene@proxysql.com>	2 months ago
Rene Cannao	17092487cf	Add protocol-labeled logger counters and FLUSH LOGS race TAP - Introduce proxysql_query_logger_logged_queries_total with protocol label (mysql, pgsql) for direct query-log accounting via Prometheus. - Wire MySQL and PgSQL loggers to increment/export the counter and hook PgSQL logger metrics into admin scrape updates. - Add reg_test_5389-flush_logs_no_drop-t.cpp to stress concurrent client traffic + PROXYSQL FLUSH LOGS and assert no dropped logs via metric delta == executed queries. - Document the new TAP with doxygen comments for env knobs and flow.	2 months ago
Rene Cannao	0aa3c5e153	fix(pgsql-eventslog): harden event ownership and keep newest rows in memory table Problem 1: - `PgSQL_Event::set_error()` stored borrowed pointers from connection-owned strings. - This could create lifetime hazards before/around event serialization and buffering. Fix: - Add explicit ownership tracking for error fields via `free_error_on_delete`. - Make `set_error()` duplicate incoming SQLSTATE/error strings. - Free owned error strings in destructor when the event does not own all fields. - Keep copy-constructor behavior for deep-copied buffered events and disable copy-assignment to avoid accidental shallow assignment. Problem 2: - `processEvents()` could repopulate `stats_pgsql_query_events` using the beginning of the drained batch after retention pruning. - This keeps older rows instead of the most recent rows when truncation is required. Fix: - Insert the tail of the drained batch (`events.end() - numEventsToInsert`) so memory table retention always preserves latest events. - Tighten row retention arithmetic using `size_t` to avoid signed/unsigned mismatch in keep/delete calculations.	2 months ago
Rene Cannao	97e5cfee90	docs(doxygen): expand PostgreSQL logger API documentation for new advanced logging components Context: - The PostgreSQL advanced query logging implementation added new event, buffer, and metrics APIs. - Several newly introduced functions lacked Doxygen coverage. Changes: - Add Doxygen documentation blocks for newly introduced logging types and APIs in `include/PgSQL_Logger.hpp`: - metric enums and metric-map indexes - `PGSQL_LOG_EVENT_TYPE` - `PgSQL_Event` class and its new/updated lifecycle + serialization methods - `PgSQL_Logger_CircularBuffer` constructor/destructor and buffer operations - `PgSQL_Logger` constructor/destructor Why: - Improve maintainability and code navigation for the new PostgreSQL advanced logging feature set. - Align implementation with project expectation that introduced features include inline Doxygen documentation.	2 months ago
Rene Cannao	acbd7d0df3	Implement PostgreSQL eventslog circular buffer and sink pipeline	2 months ago
Rene Cannao	2a46c239be	Add PostgreSQL advanced eventslog schema and variable scaffolding	2 months ago
Rene Cannao	3ca390dacb	Fix binary protocol support for prepared statements in FFTO - Implemented statement ID tracking for MySQL COM_STMT_PREPARE/EXECUTE - Implemented Parse/Bind/Execute portal mapping for PostgreSQL Extended Query - Corrected documentation regarding binary protocol support - Ensured metric parity for prepared statements in Fast Forward mode	2 months ago
Rene Cannao	3a3be5a559	Implement Fast Forward Traffic Observer (FFTO) for MySQL and PostgreSQL - Added TrafficObserver base interface - Implemented MySQLFFTO and PgSQLFFTO classes for protocol-aware observation - Integrated FFTO hooks into MySQL and PostgreSQL session handlers in FAST_FORWARD state - Added configuration variables: mysql-ffto_enabled, mysql-ffto_max_buffer_size, pgsql-ffto_enabled, pgsql-ffto_max_buffer_size - Implemented query digest extraction and reporting parity with internal stats - Added session-level bypass when packet size exceeds buffer threshold - Included architectural documentation in doc/ffto_design.md	2 months ago
Rene Cannao	eaadabbb8e	logger: fix thread-context map race in buffered logging	2 months ago
Rene Cannao	caf324f911	logger: harden buffered logging flush/rotation semantics This commit addresses three correctness regressions introduced by thread-local log buffering in PR #5364 (query logging performance): 1) stale logfile pointer/UAF race during concurrent rotate/close 2) stale logfile-open state after close 3) non-global flush behavior in admin/format-switch paths What was fixed - Make `flush_and_rotate()` consume the current logfile pointer under lock - Signature changed from `std::fstream` to `std::fstream&` - Prevents dereferencing a stale stream pointer captured before lock acquisition while another thread rotates/closes the file - Updated declaration/definition and all call sites - Add explicit synchronization for cross-thread buffer draining - Added `LogBufferThreadContext::buffer_lock` - Any path that appends or flushes a thread buffer now locks this mutex - Guarantees force-flush from admin/config paths cannot race with worker-thread appends on the same context - Restore global forced flush semantics where required - Extended `MySQL_Logger::flush` and `PgSQL_Logger::flush` to `flush(bool force = false)` - `force=false`: preserves existing low-overhead worker-loop behavior (per-thread timeout-based flush) - `force=true`: snapshots all known thread contexts and drains both events/audit buffers regardless of timeout - `flush_log()` now calls `flush(true)` before file rotation, so admin flush and format-switch operations no longer miss pending thread buffers - Avoid unintended rotation during forced draining - In `force=true` path, flush uses `rotate_fn=nullptr` - Drains buffered payload into the current file first - `flush_log()` then performs one controlled rotate/open step - Fix logfile state tracking after close - `events_close_log_unlocked()` and `audit_close_log_unlocked()` now set `logfile_open=false` when the stream is closed - Prevents write paths from treating a closed stream as open - Remove per-thread-context dependency during metadata header write - `events_open_log_unlocked()` now uses a local `LogBuffer` for metadata emission in format=1 instead of reusing a thread context buffer - Keeps open/rotate path independent from worker context lifecycle - Keep callers consistent and non-duplicative - Since `flush_log()` now force-drains internally, removed redundant explicit `flush()` calls from: - MySQL/PgSQL `eventslog_format` switch handlers - `ProxySQL_Admin::flush_logs` Behavioral outcome - No stale stream pointer use when close/rotate interleaves with flush - No false-positive logfile-open state after close - `FLUSH LOGS` and eventslog format switch now drain all known thread buffers before rotating, preventing dropped/misplaced buffered records Validation - Built modified objects directly - Ran full debug build with GENAI enabled: make clean && export PROXYSQLGENAI=1 && make debug -j24 Build completed successfully.	2 months ago

1 2 3 4 5 ...

2129 Commits (9850d9daebdb2cf2d06c4df0bb6d0eb271ccf3dd)