proxysql

Commit Graph

Author	SHA1	Message	Date
René Cannaò	fa35bda62b	Merge pull request #5069 from evkuzin/v3.0-fix-prometheus-metrics-dumplication add protocol labels for shared metrics between mysql and psql	5 months ago
Rene Cannao	73555410e9	refactor: migrate prepare_v2 SIMPLE case in FlushDigestTableToDisk template Migrated the template function FlushDigestTableToDisk to use RAII-based prepare_v2. This function is instantiated for both MySQL and PostgreSQL, eliminating 8 deprecation warnings (4 instantiations x 2 prepare_v2 calls). The RAII pattern ensures automatic cleanup and eliminates manual (*proxy_sqlite3_finalize) calls.	5 months ago
Rene Cannao	e3ee454369	fix: remove duplicate 'int rc' declarations in prepare_v2 migrated functions The RAII-based prepare_v2 migration introduced duplicate variable declarations that caused compilation errors. Removed duplicate 'int rc;' declarations in: - dump_checksums_values_table() - save_mysql_users_runtime_to_database() - save_pgsql_users_runtime_to_database() The 'int rc' variable was already declared at function scope, so the duplicate declarations after the structured bindings were redundant.	5 months ago
Rene Cannao	5b6381b329	refactor: migrate remaining prepare_v2 SIMPLE cases in ProxySQL_Admin Migrated: - save_pgsql_ldap_mapping_runtime_to_database() - save_mysql_servers_runtime_to_database() All statements are now automatically managed by RAII smart pointers.	5 months ago
Rene Cannao	2a83a30972	refactor: migrate more prepare_v2 SIMPLE cases in ProxySQL_Admin Migrated: - save_pgsql_users_runtime_to_database() - save_mysql_ldap_mapping_runtime_to_database() All statements are now automatically managed by RAII smart pointers.	5 months ago
Rene Cannao	ca9b72a9df	refactor: migrate more prepare_v2 SIMPLE cases in ProxySQL_Admin Migrated: - dump_checksums_values_table() - save_mysql_users_runtime_to_database() All statements are now automatically managed by RAII smart pointers.	5 months ago
Rene Cannao	e56da24c81	refactor: migrate more prepare_v2 SIMPLE cases in ProxySQL_Admin Migrated: - save_mysql_firewall_whitelist_rules_from_runtime() - save_pgsql_firewall_whitelist_rules_from_runtime() All statements are now automatically managed by RAII smart pointers.	5 months ago
Rene Cannao	575449cdf6	refactor: migrate all prepare_v2 SIMPLE cases to RAII API in ProxySQL_Admin Migrated the following functions: - save_mysql_query_rules_fast_routing_from_runtime() - save_pgsql_query_rules_fast_routing_from_runtime() - save_mysql_firewall_whitelist_sqli_fingerprints_from_runtime() - save_pgsql_firewall_whitelist_sqli_fingerprints_from_runtime() - save_mysql_firewall_whitelist_users_from_runtime() - save_pgsql_firewall_whitelist_users_from_runtime() All statements are now automatically managed by RAII smart pointers, eliminating manual finalize calls and potential memory leaks.	5 months ago
Rene Cannao	df1c4e7928	fix: medium priority warnings - write-strings and unused variables 1. Fix write-strings warnings (12 occurrences): - MySQL_HostGroups_Manager.cpp: Change char* to const char* for src_status and dst_status variables - MySQL_Monitor.cpp: Change ConsumerThread constructor parameter from char thread_name[16] to const char *thread_name 2. Fix unused-but-set-variable warnings (6 occurrences) by wrapping PROXYSQLGENAI-specific variables in #ifdef blocks: - stats_mcp_query_tools_counters - stats_mcp_query_tools_counters_reset - stats_mcp_query_digest - stats_mcp_query_digest_reset - stats_mcp_query_rules - runtime_mcp_query_rules These variables were assigned values but appeared unused when PROXYSQLGENAI was not defined. Now they are only defined and assigned when PROXYSQLGENAI is enabled. This eliminates 18 compiler warnings: 12 write-strings and 6 unused-but-set-variable warnings. Found during compiler warning analysis with 'make debug'.	5 months ago
Rene Cannao	9e83a81b4c	fix: critical use-after-free in child_telnet function Fix use-after-free vulnerability where the function frees the 'arg' pointer at line 2306 but then returns it at line 2323. Change the return value to NULL instead of the freed pointer. This fixes a -Wuse-after-free compiler warning and prevents potential memory corruption. Found during compiler warning analysis with 'make debug'.	5 months ago
René Cannaò	a360dc22ae	Merge pull request #5308 from sysown/v3.0_mac Support for ProxySQL compilation on macOS (Darwin/Apple Silicon)	5 months ago
René Cannaò	ce9fdc9d7a	Merge branch 'v3.0' into v3.0.6-implement_FLUSH_STATS Signed-off-by: René Cannaò <rene@proxysql.com>	5 months ago
Rene Cannao	48bc7dd7bf	Merge v4.0 GenAI features into v3.0 with conditional compilation This commit merges the experimental v4.0 GenAI/MCP features into the stable v3.0 branch using conditional compilation. All v4.0 features are disabled by default and only enabled when PROXYSQLGENAI=1 is set at compile time. Changes: Build System: - Modified main Makefile to pass PROXYSQLGENAI flag to sub-makefiles - Modified deps/Makefile to conditionally build sqlite-vec and sqlite-rembed - Modified lib/Makefile to add PSQLGA flag and include GenAI object files - Modified src/Makefile to add PSQLGA flag and conditional linking Headers (wrapped with #ifdef PROXYSQLGENAI): - All 20 new GenAI header files in include/ - Modified cpp.h, proxysql_glovars.hpp, proxysql_admin.h - Modified ProxySQL_Admin_Tables_Definitions.h for GenAI/MCP tables Source Files: - All 22 new GenAI source files in lib/ wrapped with #ifdef PROXYSQLGENAI - Modified src/main.cpp for conditional global variables and init/shutdown - Modified Admin_Handler.cpp for conditional command handlers - Modified Admin_Bootstrap.cpp for conditional table registration - Modified Admin_FlushVariables.cpp for conditional variable flushing - Modified ProxySQL_Admin.cpp for conditional admin methods - Modified ProxySQL_Admin_Stats.cpp for conditional MCP stats functions - Modified proxy_sqlite3_symbols.cpp to always compile (needed by core) - Modified MySQL_Session.cpp for conditional GenAI function calls Test Files: - Renamed test_mcp_query_rules-t to test_mcp_query_rules-t.sh - Renamed test_mcp_rag_metrics-t to test_mcp_rag_metrics-t.sh - Modified anomaly_detection-t.cpp for conditional test execution Usage: # Build without GenAI (v3.0 mode - default) make clean && make build_deps -j$(nproc) && make build_lib -j$(nproc) && make build_src -j$(nproc) # Build with GenAI (v4.0 mode) make clean && PROXYSQLGENAI=1 make build_deps -j$(nproc) && PROXYSQLGENAI=1 make build_lib -j$(nproc) && PROXYSQLGENAI=1 make build_src -j$(nproc)	5 months ago
Miro Stauder	cc1d93d08b	implement admin command 'PROXYSQL FLUSH STATS' for DEBUG builds	5 months ago
Wazir Ahmed	50de536534	MCP: Fix crash during server restarts Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	5 months ago
Rene Cannao	f2536f01d2	Merge v3.1-vec into v3.1-MCP2_QR Resolved merge conflict in lib/ProxySQL_Admin_Stats.cpp: - Kept v3.1-MCP2_QR version with proper NULL handling - Uses stmt_unique_ptr (RAII smart pointer) - Checks for NULL before calling atoll() on numeric fields - Discarded v3.1-vec version which lacked NULL checks The merged code correctly handles NULL values for: - run_id - count_star - first_seen - last_seen - sum_time - min_time - max_time	5 months ago
Rene Cannao	188aef90fe	fix: Use delete instead of free for SQLite3_result in load_mcp_query_rules_to_runtime Change free(resultset) to delete resultset in ProxySQL_Admin::load_mcp_query_rules_to_runtime. SQLite3_result is a C++ class allocated with new, so it must be deallocated with delete, not free(). Using free() causes undefined behavior and memory leaks. Addresses coderabbitai review comment.	5 months ago
Javier Jaramago Fernández	52142c4648	fix: Multiple issues with MCP query_(rules/digests) - Fixed invalid memory accesses for tables: + mcp_query_rules + stats_mcp_query_rules + stats_mcp_query_digests - Fixed inactive 'mcp_query_rules' being loaded to runtime. - Fixed hash computation in 'compute_mcp_digest'. - Fixed invalid escaping during 'stats_mcp_query_digests' gen. - Fixed digest generation for MCP arguments: + SQL queries are now preserved using 'mysql_query_digest_and_first_comment'. + TODO: Options for the tokenizer are right now hardcoded. - Added initial testing and testing plan for MCP query_(rules/digests). + TODO: Test finished on phase8. Timeouts destroy the MCP connection, leaving it unusable for subsequent queries this should be fixed for continuing testing. - TODO: There are several limitations to fix in 'validate_readonly_query'. This reflect in some query hacks in the testing. + 'SELECT' starting with comments (--) gets flagged as non-read. + 'SELECT' must have a 'SELECT .* FROM' structure. While common, simple testing queries many times lack this form.	5 months ago
Rene Cannao	709649232b	fix: Address AI code review concerns from PR #19 This commit addresses valid concerns raised by coding agents (Gemini, Copilot, CoderabbitAI): 1. Fix stats_mcp_query_digest naming conflict (ProxySQL_Admin.cpp): - Made reset and non-reset paths mutually exclusive using else block - Prevents both flags from being true, matching MySQL pattern - Ensures reset takes precedence over non-reset 2. Fix INSERT OR REPLACE sync issue (Admin_Handler.cpp): - Added DELETE before INSERT OR REPLACE in LOAD/SAVE MCP QUERY RULES - Prevents stale rules from persisting when syncing disk <-> memory - Ensures deleted source rows are also removed from target 3. Fix integer division truncation for timeout (Query_Tool_Handler.cpp): - Changed timeout_ms/1000 to (timeout_ms+999)/1000 for ceiling division - Ensures sub-second timeouts (e.g., 500ms) become at least 1 second - Prevents zero-second timeouts from causing unexpected behavior 4. Remove confusing comment (Discovery_Schema.cpp): - Simplified column count comment to be clear and accurate Note: The re_modifiers parsing code already correctly handles VARCHAR "CASELESS" to int conversion (lines 2414-2425), so that review comment was already addressed.	5 months ago
René Cannaò	d43ae6e121	Surgical fixes for macOS compatibility: headers, types, and Makefile linking	5 months ago
Rene Cannao	23aaf80cd1	fix: Address AI code review concerns for PR #19 Fixed 6 legitimate issues from AI code review: Critical fixes: - Implement stats___mcp_query_digest to populate table with actual data - Fix double trigger bug (stats_mcp_query_digest_reset substring matching) Important fixes: - Fix re_modifiers parsing (VARCHAR "CASELESS" -> int 1) - Fix TOCTOU race condition in reset path (use write lock from start) - Add column count validation before accessing row fields Documentation: - Add memory ownership documentation for evaluate_mcp_query_rules False positives ignored (Issue 2: Schema mismatch, Issue 6: SQL injection)	5 months ago
Rene Cannao	a1d9d2f1ba	docs: Add comprehensive documentation to MCP features Add detailed function-level documentation to all MCP query rules, query digest, static harvester, and catalog components. Static_Harvester.cpp: - Document all 18+ harvest functions (schemas, objects, columns, indexes, FKs, views) - Document lifecycle methods (init, close, connect, disconnect) - Document helper methods (is_time_type, is_id_like_name) - Document run management (start_run, finish_run, run_full_harvest) - Document statistics methods (get_harvest_stats) Query_Tool_Handler.cpp: - Document JSON helper functions (json_string, json_int, json_double) - Document digest tracking section with flow explanation MySQL_Catalog.cpp: - Document schema isolation architecture - Document CRUD operations (upsert, get, search, list, remove, merge) Discovery_Schema.cpp: - Document MCP query rules evaluation (evaluate_mcp_query_rules) - Document digest functions (compute_mcp_digest, fingerprint_mcp_args) - Document update/get functions for rules and digests ProxySQL_Admin_Stats.cpp: - Document stats collection functions ProxySQL_Admin.cpp: - Document load/save functions for query rules Admin_Handler.cpp: - Document MCP query rules command handlers include/ProxySQL_Admin_Tables_Definitions.h: - Add comments explaining table purposes	5 months ago
Rene Cannao	cc3cc25532	fix: Remove unused reset parameter from stats___mcp_query_rules() Change function signature from stats___mcp_query_rules(bool reset) to stats___mcp_query_rules() to match MySQL query rules pattern. The reset parameter was never used in the function body and MySQL's stats___mysql_query_rules() has no reset parameter.	5 months ago
Rene Cannao	f01fc79584	feat: Add runtime_mcp_query_rules table and fix stats_mcp_query_rules schema - Add ADMIN_SQLITE_TABLE_RUNTIME_MCP_QUERY_RULES schema (17 columns, same as mcp_query_rules) - Fix STATS_SQLITE_TABLE_MCP_QUERY_RULES to only have rule_id and hits columns - Add runtime_mcp_query_rules detection and refresh in ProxySQL_Admin - Implement save_mcp_query_rules_from_runtime(bool _runtime) for both config and runtime tables - Update get_mcp_query_rules() to return 17 columns (no hits) - get_stats_mcp_query_rules() returns 2 columns (rule_id, hits) Mirrors the MySQL query rules pattern: - mcp_query_rules: config table (17 cols) - runtime_mcp_query_rules: runtime state (17 cols) - stats_mcp_query_rules: hit counters (2 cols)	5 months ago
Rene Cannao	a816a756d4	feat: Add MCP query tool usage counters to stats schema Add stats_mcp_query_tools_counters and stats_mcp_query_tools_counters_reset tables to track MCP query tool usage statistics. - Added get_tool_usage_stats_resultset() method to Query_Tool_Handler - Defined table schemas in ProxySQL_Admin_Tables_Definitions.h - Registered tables in Admin_Bootstrap.cpp - Added pattern matching in ProxySQL_Admin.cpp - Added stats___mcp_query_tools_counters() in ProxySQL_Admin_Stats.cpp - Fixed friend declaration for track_tool_invocation() - Fixed Discovery_Schema.cpp log_llm_search() to use prepare_v2/finalize	5 months ago
Rene Cannao	1eb42c57d0	fix: Add GenAI variables to runtime_global_variables population Add flush_genai_variables___runtime_to_database() call to the central location where all modules populate runtime_global_variables table. This was missing, causing genai-* variables to not appear in runtime_global_variables.	5 months ago
Rene Cannao	3575f54969	fix: properly implement checksum_pgsql_variables gating behavior - Fixed PostgreSQL variable gating to use checksum_pgsql_variables instead of MySQL checksums: - cluster_pgsql_query_rules_diffs_before_sync now uses checksum_pgsql_variables - cluster_pgsql_servers_diffs_before_sync now uses checksum_pgsql_variables - cluster_pgsql_users_diffs_before_sync now uses checksum_pgsql_variables - cluster_pgsql_variables_diffs_before_sync already correctly used checksum_pgsql_variables - This ensures that when checksum_pgsql_variables is disabled, all PostgreSQL cluster variables are properly gated and set to 0 - Previously PostgreSQL variables were incorrectly using MySQL checksum flags for gating, which meant disabling checksum_pgsql_variables had no effect Resolves the major issue identified by AI reviews where PostgreSQL variables were not properly controlled by their checksum flag.	5 months ago
Rene Cannao	b177750242	fix: correct PostgreSQL cluster sync architecture by removing separate module infrastructure - Remove incorrect separate module declarations for pgsql_replication_hostgroups and pgsql_hostgroup_attributes from ProxySQL_Cluster.hpp - Remove separate module counter enums and atomic variables - Remove separate module metrics definitions and configurations - Remove incorrect admin variable implementations for non-distinct modules - Verify that PostgreSQL tables correctly sync as part of unified pgsql_servers module - Confirm build succeeds and no remaining references to removed infrastructure These tables are part of the pgsql_servers module and should not be treated as separate modules. The PgSQL_HostGroups_Manager::commit_update_checksums_from_tables() function already correctly includes both tables in the unified checksum computation. Resolves the architectural error where separate module infrastructure was added for tables that are not distinct modules.	5 months ago
Rene Cannao	f9e3e85e68	fix: address all AI review concerns for PostgreSQL cluster sync - Fix critical Admin_FlushVariables bug where PostgreSQL SELECT appended to MySQL checksum query - Add security check for integer overflow in pgsql_variable_validate_integer - Fix test suite tuple indexing and NULL safety issues - Add missing admin variables for PostgreSQL replication hostgroups and attributes - Implement checksum_pgsql_variables get/set variable behavior - Fix misleading log messages for PostgreSQL disable operations - Fix cluster_pgsql_variables_diffs_before_sync gating logic - Add missing sync_delayed metrics for pgsql modules - Fix CLUSTER_QUERY_RUNTIME_PGSQL_SERVERS macro numeric vs VARCHAR comparison Resolves issues identified by coderabbitai and gemini-code-assist reviews in PR 5297.	5 months ago
René Cannaò	5154f9e4c4	Merge branch 'v3.0' into fix/postgresql-cluster-sync_2 Signed-off-by: René Cannaò <rene@proxysql.com>	5 months ago
Rene Cannao	cf8692ace0	fix: address AI review feedback for PostgreSQL cluster sync - Fix CLUSTER_QUERY_RUNTIME_PGSQL_SERVERS macro to use proper VARCHAR string literals - Fix inconsistent gating/reset behavior for pgsql diffs when mysql checksum flags disabled - Fix copy/paste log text errors in ProxySQL_Admin.cpp - Improve test suite robustness with better NULL checks - Verify all PostgreSQL variables are properly included in admin_variables_names array Addresses feedback from coderabbitai and gemini-code-assist reviews in PR 5234.	5 months ago
Rene Cannao	c5916100f1	fix: complete all AI review fixes for PostgreSQL cluster synchronization This commit addresses the remaining issues identified by AI reviews in PR 5234: - Fixed CLUSTER_QUERY_RUNTIME_PGSQL_SERVERS macro to resolve numeric vs VARCHAR comparison issue - Verified that PostgreSQL variables are already included in admin_variables_names array - Fixed inconsistent gating/reset behavior for pgsql diffs when mysql checksum flags are disabled (added pgsql diffs reset for checksum_mysql_query_rules, checksum_mysql_servers, checksum_mysql_users) - Initialized atomic counters in ProxySQL_Cluster constructor to value 3 - Added missing get_peer_to_sync_pgsql_variables wrapper function (implementation + header declaration) - Fixed test issues in test_cluster_sync_pgsql-t.cpp (NULL checks, tuple indexing, test plan) - Updated PostgreSQL modules documentation to include pgsql_variables - Standardized SQL string quoting to single quotes consistently All compilation issues have been resolved and the codebase now has consistent PostgreSQL cluster synchronization behavior across all modules. Signed-off-by: Rene <rene@localhost>	5 months ago
Rene Cannao	119ca5003a	Fix compilation errors in debug build - Move explicit template instantiations for send_ok_msg_to_client and send_error_msg_to_client to after template definitions - Add missing closing brace for init_mcp_variables() - Fix missing #endif and closing brace for GloMCPH shutdown block	6 months ago
René Cannaò	313f637cf0	Merge branch 'v3.1-vec' into v3.1-MCP1 Signed-off-by: René Cannaò <rene.cannao@gmail.com>	6 months ago
Rene Cannao	d64b4c85d8	fix: resolve PostgreSQL cluster sync compilation issues - Add missing set_metrics() function to ProxySQL_Node_Entry class - Fix undefined 'd_query' variable in PostgreSQL variables function - Remove deprecated PostgreSQL functions integrated into pgsql_servers sync - Add missing admin variables for PostgreSQL modules - Fix structural compilation issues in set_checksums function - PostgreSQL cluster sync now builds successfully for both debug and regular builds Resolves linking errors and compilation failures that prevented debug and regular builds from completing successfully.	6 months ago
Rene Cannao	5a85ef04f6	Fix MCP variables persistence and add DISK command support This commit fixes several issues with MCP (Model Context Protocol) variables not being properly persisted across storage layers and adds support for DISK commands. Changes: 1. lib/Admin_FlushVariables.cpp: - Fixed flush_mcp_variables___runtime_to_database() to properly insert variables into runtime_global_variables using db->execute() with formatted strings (matching admin pattern) - Fixed SQL format string to avoid double-prefix bug (qualified_name already contains "mcp-" prefix) - Fixed lock ordering by releasing outer wrlock before calling runtime_to_database with use_lock=true, then re-acquiring - Removed explicit BEGIN/COMMIT transactions to match admin pattern 2. lib/Admin_Handler.cpp: - Added MCP DISK command handlers that rewrite commands to SQL queries: * LOAD MCP VARIABLES FROM DISK -> INSERT OR REPLACE INTO main.global_variables * SAVE MCP VARIABLES TO DISK -> INSERT OR REPLACE INTO disk.global_variables * SAVE MCP VARIABLES FROM MEMORY/MEM -> INSERT OR REPLACE INTO disk.global_variables - Separated DISK command handlers from MEMORY/RUNTIME handlers 3. lib/ProxySQL_Admin.cpp: - Added flush_mcp_variables___runtime_to_database() call to stats section to ensure MCP variables are repopulated when runtime_global_variables is cleared and refreshed 4. tests/mcp_module-t.cpp: - Added verbose diagnostic output throughout tests - Added section headers and test numbers for clarity - Added variable value logging and error logging All 52 MCP module tests now pass.	6 months ago
Rene Cannao	ab2c4f34b0	feat: add PostgreSQL variables to unified cluster sync - Add PostgreSQL type handling in pull_global_variables_from_peer() - Add PostgreSQL peer selection call using get_peer_to_sync_pgsql_variables() - Integrate PostgreSQL into the data-driven variable sync architecture This enables automatic peer discovery and selection for PostgreSQL variables synchronization across cluster nodes.	6 months ago
Rene Cannao	1beb5b9e5b	fix: address review comments for PostgreSQL cluster sync PR - Fix SQL string literals in CLUSTER_QUERY_PGSQL_SERVERS_V2 to use single quotes - Initialize all std::atomic<int> counters in ProxySQL_Cluster constructor to 3 - Add missing cluster_pgsql_variables_diffs_before_sync and cluster_pgsql_variables_save_to_disk admin variables - Add get_peer_to_sync_pgsql_variables wrapper function for consistency - Fix copy/paste log text error in set_variable function for mysql_query_rules - Make PgSQL diffs consistent with MySQL diffs when checksum flags are disabled - Remove TAP test calls from helper function, move assertions to main - Fix tuple size mismatch in string_format calls (remove extraneous 12th element) - Update test plan from 6 to 10 to match actual test count Resolves all issues identified by gemini-code-assist and coderabbitai review comments.	6 months ago
Rene Cannao	81c53896bc	Fix MCP module TAP test failures - Add MCP variables to load_save_disk_commands map for LOAD/SAVE commands - Add MCP variable validation in is_valid_global_variable() for SET commands - Implement has_variable() method in MCP_Threads_Handler - Add CHECKSUM command handlers for MCP VARIABLES (DISK/MEMORY/MEM) Test results improved from 28 passed / 16 failed to 49 passed / 3 failed. Remaining 3 failures are test expectation issues (boolean representation).	6 months ago
Rene Cannao	87fff9e046	Add MCP (Model Context Protocol) module skeleton Add new MCP module supporting multiple MCP server endpoints over HTTPS with JSON-RPC 2.0 protocol skeleton. Each endpoint (/mcp/config, /mcp/observe, /mcp/query, /mcp/admin, /mcp/cache) is a distinct MCP server with its own authentication configuration. Features: - HTTPS server using existing ProxySQL TLS certificates - JSON-RPC 2.0 skeleton implementation (actual protocol TBD) - 5 MCP endpoints with per-endpoint auth configuration - LOAD/SAVE MCP VARIABLES admin commands - Configuration file support (mcp_variables section) Implementation follows GenAI module pattern: - MCP_Threads_Handler: Main module handler with variable management - ProxySQL_MCP_Server: HTTPS server wrapper using libhttpserver - MCP_JSONRPC_Resource: Base endpoint class with JSON-RPC skeleton	6 months ago
Rene Cannao	c476f56f97	Add initial GenAI module placeholder Implement a new GenAI module for ProxySQL with basic infrastructure: - GenAI_Threads_Handler class for managing GenAI module configuration - Support for genai- prefixed variables in global_variables table - Dummy variables: genai-var1 (string) and genai-var2 (integer) - Config file support via genai_variables section - Flush functions for runtime_to_database and database_to_runtime - Module lifecycle: initialization at startup, graceful shutdown - LOAD/SAVE GENAI VARIABLES admin command infrastructure Core functionality verified: - Config file loading works - Variables persist in global_variables table - Disk save/load via SQL works - Module initializes and shuts down properly Related files: - include/GenAI_Thread.h: New GenAI thread handler class - lib/GenAI_Thread.cpp: Implementation with dummy variables - lib/Admin_Handler.cpp: Added GENAI command vectors and handlers - lib/Admin_FlushVariables.cpp: Added genai flush functions - lib/ProxySQL_Admin.cpp: Added init_genai_variables() and load_save_disk_commands entry - include/proxysql_admin.h: Added function declarations - lib/Makefile: Added GenAI_Thread.oo to build - src/main.cpp: Added module initialization and cleanup - src/proxysql.cfg: Added genai_variables configuration section	6 months ago
Rene Cannao	efe0d4fe61	Add extensive doxygen documentation for vacuum_stats and stats_pgsql_stat_activity This commit documents: 1. The vacuum_stats() function's purpose, behavior, and the reason why stats_pgsql_stat_activity is excluded from bulk deletion operations 2. The fact that stats_pgsql_stat_activity is a SQL VIEW (not a table) and attempting DELETE on it would cause SQLite error: "cannot modify stats_pgsql_stat_activity because it is a view" The documentation explains: - Why TRUNCATE stats_mysql_query_digest triggers vacuum_stats(true) - Why both MySQL and PostgreSQL tables are cleared regardless of protocol - How the view is automatically cleared via its underlying table stats_pgsql_processlist - The importance of keeping the view excluded from deletion lists	6 months ago
Rene Cannao	e2d64114d4	Refactor ProxySQL_Cluster: Eliminate code duplication and modernize atomic operations This commit implements two major architectural improvements to ProxySQL clustering: Major Changes: - Data-driven approach eliminates 95 lines of repetitive code in set_checksums() - Modern C++ atomics replace legacy GCC __sync_* built-ins - Improved maintainability and performance across cluster synchronization Code Duplication Elimination: - Replaced 142 lines of nearly identical if-statements with 47 lines of data-driven code - Added ChecksumModuleInfo structure with member pointers for unified processing - Generalized sync message generation using snprintf() templates - Single loop now handles all 15 cluster modules (MySQL + PostgreSQL) Atomic Operations Modernization: - Converted all cluster_*_diffs_before_sync variables from int to std::atomic<int> - Replaced __sync_fetch_and_add() with .load() for read operations (more efficient) - Replaced __sync_lock_test_and_set() with direct assignment for write operations - Updated member pointer types to handle atomic variables correctly - Ensures thread safety while maintaining identical functionality Files Modified: - include/ProxySQL_Cluster.hpp: Added <atomic> include and std::atomic<int> declarations - lib/ProxySQL_Cluster.cpp: Implemented data-driven set_checksums() and atomic operations - lib/ProxySQL_Admin.cpp: Updated all cluster variable writes to use atomic operations Technical Benefits: - 67% reduction in repetitive code for cluster checksum processing - Modern C++11 atomic operations with better performance characteristics - Type safety with proper atomic types instead of compiler built-ins - Consistent error handling and memory management patterns - Improved maintainability for adding new cluster modules Impact: - Maintains exact same functionality while dramatically improving code quality - Better performance for read operations (load vs __sync_fetch_and_add) - Foundation for future threading optimizations - Cleaner, more maintainable clustering codebase	7 months ago
Rene Cannao	e0dec1c54c	fix: Complete pgsql_servers_v2 checksum integration in runtime_checksums_values	7 months ago
Rene Cannao	b67c7c1d1c	feat: Implement PostgreSQL cluster synchronization for issue #5147 This commit implements comprehensive PostgreSQL cluster synchronization functionality to resolve issue #5147 where ProxySQL didn't sync PostgreSQL settings between cluster instances. Changes Made: ### 1. Added PostgreSQL Cluster Query Definitions (ProxySQL_Cluster.hpp) - CLUSTER_QUERY_RUNTIME_PGSQL_SERVERS - CLUSTER_QUERY_PGSQL_SERVERS_V2 - CLUSTER_QUERY_PGSQL_USERS - CLUSTER_QUERY_PGSQL_QUERY_RULES - CLUSTER_QUERY_PGSQL_QUERY_RULES_FAST_ROUTING ### 2. Extended Cluster Structure (ProxySQL_Cluster.hpp & proxysql_admin.h) - Added PostgreSQL checksum values to ProxySQL_Node_Entry - Added cluster admin variables: - cluster_pgsql_query_rules_diffs_before_sync - cluster_pgsql_servers_diffs_before_sync - cluster_pgsql_users_diffs_before_sync - cluster_pgsql__save_to_disk variables ### 3. Implemented PostgreSQL Peer Pull Functions (ProxySQL_Cluster.cpp) - pull_pgsql_query_rules_from_peer() - pull_runtime_pgsql_servers_from_peer() - pull_pgsql_servers_v2_from_peer() - pull_pgsql_users_from_peer() - get_peer_to_sync_pgsql_() functions ### 4. Added Admin Variable Support (ProxySQL_Admin.cpp) - Added PostgreSQL cluster variables to admin interface - Implemented GET and SET handlers for all new variables - Integrated with existing variable initialization system ### 5. Integrated PostgreSQL Sync into Cluster Monitoring - Added PostgreSQL diff variables to monitoring logic - Implemented complete sync detection and conflict resolution - Added debug and error logging for PostgreSQL sync operations Key Features: - Full compatibility with existing MySQL cluster sync patterns - Configurable sync thresholds and persistence options - Checksum-based change detection and conflict resolution - Comprehensive logging and debug support - Reuses existing MySQL sync infrastructure for consistency This implementation enables ProxySQL clusters to automatically synchronize PostgreSQL servers, users, and query rules between all cluster members, providing the same level of high availability and consistency for PostgreSQL as was already available for MySQL. Resolves: #5147	7 months ago
Rahim Kanji	38befb845f	Fixed PROXYSQL STOP	8 months ago
Wazir Ahmed	5219d4be93	processlist: Prevent crash due to SQLITE_TOOBIG - Add new mysql/pgsql variable `processlist_max_query_length`. - Min: 1K - Max: 32M - Default: 2M - Truncate current query based on the configuration before inserting into `stats_*_processlist` tables. - Refactor/fix code related to other processlist configurations. 1. `session_idle_show_processlist` value was not updated in `ProxySQL_Admin.variables`. 2. Pass processlist config as an argument to `MySQL_Threads_Handler::SQL3_Processlist` instead of using thread-local variables. Signed-off-by: Wazir Ahmed <wazir@proxysql.com>	8 months ago
René Cannaò	ecad6f9249	Merge pull request #5160 from sysown/v3.0_disable_pgsql_prometheus_5159 Separate Query Cache Metrics for MySQL and PostgreSQL & Disable Prometheus Metrics for PostgreSQL	9 months ago
René Cannaò	abafaccbc0	Merge pull request #5156 from sysown/v3.0_pgsql_stat_activity_issue_4682 Add backend_pid and backend_state to stats_pgsql_processlist and create stats_pgsql_stat_activity view	9 months ago
Rahim Kanji	5bfb6e216c	Few fixes	9 months ago
Rahim Kanji	e6d0338959	Turning off Prometheus metrics collection for PostgreSQL modules in ProxySQL	9 months ago
Rahim Kanji	fc4d7f76a0	Added 'SHOW FULL PGSQL ACTIVITY' and 'SHOW PGSQL ACTIVITY' handling	9 months ago
Rahim Kanji	f3ea179384	Add backend PID/state to stats_pgsql_processlist and create pg_stat_activity-style view - Added `backend_pid` and `backend_state` columns to `stats_pgsql_processlist` to display PostgreSQL backend process ID and connection state. - Created `stats_pgsql_stat_activity` view on top of `stats_pgsql_processlist` with column aliases matching PostgreSQL's `pg_stat_activity` for consistency.	9 months ago
Rahim Kanji	1e988c665e	Removed admin_handler_command_kill_connection	9 months ago
Evgeny Kuzin	949eda1ccd	generate postgres metrics in addition to mysql metrics	11 months ago
Rahim Kanji	703a5f35f5	Added stats_pgsql_prepared_statements_info table	11 months ago
Rahim Kanji	bc87d0d901	* For a Describe Portal message, peek at the next message in the extended query frame. If it is an Execute message, indicating that client is sending a Bind/Describe/Execute sequence skip processing the Describe Portal. This is because libpq automatically includes a Describe Portal during execution, and sending it again would be redundant. * Removed server_capabilities from PgSQL modules	12 months ago
René Cannaò	c48c3310a6	Check if database file exists before checking if it is writable Related to `bf3cca8639`	1 year ago
René Cannaò	bf3cca8639	exit if database file is not writable If database file (proxysql.db) is not writable, exit immediately instead of crashing because it is not writable. See issue #4893	1 year ago
René Cannaò	cb53db81ca	Export MySQL_Logger metrics for v3.0 Port of #4802	1 year ago
René Cannaò	2551d350e9	Port to 3.0 of in-memory query logging See #4713 for further details	1 year ago
Javier Jaramago Fernández	9e941854c3	Replace table based 'VACUUM' for SQLite3 'stats' database Introduced generic approach based on the potential benefits for 'VACUUM' over the 'stats' database instead of the previous selective, table based, approach.	1 year ago
René Cannaò	b9dcc9420a	PROXYSQL FLUSH LOGS forces debug database flush Also increased barching size from 31 to 100	1 year ago
René Cannaò	c2f1871adf	Make set_thread_name() optional	1 year ago
Rahim Kanji	6d614a4b99	Merge branch 'v3.0'	2 years ago
Rahim Kanji	110ce8d511	Fixed crash caused by sending single semicolon (;) on admin interface	2 years ago
Rahim Kanji	720441af20	Refactored and Optimized Query Cache with PgSQL Support * Added Query Cache Support for PgSQL: * Introduced new variables for PgSQL Query Cache management: * pgsql-query_cache_size_MB * pgsql-query_cache_soft_ttl_pct * pgsql-query_cache_stores_empty_result * pgsql-query_cache_handle_warnings (not yet utilized) * Overwriting Transaction Status: * Sending current transaction state to the client when a result is returned from the query cache. * This currently does not differentiate between normal and error transaction states—both are treated as active transaction states. * Cache Lifetime Management: Replaced manual reference counting with std::shared_ptr to efficiently manage the lifecycle of cache entries, ensuring proper memory deallocation and reducing complexity. * Unified Purging Logic: Refactored the query cache purging logic to use a single thread for both MySQL and PgSQL cache entries. * Code Cleanup: Removed unnecessary methods and data members to streamline the codebase and improve maintainability. * Performance Optimization: Improved memory management, reducing memory footprint and enhancing performance.	2 years ago
Rahim Kanji	0229daeb64	Added stats_pgsql_query_digest and stats_pgsql_query_digest_reset tables	2 years ago
René Cannaò	d4bd93672e	Merge pull request #79 from sysown/v2.x_refactore_query_processor Refactored Query Processor [WIP]	2 years ago
Rahim Kanji	e723082a1b	Refactored Query Processor Architecture - Introduced separate query processors for MySQL and PostgreSQL. - Removed unnecessary GTID logic from `PgSQL_Session`. - Added a command counter for PostgreSQL queries. - Refactored `Query_Processor` to serve as a base class containing core functionalities applicable to both MySQL and PostgreSQL. Changes: - Removed GTID-related code from `PgSQL_Logger` and `PgSQL_Session`: - bool with_gtid; - char gtid_buf[128]; - int gtid_hid; - Added new PostgreSQL-specific tables: - `pgsql_query_rules`, `pgsql_firewall_whitelist_rules`, `pgsql_query_rules_fast_routing` - `runtime_pgsql_query_rules`, `runtime_pgsql_firewall_whitelist_rules`, `runtime_pgsql_query_rules_fast_routing` - Added PostgreSQL-related variables: pgsql_thread___commands_stats pgsql_thread___query_digests pgsql_thread___query_digests_lowercase pgsql_thread___query_digests_no_digits pgsql_thread___query_digests_replace_null pgsql_thread___query_digests_normalize_digest_text pgsql_thread___query_digests_track_hostname pgsql_thread___query_digests_keep_comment pgsql_thread___query_digests_max_digest_length pgsql_thread___query_digests_max_query_length pgsql_thread___query_digests_grouping_limit pgsql_thread___query_digests_groups_grouping_limit pgsql_thread___enable_load_data_local_infile pgsql_thread___auditlog_filename pgsql_thread___auditlog_filesize pgsql_thread___eventslog_filename pgsql_thread___eventslog_filesize pgsql_thread___eventslog_default_log pgsql_thread___eventslog_format pgsql_thread___firewall_whitelist_errormsg pgsql_thread___firewall_whitelist_enabled pgsql_thread___query_processor_iterations pgsql_thread___query_processor_regex	2 years ago
René Cannaò	cbf4628ea3	Merge remote-tracking branch 'proxysql27/v2.7' into v2.x_postgres_merge27_20240917	2 years ago
Javier Jaramago Fernández	d516ca5e84	Merge branch 'v2.x_postgres' of https://github.com/sysown/proxysql_postgresql into v2.x-postgres_dual_password	2 years ago
René Cannaò	cf9d5fe21a	Merge pull request #4574 from sysown/set-thread-names Set thread names	2 years ago
Rahim Kanji	745449fcbc	Removed Client_Session.h and Client_Session.cpp	2 years ago
Javier Jaramago Fernández	bbda0a86eb	Add initial impl for dual-password support	2 years ago
Rahim Kanji	5bf827d8e4	Removed type selector	2 years ago
Rahim Kanji	3112a56da4	Added 'stats_pgsql_client_host_cache' and 'stats_pgsql_client_host_cache_reset' tables.	2 years ago
Rahim Kanji	e939cf088e	Added 'stats_pgsql_errors' and 'stats_pgsql_errors_reset' tables	2 years ago
Rahim Kanji	43d3ec82c8	Added stats_pgsql_processlist table * Implemented 'SHOW FULL PGSQL PROCESSLIST' command. * Implemented 'SHOW PGSQL PROCESSLIST' command.	2 years ago
Rahim Kanji	0b8df6df00	Added stats_pgsql_users table	2 years ago
Rahim Kanji	b4e1e1eea7	Added stats_pgsql_free_connections table	2 years ago
René Cannaò	58fa5ee5ec	Use uppercase in stats_mysql_query_digest.digest This fixes a regression introduced in `14ba7475d4` . Up to ProxySQL 2.5.1 , stats_mysql_query_digest.digest was uppercase . From ProxySQL 2.5.2 to 2.6.4 it became lowercase . This because we switched from: sprintf(... 0x16llX ...) to: ostringstream ... << "0x" << std::hex ...	2 years ago
Rahim Kanji	9cfe5b1761	Added stats_pgsql_connection_pool & stats_pgsql_connection_pool_reset tables	2 years ago
Rahim Kanji	c35a6a24cf	Added stats_pgsql_global table	2 years ago
Rahim Kanji	2730a41a38	Remove 'gtid_port' column from pgsql_servers table	2 years ago
Rahim Kanji	3c870e5c05	Removed unused columns + Bug fixes * Removed unused columns from pgsql_users. * Fixed checksum computation for mysql_users. * Fixed saving of pgsql_users and pgsql_servers to disk during initial load of ProxySQL from the configuration file..	2 years ago
Rene Cannao	7fe013fac2	Merge branch 'proxysql_v2.x_240813' into v2.x_postgres_merge_v2.x_240813 This merges ProxySQL v2.x up to 2024-08-13	2 years ago
Javier Jaramago Fernández	130d3fe04a	Issue warning when time interval variable gets rounded Variables specifying timing intervals, like 'stats_mysql_connections', may get rounded to the closer, valid time interval. Now a warning is issued for making the user aware of this rounding operation.	2 years ago
Rene Cannao	29383c7d47	Initialize all_modules_started also for release	2 years ago
Rene Cannao	28aba93526	Fix bug #44	2 years ago
Rene Cannao	a1fd4af1a6	Splitting ProxySQL_Admin	2 years ago
René Cannaò	01bb6e9a08	PRAGMA journal_mode=OFF on debug database PRAGMA journal_mode=OFF should drastically reduce CPU overhead	2 years ago
Rene Cannao	a3193ad1dd	PRAGMA journal_mode=OFF on debug database PRAGMA journal_mode=OFF should drastically reduce CPU overhead	2 years ago
Rene Cannao	465eaef87e	Merge branch 'v2.x_postgres' into v2.x_240630	2 years ago
René Cannaò	25f02a33ce	Merge pull request #4496 from sysown/v2.6.x-freebsd_fixes various FreeBSD related fixes	2 years ago
René Cannaò	2eb1d0d1c1	Merge pull request #20 from sysown/v2.x_postgres_backend Added generic error handling, transaction detection and handling	2 years ago
Andriy Utkin	5e79afa8e0	Set thread names	2 years ago
Rahim Kanji	c5c9b4c8d7	Changed PostgreSQL default port * Changed admin port from 6034 to 6132 * Changed pgsql backend port from 6035 to 6133	2 years ago
Rahim Kanji	d3e9c9c85a	Merge remote-tracking branch 'remotes/Master/v2.x' into v2.x_proxysql_postgres_merge	2 years ago
Javier Jaramago Fernández	188faae40d	Remove old commented/outdated code on 'ProxySQL_Admin'	2 years ago

1 2 3 4 5 ...

1024 Commits (4e39f3bdc4888ee7bd34b3fcba3b986503efe20b)