47 Commits (ade0130e67e88b9052d0dc4e76d48d98d3328485)

Author	SHA1	Message	Date
Rene Cannao	18cc246209	fix: Add missing proxy declarations to MAIN_PROXY_SQLITE3 branch ... Add proxy_sqlite3_enable_load_extension and proxy_sqlite3_global_stats_row_step to the #else (MAIN_PROXY_SQLITE3) branch in sqlite3db.h to maintain symmetry with the extern declarations and prevent ODR violations. Addresses coderabbitai review comment.	1 month ago
Rene Cannao	b4f521c634	Merge v3.1-MCP2 into v3.1-vec ... Conflict resolution summary: **Features preserved from v3.1-vec:** - mcp_use_ssl variable (HTTP/HTTPS mode support) - RAG tool handler support - FTS (Full Text Search) functionality in MySQL_Tool_Handler **Changes accepted from v3.1-MCP2:** - Schema isolation for catalog (removed mcp_catalog_path/fts_path variables) - Schema parameter added to catalog functions - Comprehensive query rewriting improvements in Query_Tool_Handler - Format string fixes in vector_db_performance-t.cpp **Key decisions:** - catalog_path/fts_path: Removed per v3.1-MCP2 commit 35b0b224 (intentional) - MySQL_Catalog.cpp: Accepted v3.1-MCP2 version (schema isolation improvements) - Query_Tool_Handler.cpp: Accepted v3.1-MCP2 version (significant improvements) - Makefile: Kept v3.1-vec's EXCLUDE_TRACKING_VARIABLES (correct spelling) **Build notes:** - MySQL_Tool_Handler now requires schema parameter for catalog operations - Catalog path is hardcoded to datadir/mcp_catalog.db - FTS functionality preserved but may need updates for schema isolation	1 month ago
Rene Cannao	0db022a179	Apply fixes	1 month ago
Rene Cannao	7bf9121057	sqlite3: fix duplicate proxy declarations and add forward declarations	1 month ago
Rene Cannao	8dc4246bdc	Introduce canonical proxy_sqlite3 symbol TU; update lib Makefile; remove MAIN_PROXY_SQLITE3 from main.cpp	1 month ago
Rahim Kanji	14aef13827	Add method to execute already prepared SQLite3 statements, supporting ... both non-SELECT and SELECT queries.	2 months ago
Rene Cannao	a577491f42	Refactor issue 4855 fix: Use sqlite3_total_changes64 difference approach ... PROBLEM: The initial fix used a DDL detection approach which required maintaining a list of query types that should return 0 affected rows. This approach was brittle and could miss edge cases like commented queries or complex statements. SOLUTION: Instead of detecting DDL queries, use sqlite3_total_changes64() to measure the actual change count before and after each query execution. The difference between total_changes before and after represents the true affected rows count for the current query, regardless of query type. CHANGES: - Added proxy_sqlite3_total_changes64 function pointer and initialization - Rewrote execute_statement() and execute_statement_raw() to use total_changes difference approach - This automatically handles all query types (DDL, DML, comments, etc.) - Added comprehensive TAP test covering INSERT, CREATE, DROP, VACUUM, UPDATE, and BEGIN operations BENEFITS: - More robust and accurate than DDL detection approach - Handles edge cases like commented queries automatically - No maintenance overhead for new query types - Simpler and cleaner implementation - Still fixes both Admin interface and SQLite3 Server This approach is mathematically sound: affected_rows = total_changes_after - total_changes_before, which gives the exact number of rows changed by the current query execution. Fixes #4855	3 months ago
Javier Jaramago Fernández	25d5681c61	Fix potential overflow for 'sqlite3_status' memory metrics ... The previous call have been replaced by 'sqlite3_status64' when querying 'SQLITE_STATUS_MEMORY_USED' to avoid potential integer overflows.	8 months ago
Javier Jaramago Fernández	6f2798abe0	Fix memory leak in 'ProxySQL_Admin::stats___mysql_global' - Closes #4976 ... The resource leak was associated to prepared statements (sqlite3_stmt). Since the detection of these leaks is tricky, because the resources held by 'SQLite3' never lose their references, a helper type ('stmt_unique_ptr') and a safer variation for 'prepare_v2' have been introduced. This helper type should automatically handle the lifetime of the statements, avoiding these kind of leaks in the future. The non-safe alternative has been flagged as 'deprecated', so it can be easily identified in later refactors or slowly removed during development.	8 months ago
René Cannaò	e958946b82	Handle STATUS command without backends ... Handles special queries executed by the STATUS command in mysql cli . Specifically: - "select DATABASE(), USER() limit 1" - "select @@character_set_client, @@character_set_connection, @@character_set_server, @@character_set_database limit 1" Closes #4396 Closes #4426	2 years ago
Javier Jaramago Fernández	3d06427efe	Improve simulator support for AWS Aurora ... - Preserve old simulator behavior via 'TEST_AURORA_RANDOM' macro. - Add new deterministic behavior for testing particular scenarios. - Add support for simulating multiple clusters via an updated 'REPLICA_HOST_STATUS' table and hostgroup based selection. - Provide a sensible default population for manual testing.	3 years ago
René Cannaò	d4d2298a6f	Debug logging to disk ... Introduced new schema debugdb_disk (on disk) and table debug_log . Debug entries will now go to this new table based on the value of variable admin-debug_output . If: 1) default behavior: write to stderr 2) write only to debugdb_disk.debug_log 3) 3 = 1+2 = write to stderr and to debugdb_disk.debug_log All these functionalities exist only on debug build. Table debugdb_disk.debug_log has also a column named "note" that can be used to write user notes during debugging.	3 years ago
Javier Jaramago Fernández	76b64bed95	Added extra logging reporting all galera nodes status when ProxySQL moves a node from hostgroup	5 years ago
René Cannaò	35c851b9fa	Moved LoadPlugin_sqlite3_plugin into SQLite3DB ... In this way we can initialize it from other applications, for example TAP tests	6 years ago
René Cannaò	c808c1f1a8	Move definition of sqlite3 funcs out of main.cpp	6 years ago
René Cannaò	2bbd84d2c9	Support for SQLite3 plugin ... This commit prepares proxysql for further development related to SQLite3. Specifically, it allows to either use built-in SQLite3 library, or to load it from a plugin.	6 years ago
René Cannaò	cec0404cba	Support for LIMIT/OFFSET in SQLite3_result()	6 years ago
Valentin Rakush	d5a7836011	Overload execute_statement method in sqlite3db	6 years ago
René Cannaò	27284444dd	Optimization on SQLite3 internal library	6 years ago
René Cannaò	7df2227c7c	Merge pull request #2182 from sysown/v2.0.6-perfbench1 ... V2.0.6 perfbench1	7 years ago
René Cannaò	ee5b54d983	Create a wrapper around sqlite3_prepare_v2 ... This should fix several crashes if the internal db is locked or busy	7 years ago
René Cannaò	2fec51c0a9	A lot of changes to make query digest map faster ... Extended SQLite3_result to support a mutex for concurrent rows insert. Extensively rewritten get_query_digests() and get_query_digests_reset() to support multiple threads, and concurrently generate a resultset. Function get_query_digests_reset() defer the deletion of objects from query digest map after generating the resultset for Admin. This drastically reduce the amount of time that query digest map is locked. Rewritten get_query_digests_total_size() to make it multi-threaded. Added 3 new functions for purging: - purge_query_digests() : wrapper - purge_query_digests_sync() : synchronous purge of query digest map, either single-threaded or multi-threaded - purge_query_digests_async() : asynchronous purge of query digest map, single-threaded only If table `stats_mysql_query_digest_reset` is queried but not `stats_mysql_query_digest`, only the first one is populated. If table `stats_mysql_query_digest_reset` is queried and also `stats_mysql_query_digest`, the second one is populated as a copy from the first. Implemented new SQL `TRUNCATE [TABLE]` for `[stats.]stats_mysql_query_digest[_reset]`: this will purge the query digest map as fast as possible without creating any table in SQLite. For testing/benchmark: - modified ProxySQL_Test___GenerateRandomQueryInDigestTable() to generate more random data and randomly switch `mysql_thread___query_digests_normalize_digest_text` Extended command `PROXYSQLTEST` with the following tests: - 1 : generates entries in digest map - 2 : gets all the entries from the digest map, but without writing to DB, with multiple threads - 3 : gets all the entries from the digest map and reset, but without writing to DB, with multiple threads - 4 : purges the digest map, synchronously, in single thread - 5 : purges the digest map, synchronously, in multiple threads - 6 : purges the digest map, asynchronously, in single thread (this seems the faster) Replaced some sprintf() with new function my_itoa() . Modified class QP_query_digest_stats() to include username, schemaname and client_address if no longer than 24 bytes. This makes storage requirement for QP_query_digest_stats() bigger, but reduces a lot of calls to memory management. Modified QP_query_digest_stats::get_row() to drastically reduce the number of malloc/free and string copy. Overall, this commit has the following effects: - it increases the memory usage for query digest map. On test workload, this results in 10% more memory usage - drastically reduces the calls to memory management - populating `stats_mysql_query_digest_reset` is 3 times faster - the amount of time in which query digest map is locked is reduced by 95%	7 years ago
René Cannaò	9b14103236	Collect memory statistics about query rules	7 years ago
René Cannaò	b5d8c4d3ba	Added table mysql_query_rules_fast_routing ... This table is responsible for performing routing for very specific use case scenarios, like having tens of thousands of schemas	8 years ago
René Cannaò	4fb9e3f88b	Dump in error log status of mysql_servers during reconfiguration ... Close issue #1255	8 years ago
René Cannaò	ee8110a515	Enhancement on Cluster solution ... This commit introduces: 2 new tables: * `runtime_checksums_values` : stores checksums of configurations in runtime. For now for `mysql_query_rules`, `mysql_servers` and `mysql_users` * `stats_proxysql_servers_checksums` : when clustering is enabled, it collects all metrics from `runtime_checksums_values` from all its peers 3 new global variables that defines it checksum needs to be generated during `LOAD ... TO RUNTIME` * `admin-checksum_mysql_query_rules` * `admin-checksum_mysql_servers` * `admin-checksum_mysql_users` ProxySQL Cluster connections now have timeouts: * 1 second timeout for CONNECT and WRITE * 60 seconds timeout for READ (useful for long poll)	9 years ago
René Cannaò	aa8f0c5c43	Remove spinlock from class SQLite3DB #977	9 years ago
René Cannaò	f550cf1340	Speeding up user management ... LOAD MYSQL USERS TO RUNTIME is not faster Introduced LOAD MYSQL USER username TO RUNTIME	9 years ago
René Cannaò	a6c53d0d12	Revert "Refactor header includes."	9 years ago
Maxim Bublis	30dbcc21a8	Refactor header includes. ... * reduce build time by 25%; * remove circular dependency on `cpp.h` and other headers. * fixes issue with `my_global.h` from mariadbclient which redefines `__attribute__`, that leads to broken STL and GCC intrinsics if `<algorithm>` wasn't included prior including this header; * fixes another issue with redefined `__attribute__` which leads to `_conn_exchange_t` not being aligned to cache line size; * removes `pthread_setstacksize` calls as `my_global.h` redefines `pthread_setstacksize` to be `pthread_dummy(0)` which led to stack size being never adjusted; * add missing include guards to some headers; * remove unused proxysql_hash.h header.	9 years ago
Maxim Bublis	043a321e55	Cleanup legacy codebase.	9 years ago
René Cannaò	06d7649ffc	Reduce allocations in SQLite3_row #809 ... This optimization, combined with the optimization already applied in #802, improves the execution time of LOAD MYSQL USERS TO RUNTIME by 12x	9 years ago
René Cannaò	c68bde68ee	Minor optimizatin sqlite3db	9 years ago
ethanblackburn	5885e5f5c2	Add checksum mysql replication hostgroups command	10 years ago
ethanblackburn	2ab4833498	Add checksum function to SQLite3_result	10 years ago
René Cannaò	921ccd5813	Bugfix on mysql_query_rules ... better handling of NULL fields	11 years ago
René Cannaò	e6067a953d	External locking for SQLite3DB() ... Closes #368	11 years ago
René Cannaò	dca5b434fc	Many bug fixes: ... Simplified the ping within MySQL_Thread SQLite3_row::add_fields() to handle empty string prot_status was not initialized in MySQL_Protocol Session's default schema was always mysql_thread___default_schema , now fixed with correct value default_schema is now read from config file	11 years ago
René Cannaò	ba16850d2d	Adding function SQLite3DB::return_one_int()	11 years ago
René Cannaò	322a4f021f	Enhancement on Query Processor and draft statistics interface ... - changed hits from uint64_t to int; - QP_rule_t has a pointer to its parent : this is done to speed up lookup - implemented get_current_query_rules() to execute SAVE MYSQL QUERY RULES FROM RUNTIME (issue #93) - implemented get_stats_query_rules() to save statistics on stats_mysql_query_rules (issue #139) - created 2 new classes to easily process statistics in Query Processor : QP_rule_text and QP_rule_text_hitsonly - extended classes SQLite3_row and SQLite3_result - every __admin_refresh_interval milliseconds Admin refreshes statistics - defined table stats_mysql_query_rules - deprecated __attach_configdb_to_admindb() , replaced with __attach_db_to_admindb (used in issue #142) - to admindb are now attached both configdb and statsdb (issue #142) - drafted a way to update hits on Query Processor . This method will go away when issue #146 will be implemented - fixed crashing bug in issue #145	11 years ago
René Cannaò	ccc40a0b2a	Work in issues #91, #92 and #93	11 years ago
René Cannaò	a49e991fea	Better handle of NULL value returned by SQLite3	11 years ago
René Cannaò	ee34919f9f	Issues #19 and #23 ... For issue #19: - create MySQL_Protocol::get_myds() - create MySQL_Protocol::generate_pkt_row() - modified MySQL_Session::(*admin_func)(MySQL_Session *) to MySQL_Session::(*admin_func)(MySQL_Session *,ProxySQL_Admin *,PtrSize_t *) so it can access more data - fixed an invalid strdup in SQLite3_row::add_fields() - added SQLite3_result::rows_count - SQLite3DB instances in Standard_ProxySQL_Admin are now public - create Standard_ProxySQL_Admin::SQLite3_to_MySQL() For issue #23 : - added __dump_pkt() in mysql_data_stream.cpp Bug fix: Added a missing pthread_mutex_unlock() in Admin/child_mysql() Minor clean-ups	11 years ago
René Cannaò	dbae40a9a9	Issue #16 ... Added functions: - Standard_ProxySQL_Admin::flush_debug_levels_mem_to_db - Standard_ProxySQL_Admin::flush_debug_levels_db_to_mem debug_levels is populated at start up	11 years ago
René Cannaò	43a82215b0	Issue #12 ... Added functions: - __add_active_users - __delete_inactive_users - __refresh_users Fixed small bug in SQLite3_result::add_column_definition() MySQL Authentication module is now loaded before the Admin module	11 years ago
René Cannaò	5ffd3c8924	Issue #4 , #5 , #6 and #7 ... For issue #4: - removed fill_table__server_status() - removed ADMIN_SQLITE_TABLE_MYSQL_SERVER_STATUS - fixed ADMIN_SQLITE_TABLE_MYSQL_SERVER For issue #5: - created functions __insert_or_ignore_maintable_select_disktable() , __delete_disktable() , __insert_or_replace_disktable_select_maintable() and __attach_configdb_to_admindb() For issue #6 and #7: - table ADMIN_SQLITE_TABLE_MYSQL_USERS has fields use_ssl, backend and frontend	11 years ago
René Cannaò	68d6786320	first commit	11 years ago