47 Commits (61fc7a129cd90cef3fcbb40b4ef786462e2de651)

Author	SHA1	Message	Date
Rene Cannao	18cc246209	fix: Add missing proxy declarations to MAIN_PROXY_SQLITE3 branch ... Add proxy_sqlite3_enable_load_extension and proxy_sqlite3_global_stats_row_step to the #else (MAIN_PROXY_SQLITE3) branch in sqlite3db.h to maintain symmetry with the extern declarations and prevent ODR violations. Addresses coderabbitai review comment.	3 months ago
Rene Cannao	b4f521c634	Merge v3.1-MCP2 into v3.1-vec ... Conflict resolution summary: **Features preserved from v3.1-vec:** - mcp_use_ssl variable (HTTP/HTTPS mode support) - RAG tool handler support - FTS (Full Text Search) functionality in MySQL_Tool_Handler **Changes accepted from v3.1-MCP2:** - Schema isolation for catalog (removed mcp_catalog_path/fts_path variables) - Schema parameter added to catalog functions - Comprehensive query rewriting improvements in Query_Tool_Handler - Format string fixes in vector_db_performance-t.cpp **Key decisions:** - catalog_path/fts_path: Removed per v3.1-MCP2 commit 35b0b224 (intentional) - MySQL_Catalog.cpp: Accepted v3.1-MCP2 version (schema isolation improvements) - Query_Tool_Handler.cpp: Accepted v3.1-MCP2 version (significant improvements) - Makefile: Kept v3.1-vec's EXCLUDE_TRACKING_VARIABLES (correct spelling) **Build notes:** - MySQL_Tool_Handler now requires schema parameter for catalog operations - Catalog path is hardcoded to datadir/mcp_catalog.db - FTS functionality preserved but may need updates for schema isolation	3 months ago
Rene Cannao	0db022a179	Apply fixes	3 months ago
Rene Cannao	7bf9121057	sqlite3: fix duplicate proxy declarations and add forward declarations	3 months ago
Rene Cannao	8dc4246bdc	Introduce canonical proxy_sqlite3 symbol TU; update lib Makefile; remove MAIN_PROXY_SQLITE3 from main.cpp	3 months ago
Rahim Kanji	14aef13827	Add method to execute already prepared SQLite3 statements, supporting ... both non-SELECT and SELECT queries.	4 months ago
Rene Cannao	a577491f42	Refactor issue 4855 fix: Use sqlite3_total_changes64 difference approach ... PROBLEM: The initial fix used a DDL detection approach which required maintaining a list of query types that should return 0 affected rows. This approach was brittle and could miss edge cases like commented queries or complex statements. SOLUTION: Instead of detecting DDL queries, use sqlite3_total_changes64() to measure the actual change count before and after each query execution. The difference between total_changes before and after represents the true affected rows count for the current query, regardless of query type. CHANGES: - Added proxy_sqlite3_total_changes64 function pointer and initialization - Rewrote execute_statement() and execute_statement_raw() to use total_changes difference approach - This automatically handles all query types (DDL, DML, comments, etc.) - Added comprehensive TAP test covering INSERT, CREATE, DROP, VACUUM, UPDATE, and BEGIN operations BENEFITS: - More robust and accurate than DDL detection approach - Handles edge cases like commented queries automatically - No maintenance overhead for new query types - Simpler and cleaner implementation - Still fixes both Admin interface and SQLite3 Server This approach is mathematically sound: affected_rows = total_changes_after - total_changes_before, which gives the exact number of rows changed by the current query execution. Fixes #4855	5 months ago
Javier Jaramago Fernández	25d5681c61	Fix potential overflow for 'sqlite3_status' memory metrics ... The previous call have been replaced by 'sqlite3_status64' when querying 'SQLITE_STATUS_MEMORY_USED' to avoid potential integer overflows.	9 months ago
Javier Jaramago Fernández	6f2798abe0	Fix memory leak in 'ProxySQL_Admin::stats___mysql_global' - Closes #4976 ... The resource leak was associated to prepared statements (sqlite3_stmt). Since the detection of these leaks is tricky, because the resources held by 'SQLite3' never lose their references, a helper type ('stmt_unique_ptr') and a safer variation for 'prepare_v2' have been introduced. This helper type should automatically handle the lifetime of the statements, avoiding these kind of leaks in the future. The non-safe alternative has been flagged as 'deprecated', so it can be easily identified in later refactors or slowly removed during development.	10 months ago
René Cannaò	e958946b82	Handle STATUS command without backends ... Handles special queries executed by the STATUS command in mysql cli . Specifically: - "select DATABASE(), USER() limit 1" - "select @@character_set_client, @@character_set_connection, @@character_set_server, @@character_set_database limit 1" Closes #4396 Closes #4426	2 years ago
Javier Jaramago Fernández	3d06427efe	Improve simulator support for AWS Aurora ... - Preserve old simulator behavior via 'TEST_AURORA_RANDOM' macro. - Add new deterministic behavior for testing particular scenarios. - Add support for simulating multiple clusters via an updated 'REPLICA_HOST_STATUS' table and hostgroup based selection. - Provide a sensible default population for manual testing.	3 years ago
René Cannaò	d4d2298a6f	Debug logging to disk ... Introduced new schema debugdb_disk (on disk) and table debug_log . Debug entries will now go to this new table based on the value of variable admin-debug_output . If: 1) default behavior: write to stderr 2) write only to debugdb_disk.debug_log 3) 3 = 1+2 = write to stderr and to debugdb_disk.debug_log All these functionalities exist only on debug build. Table debugdb_disk.debug_log has also a column named "note" that can be used to write user notes during debugging.	3 years ago
Javier Jaramago Fernández	76b64bed95	Added extra logging reporting all galera nodes status when ProxySQL moves a node from hostgroup	5 years ago
René Cannaò	35c851b9fa	Moved LoadPlugin_sqlite3_plugin into SQLite3DB ... In this way we can initialize it from other applications, for example TAP tests	6 years ago
René Cannaò	c808c1f1a8	Move definition of sqlite3 funcs out of main.cpp	6 years ago
René Cannaò	2bbd84d2c9	Support for SQLite3 plugin ... This commit prepares proxysql for further development related to SQLite3. Specifically, it allows to either use built-in SQLite3 library, or to load it from a plugin.	6 years ago
René Cannaò	cec0404cba	Support for LIMIT/OFFSET in SQLite3_result()	6 years ago
Valentin Rakush	d5a7836011	Overload execute_statement method in sqlite3db	6 years ago
René Cannaò	27284444dd	Optimization on SQLite3 internal library	6 years ago
René Cannaò	7df2227c7c	Merge pull request #2182 from sysown/v2.0.6-perfbench1 ... V2.0.6 perfbench1	7 years ago
René Cannaò	ee5b54d983	Create a wrapper around sqlite3_prepare_v2 ... This should fix several crashes if the internal db is locked or busy	7 years ago
René Cannaò	2fec51c0a9	A lot of changes to make query digest map faster ... Extended SQLite3_result to support a mutex for concurrent rows insert. Extensively rewritten get_query_digests() and get_query_digests_reset() to support multiple threads, and concurrently generate a resultset. Function get_query_digests_reset() defer the deletion of objects from query digest map after generating the resultset for Admin. This drastically reduce the amount of time that query digest map is locked. Rewritten get_query_digests_total_size() to make it multi-threaded. Added 3 new functions for purging: - purge_query_digests() : wrapper - purge_query_digests_sync() : synchronous purge of query digest map, either single-threaded or multi-threaded - purge_query_digests_async() : asynchronous purge of query digest map, single-threaded only If table `stats_mysql_query_digest_reset` is queried but not `stats_mysql_query_digest`, only the first one is populated. If table `stats_mysql_query_digest_reset` is queried and also `stats_mysql_query_digest`, the second one is populated as a copy from the first. Implemented new SQL `TRUNCATE [TABLE]` for `[stats.]stats_mysql_query_digest[_reset]`: this will purge the query digest map as fast as possible without creating any table in SQLite. For testing/benchmark: - modified ProxySQL_Test___GenerateRandomQueryInDigestTable() to generate more random data and randomly switch `mysql_thread___query_digests_normalize_digest_text` Extended command `PROXYSQLTEST` with the following tests: - 1 : generates entries in digest map - 2 : gets all the entries from the digest map, but without writing to DB, with multiple threads - 3 : gets all the entries from the digest map and reset, but without writing to DB, with multiple threads - 4 : purges the digest map, synchronously, in single thread - 5 : purges the digest map, synchronously, in multiple threads - 6 : purges the digest map, asynchronously, in single thread (this seems the faster) Replaced some sprintf() with new function my_itoa() . Modified class QP_query_digest_stats() to include username, schemaname and client_address if no longer than 24 bytes. This makes storage requirement for QP_query_digest_stats() bigger, but reduces a lot of calls to memory management. Modified QP_query_digest_stats::get_row() to drastically reduce the number of malloc/free and string copy. Overall, this commit has the following effects: - it increases the memory usage for query digest map. On test workload, this results in 10% more memory usage - drastically reduces the calls to memory management - populating `stats_mysql_query_digest_reset` is 3 times faster - the amount of time in which query digest map is locked is reduced by 95%	7 years ago
René Cannaò	9b14103236	Collect memory statistics about query rules	7 years ago
René Cannaò	b5d8c4d3ba	Added table mysql_query_rules_fast_routing ... This table is responsible for performing routing for very specific use case scenarios, like having tens of thousands of schemas	8 years ago
René Cannaò	4fb9e3f88b	Dump in error log status of mysql_servers during reconfiguration ... Close issue #1255	8 years ago
René Cannaò	ee8110a515	Enhancement on Cluster solution ... This commit introduces: 2 new tables: * `runtime_checksums_values` : stores checksums of configurations in runtime. For now for `mysql_query_rules`, `mysql_servers` and `mysql_users` * `stats_proxysql_servers_checksums` : when clustering is enabled, it collects all metrics from `runtime_checksums_values` from all its peers 3 new global variables that defines it checksum needs to be generated during `LOAD ... TO RUNTIME` * `admin-checksum_mysql_query_rules` * `admin-checksum_mysql_servers` * `admin-checksum_mysql_users` ProxySQL Cluster connections now have timeouts: * 1 second timeout for CONNECT and WRITE * 60 seconds timeout for READ (useful for long poll)	9 years ago
René Cannaò	aa8f0c5c43	Remove spinlock from class SQLite3DB #977	9 years ago
René Cannaò	f550cf1340	Speeding up user management ... LOAD MYSQL USERS TO RUNTIME is not faster Introduced LOAD MYSQL USER username TO RUNTIME	9 years ago
René Cannaò	a6c53d0d12	Revert "Refactor header includes."	9 years ago
Maxim Bublis	30dbcc21a8	Refactor header includes. ... * reduce build time by 25%; * remove circular dependency on `cpp.h` and other headers. * fixes issue with `my_global.h` from mariadbclient which redefines `__attribute__`, that leads to broken STL and GCC intrinsics if `<algorithm>` wasn't included prior including this header; * fixes another issue with redefined `__attribute__` which leads to `_conn_exchange_t` not being aligned to cache line size; * removes `pthread_setstacksize` calls as `my_global.h` redefines `pthread_setstacksize` to be `pthread_dummy(0)` which led to stack size being never adjusted; * add missing include guards to some headers; * remove unused proxysql_hash.h header.	9 years ago
Maxim Bublis	043a321e55	Cleanup legacy codebase.	9 years ago
René Cannaò	06d7649ffc	Reduce allocations in SQLite3_row #809 ... This optimization, combined with the optimization already applied in #802, improves the execution time of LOAD MYSQL USERS TO RUNTIME by 12x	10 years ago
René Cannaò	c68bde68ee	Minor optimizatin sqlite3db	10 years ago
ethanblackburn	5885e5f5c2	Add checksum mysql replication hostgroups command	10 years ago
ethanblackburn	2ab4833498	Add checksum function to SQLite3_result	10 years ago
René Cannaò	921ccd5813	Bugfix on mysql_query_rules ... better handling of NULL fields	11 years ago
René Cannaò	e6067a953d	External locking for SQLite3DB() ... Closes #368	11 years ago
René Cannaò	dca5b434fc	Many bug fixes: ... Simplified the ping within MySQL_Thread SQLite3_row::add_fields() to handle empty string prot_status was not initialized in MySQL_Protocol Session's default schema was always mysql_thread___default_schema , now fixed with correct value default_schema is now read from config file	11 years ago
René Cannaò	ba16850d2d	Adding function SQLite3DB::return_one_int()	11 years ago
René Cannaò	322a4f021f	Enhancement on Query Processor and draft statistics interface ... - changed hits from uint64_t to int; - QP_rule_t has a pointer to its parent : this is done to speed up lookup - implemented get_current_query_rules() to execute SAVE MYSQL QUERY RULES FROM RUNTIME (issue #93) - implemented get_stats_query_rules() to save statistics on stats_mysql_query_rules (issue #139) - created 2 new classes to easily process statistics in Query Processor : QP_rule_text and QP_rule_text_hitsonly - extended classes SQLite3_row and SQLite3_result - every __admin_refresh_interval milliseconds Admin refreshes statistics - defined table stats_mysql_query_rules - deprecated __attach_configdb_to_admindb() , replaced with __attach_db_to_admindb (used in issue #142) - to admindb are now attached both configdb and statsdb (issue #142) - drafted a way to update hits on Query Processor . This method will go away when issue #146 will be implemented - fixed crashing bug in issue #145	11 years ago
René Cannaò	ccc40a0b2a	Work in issues #91, #92 and #93	11 years ago
René Cannaò	a49e991fea	Better handle of NULL value returned by SQLite3	11 years ago
René Cannaò	ee34919f9f	Issues #19 and #23 ... For issue #19: - create MySQL_Protocol::get_myds() - create MySQL_Protocol::generate_pkt_row() - modified MySQL_Session::(*admin_func)(MySQL_Session *) to MySQL_Session::(*admin_func)(MySQL_Session *,ProxySQL_Admin *,PtrSize_t *) so it can access more data - fixed an invalid strdup in SQLite3_row::add_fields() - added SQLite3_result::rows_count - SQLite3DB instances in Standard_ProxySQL_Admin are now public - create Standard_ProxySQL_Admin::SQLite3_to_MySQL() For issue #23 : - added __dump_pkt() in mysql_data_stream.cpp Bug fix: Added a missing pthread_mutex_unlock() in Admin/child_mysql() Minor clean-ups	12 years ago
René Cannaò	dbae40a9a9	Issue #16 ... Added functions: - Standard_ProxySQL_Admin::flush_debug_levels_mem_to_db - Standard_ProxySQL_Admin::flush_debug_levels_db_to_mem debug_levels is populated at start up	12 years ago
René Cannaò	43a82215b0	Issue #12 ... Added functions: - __add_active_users - __delete_inactive_users - __refresh_users Fixed small bug in SQLite3_result::add_column_definition() MySQL Authentication module is now loaded before the Admin module	12 years ago
René Cannaò	5ffd3c8924	Issue #4 , #5 , #6 and #7 ... For issue #4: - removed fill_table__server_status() - removed ADMIN_SQLITE_TABLE_MYSQL_SERVER_STATUS - fixed ADMIN_SQLITE_TABLE_MYSQL_SERVER For issue #5: - created functions __insert_or_ignore_maintable_select_disktable() , __delete_disktable() , __insert_or_replace_disktable_select_maintable() and __attach_configdb_to_admindb() For issue #6 and #7: - table ADMIN_SQLITE_TABLE_MYSQL_USERS has fields use_ssl, backend and frontend	12 years ago
René Cannaò	68d6786320	first commit	12 years ago