90 Commits (fcaf342803c835dbb7e88acb56f4e2a2017f7947)

Author	SHA1	Message	Date
René Cannaò	6bed334db1	stats_history.history_mysql_query_digest #2368 ... Implementation of new table stats_history.history_mysql_query_digest #2368 Implemented command `SAVE MYSQL DIGEST TO DISK`. Save to disk with really minimal memory footprint compared to querying `stats_mysql_query_digest` or `stats_mysql_query_digest_reset` . Implemented new admin variable `admin-stats_mysql_query_digest_to_disk` to automatically save to disk.	7 years ago
René Cannaò	a0ea0c118c	V2.0.8 fast routing (#2309) ... * Implementation of unit tests related to mysql_query_rules_fast_routing * Implementation of rules_fast_routing with khash * Adding memory usage for new fast routing algo	7 years ago
Valentin Rakush	757684c6d9	[FR] Group Replication Timeout count	7 years ago
René Cannaò	2fec51c0a9	A lot of changes to make query digest map faster ... Extended SQLite3_result to support a mutex for concurrent rows insert. Extensively rewritten get_query_digests() and get_query_digests_reset() to support multiple threads, and concurrently generate a resultset. Function get_query_digests_reset() defer the deletion of objects from query digest map after generating the resultset for Admin. This drastically reduce the amount of time that query digest map is locked. Rewritten get_query_digests_total_size() to make it multi-threaded. Added 3 new functions for purging: - purge_query_digests() : wrapper - purge_query_digests_sync() : synchronous purge of query digest map, either single-threaded or multi-threaded - purge_query_digests_async() : asynchronous purge of query digest map, single-threaded only If table `stats_mysql_query_digest_reset` is queried but not `stats_mysql_query_digest`, only the first one is populated. If table `stats_mysql_query_digest_reset` is queried and also `stats_mysql_query_digest`, the second one is populated as a copy from the first. Implemented new SQL `TRUNCATE [TABLE]` for `[stats.]stats_mysql_query_digest[_reset]`: this will purge the query digest map as fast as possible without creating any table in SQLite. For testing/benchmark: - modified ProxySQL_Test___GenerateRandomQueryInDigestTable() to generate more random data and randomly switch `mysql_thread___query_digests_normalize_digest_text` Extended command `PROXYSQLTEST` with the following tests: - 1 : generates entries in digest map - 2 : gets all the entries from the digest map, but without writing to DB, with multiple threads - 3 : gets all the entries from the digest map and reset, but without writing to DB, with multiple threads - 4 : purges the digest map, synchronously, in single thread - 5 : purges the digest map, synchronously, in multiple threads - 6 : purges the digest map, asynchronously, in single thread (this seems the faster) Replaced some sprintf() with new function my_itoa() . Modified class QP_query_digest_stats() to include username, schemaname and client_address if no longer than 24 bytes. This makes storage requirement for QP_query_digest_stats() bigger, but reduces a lot of calls to memory management. Modified QP_query_digest_stats::get_row() to drastically reduce the number of malloc/free and string copy. Overall, this commit has the following effects: - it increases the memory usage for query digest map. On test workload, this results in 10% more memory usage - drastically reduces the calls to memory management - populating `stats_mysql_query_digest_reset` is 3 times faster - the amount of time in which query digest map is locked is reduced by 95%	7 years ago
René Cannaò	b12caa1d8c	Adding support for automated testing of Galera ... Using SQLite3 Server as backend	7 years ago
René Cannaò	58ff4ff892	Merge branch 'v2.0.4' into v2.0.1-aurora	7 years ago
René Cannaò	495c933624	Adding table stats.stats_mysql_free_connections ... This table exports metrics about connections in connection pool	7 years ago
René Cannaò	528d8cac38	Improved debugging ... Added new command `PROXYSQL INTERNAL SESSION` that clients can execute to receive internal information about their own connection in JSON format. Added JSON library. Recompiled SQLite3 to support JSON. Added new column `extended_info` in `stats_mysql_processlist`. Added new mysql variable `mysql-show_processlist_extended` that determine the content of `stats_mysql_processlist.extended_info`: - 0 : no info - 1 : JSON format - 2 : JSON format with pretty printing	7 years ago
René Cannaò	2e20854684	Second commit (not complete) to support Aurora	7 years ago
René Cannaò	b82a1c18ac	Merge pull request #1857 from sysown/v2.0.0-digestSplit ... V2.0.0 digest split	7 years ago
René Cannaò	782aae072e	Implement automatic vacuum of stats schema #1011	7 years ago
René Cannaò	47bf5faff7	Second large commit to support LDAP ... Functions add() and lookup() in MySQL_LDAP_Authentication have support for backend_username. Added mysql_ldap_mapping table. Created Admin::init_ldap() to be called after LDAP initialization. Added better LDAP caching. LOAD LDAP MAPPING TO RUNTIME cleans part of the cache (association to backend user). All queries will have a comment "proxysql-ldap-user=%s" to track original user	8 years ago
René Cannaò	d96108123d	Scheleton for LDAP support	8 years ago
René Cannaò	601f9b0862	Second commit to track MySQL errors	8 years ago
René Cannaò	b5d8c4d3ba	Added table mysql_query_rules_fast_routing ... This table is responsible for performing routing for very specific use case scenarios, like having tens of thousands of schemas	8 years ago
René Cannaò	f4a0c4a2b5	ProxySQL reads GTID information from proxysql_mysqlbinlog ... This is the first commit to pull data from proxysql_mysqlbinlog. It is still in Alpha phase, as it is missing a log of important logics, like error handling, retry mechanism, timeouts, etc	8 years ago
René Cannaò	ea9fa03ab1	Added table stats.stats_mysql_prepared_statements_info ... This new table exports global metrics on prepared statements	8 years ago
René Cannaò	785e66c928	Display interfaces in HTTP	9 years ago
René Cannaò	6fbd6f52ec	First implementation of web interface	9 years ago
René Cannaò	7dbcbc3e7a	Bug fix for #1188 ... * all commands expected in Admin are serialized * Cluster get mysql servers information directly from MyHGM * Before applying MySQL servers changes, Cluster fetch again the checksum to verify it didn't change	9 years ago
René Cannaò	46193a634c	Further development of SQLite3 Server ... Added variable for SQLite3 Server Added new command: * LOAD SQLITESERVER VARIABLES FROM MEMORY / TO RUNTIME * LOAD SQLITESERVER VARIABLES FROM DISK / TO MEMORY * SAVE SQLITESERVER VARIABLES FROM RUNTIME / TO MEMORY * SAVE SQLITESERVER VARIABLES FROM MEMORY / TO DISK Connections to SQLite3 Server uses the same MySQL users in `mysql_users` Fixed minor issues related to ClickHouse Serve Fixed also some Makefile errors	9 years ago
René Cannaò	0b0e6198d2	Added ClickHouse variables ... Set dynmic hostname/port New commands: * LOAD CLICKHOUSE VARIABLES FROM MEMORY / TO RUNTIME * LOAD CLICKHOUSE VARIABLES FROM DISK / TO MEMORY * SAVE CLICKHOUSE VARIABLES FROM RUNTIME / TO MEMORY * SAVE CLICKHOUSE VARIABLES FROM MEMORY / TO DISK	9 years ago
René Cannaò	62898623df	Add optional support for ClickHouse	9 years ago
René Cannaò	a703a9172c	Further implementation for ClickHouse ... Added --clickhouse-server variable Added Admin's table: * clickhouse_users * runtime_clickhouse_users * disk.clickhouse_users Added commands to handle ClickHouse users Extended number of commmands supported for ClickHouse	9 years ago
René Cannaò	e3c4cb9964	A lot of changes all related to Cluster ... In `MySQL_Cluster` class added functions to sync with remote node, like: - `pull_mysql_query_rules_from_peer()` - `pull_mysql_servers_from_peer()` - `pull_mysql_users_from_peer()` - `pull_proxysql_servers_from_peer()` Added 8 new global variables in Admin. 4 variables determine after how many different checks the remote configuration will be synced: - cluster_mysql_query_rules_diffs_before_sync - cluster_mysql_servers_diffs_before_sync - cluster_mysql_users_diffs_before_sync - cluster_proxysql_servers_diffs_before_sync 4 variables determine if after a remote sync the changes need to be written to disk: - cluster_mysql_query_rules_save_to_disk - cluster_mysql_servers_save_to_disk - cluster_mysql_users_save_to_disk - cluster_proxysql_servers_save_to_disk Table `proxysql_servers` is now automatically loaded from disk to memory and into runtime at startup. Added new Admin's command `LOAD PROXYSQL SERVERS FROM CONFIG` to load `proxysql_servers` from config file to memory (not runtime). Internal structures with credentials in MySQL_Authentication moved from `unsorted_map` to `map` : this to ensure the right order when generating the checksum. Config file supports both `address` and `hostname` for `mysql_servers` and `proxysql_servers` , #1091 `ProxySQL_Admin::load_proxysql_servers_to_runtime()` now has a lock or no lock option, to avoid deadlock For now, Cluster module is quite verbose.	9 years ago
René Cannaò	2a76f95e31	Further enhancement on Cluster ... Extended class ProxySQL_Checksum_Value() in ProxySQL_Cluster module to support further metrics Implemeted `SELECT GLOBAL_CHECKSUM()` and relative tracking of global checksums Added variable `admin-cluster_check_status_frequency` to check peer's global status at regular intervals	9 years ago
René Cannaò	ee8110a515	Enhancement on Cluster solution ... This commit introduces: 2 new tables: * `runtime_checksums_values` : stores checksums of configurations in runtime. For now for `mysql_query_rules`, `mysql_servers` and `mysql_users` * `stats_proxysql_servers_checksums` : when clustering is enabled, it collects all metrics from `runtime_checksums_values` from all its peers 3 new global variables that defines it checksum needs to be generated during `LOAD ... TO RUNTIME` * `admin-checksum_mysql_query_rules` * `admin-checksum_mysql_servers` * `admin-checksum_mysql_users` ProxySQL Cluster connections now have timeouts: * 1 second timeout for CONNECT and WRITE * 60 seconds timeout for READ (useful for long poll)	9 years ago
René Cannaò	f583982ace	Drafting support for ProxySQL Cluster	9 years ago
René Cannaò	1414d4430f	Introduced new table stats_memory_metrics ... Here an example: ``` Admin> SELECT * FROM stats_memory_metrics ORDER BY variable_name; +----------------------+----------------+ | Variable_Name | Variable_Value | +----------------------+----------------+ | Auth_memory | 836 | | SQLite3_memory_bytes | 1636648 | | jemalloc_active | 3469312 | | jemalloc_allocated | 3055248 | | jemalloc_mapped | 14680064 | | jemalloc_metadata | 1480576 | | jemalloc_resident | 4542464 | | jemalloc_retained | 0 | | query_digest_memory | 459 | +----------------------+----------------+ 9 rows in set (0.00 sec) ```	9 years ago
René Cannaò	826495da07	Remove spinlock from Admin #977	9 years ago
René Cannaò	f550cf1340	Speeding up user management ... LOAD MYSQL USERS TO RUNTIME is not faster Introduced LOAD MYSQL USER username TO RUNTIME	9 years ago
René Cannaò	817cda3f12	New table stats_mysql_connection_pool_reset #753	9 years ago
René Cannaò	f183ea1651	Introduced command PROXYSQL FLUSH CONFIGDB #923 ... New command `PROXYSQL FLUSH CONFIGDB` allows to close and reopen configdb (`proxysql.db` on disk). Note: It is important that the (new) configdb is a valid one. A misconfigured configdb will cause undefined behaviors.	9 years ago
René Cannaò	f0f85a44fe	Introduced table stats.stats_mysql_users ... Example: ``` mysql> select * from stats_mysql_users; +----------+----------------------+--------------------------+ | username | frontend_connections | frontend_max_connections | +----------+----------------------+--------------------------+ | root | 0 | 10000 | | user1 | 0 | 10000 | | user2 | 1 | 10000 | +----------+----------------------+--------------------------+ 3 rows in set (0.00 sec) ```	9 years ago
René Cannaò	e369076bfb	Merge pull request #880 from Dan83/master ... Replace the spinlock on ProxySQL_Admin::wrlock with a pthread mutex or wrlock	9 years ago
Daniele	fda5470688	Replace the spinlock on ProxySQL_Admin::wrlock with a pthread mutex or wrlock	9 years ago
René Cannaò	ee5158a46e	Improvements for sqlite3 #856	9 years ago
René Cannaò	a6c53d0d12	Revert "Refactor header includes."	9 years ago
Maxim Bublis	30dbcc21a8	Refactor header includes. ... * reduce build time by 25%; * remove circular dependency on `cpp.h` and other headers. * fixes issue with `my_global.h` from mariadbclient which redefines `__attribute__`, that leads to broken STL and GCC intrinsics if `<algorithm>` wasn't included prior including this header; * fixes another issue with redefined `__attribute__` which leads to `_conn_exchange_t` not being aligned to cache line size; * removes `pthread_setstacksize` calls as `my_global.h` redefines `pthread_setstacksize` to be `pthread_dummy(0)` which led to stack size being never adjusted; * add missing include guards to some headers; * remove unused proxysql_hash.h header.	9 years ago
Maxim Bublis	043a321e55	Cleanup legacy codebase.	9 years ago
René Cannaò	b89de59f06	Changed transaction_persistent=1 by default #793 ... Supports upgrade from previous versions	10 years ago
René Cannaò	e1d636b717	Merge branch 'v1.2.3-691' into v1.2.3	10 years ago
René Cannaò	128ff3deda	Introduce new variable admin-hash_passwords #676 ... New variable admin-hash_passwords , default true When set, passwords in mysql_users are automatically hashed in runtime when running LOAD MYSQL USERS TO RUNTIME. For backward compatibility, passwords in mysql_users in memory and disk aren't automatically hashed. Although, they can be easily hashed running SAVE MYSQL USERS TO MEMORY (and eventually SAVE MYSQL USERS TO DISK) immediately after running LOAD MYSQL USERS TO RUNTIME	10 years ago
René Cannaò	99abb48c21	Implementation of runtime_mysql_users #691	10 years ago
Daniël van Eeden	71d4598ebd	Add missing vector include ... Related issue: https://github.com/sysown/proxysql/issues/671	10 years ago
René Cannaò	7dec343d82	Implemented LOAD SCHEDULER FROM CONFIG , #658 ... Command "LOAD SCHEDULER FROM CONFIG" is now implemented. Config file can also be read with --initial, --reload, and when a db file is not present yet. An example was added in src/proxysql.cfg	10 years ago
René Cannaò	dfba2422b7	Implementation of scheduler.active #649	10 years ago
René Cannaò	fe39671edf	Adding comment field per issue #643 ... Added: * scheduler.comment VARCHAR NOT NULL DEFAULT '' Like the other tables, scheduler supports online upgrade from previous version (without such field) at startup.	10 years ago
René Cannaò	13c9fe0ebb	Support extra syntax in Admin module ... - SELECT @@max_allowed_packet - SELECT @@variable - SHOW VARIABLES LIKE - SHOW VARIABLES WHERE	10 years ago
René Cannaò	f0416937cf	Added table runtime_global_variables	10 years ago