proxysql

Commit Graph

Author	SHA1	Message	Date
René Cannaò	449f4eb659	Adding debug_filters	6 years ago
René Cannaò	03bf53aec8	Preparing the cluster module to sync variables Added admin variables: * cluster_mysql_variables_save_to_disk * cluster_admin_variables_save_to_disk * cluster_mysql_variables_diffs_before_sync * cluster_admin_variables_diffs_before_sync When flush_admin_variables___database_to_runtime() is executed, flush_admin_variables___runtime_to_database() is also called to update admin variable checksum. When flush_mysql_variables___database_to_runtime() is executed, flush_mysql_variables___runtime_to_database() is also called to update mysql variable checksum. flush_mysql_variables___runtime_to_database() was also modified to avoid locking twice.	6 years ago
Javier Jaramago Fernández	225c253607	Added new 'prometheus_memory_metrics_interval' admin variable	6 years ago
Javier Jaramago Fernández	a4e4e57343	Moved all 'metrics_maps' to source files	6 years ago
Javier Jaramago Fernández	864535d3f9	Added support for GCC 4.8 and removed per-module metrics initialization functions - GCC 4.8 and 5 have issues with uniform initialization syntax, for this reason, all it's uses have been replaced with assignements. - Per-module metrics initialization functions are no longer required, they have been replaced with the new generic ones.	6 years ago
Javier Jaramago Fernández	d07c95b2e9	Added metrics help and replaced metrics impl with new array based one for 'proxysql_admin' module	6 years ago
Javier Jaramago Fernández	c6f75ea454	Added more metrics, fixed some invalid updated ones and standarized the 'update_modules_metrics' callback	6 years ago
Javier Jaramago Fernández	472207b951	Added new 'stmt', 'memory' prometheus metrics and callback with new metrics updates	6 years ago
Javier Jaramago Fernández	621332d423	Added new prometheus exposer to ProxySQL_Admin	6 years ago
René Cannaò	9c2f0bf3ac	Adding tests to benchmark multiple lookups in mysql_query_rules_fast_routing	6 years ago
Valentin Rakush	f0687f3688	Implement restapi functionality - restapi_routes table in admin module - calling python script (json validation, error handling) - scripts: metrics, export_users - tests adjusted	6 years ago
René Cannaò	b59fad2d42	Merge pull request #2413 from val214/v2.0.9-select-config-file V2.0.9 select config file [FR]	6 years ago
René Cannaò	e9dc7a9b60	Implementation of SQLI fingerprints whitelist libsqlinjection generates a lot of false positives. This commit introduces a new table: mysql_firewall_whitelist_sqli_fingerprints This table can list fingerprints generated by libsqlinjection: if the fingerprint is listed in this table, proxysql will consider it as a false positive. This commit also enables SQLi algorithm only if the query is not already explicitly whitelisted.	6 years ago
Valentin Rakush	6bd6c0a34b	Move config file read/write functions to separate file.	6 years ago
Valentin Rakush	52b5ee9486	export proxysql configuration to resultset	6 years ago
Valentin Rakush	9430cabee3	SELECT CONFIG FILE implementation	6 years ago
René Cannaò	9afe6824ff	Merge pull request #2407 from sysown/v2.0.9-libmicrohttpd-0.9.68 V2.0.9 libmicrohttpd 0.9.68	6 years ago
René Cannaò	485d313514	Further development of firewall whitelist. * introduced 2 new variables: - mysql-firewall_whitelist_enabled : defines if the firewall with whitelist algorithm is enabled or not - firewall_whitelist_errormsg : error message returned to the client if an error message is not already set * Query_Processor returns firewall_whitelist_mode to the calling MySQL_Session, but the logging on error log currently happens within the Query_Processor itself * if mysql-firewall_whitelist_enabled is turned on and a user it is not found in mysql_firewall_whitelist_users , the default behavior is that query is rejected * lookup on mysql_firewall_whitelist_users happens for username@ip (specific IP) and username@'' (all IPs) . Match on network subnet is currently not implemented * implemented SAVE MYSQL FIREWALL FROM RUNTIME * fixed implementation of SAVE MYSQL FIREWALL TO DISK * runtime configuration can be viewed in runtime_mysql_firewall_whitelist_rules and runtime_mysql_firewall_whitelist_users	6 years ago
René Cannaò	1673249da4	Adding libhttpserver , and a simple "Hello World" libhttpserver will take care of REST API. The feature is currently disabled.	6 years ago
René Cannaò	80e92b144d	[WIP] Second commit for mysql firewall whitelist Improved also compiling time	6 years ago
René Cannaò	5023f2d430	Expending tests for firewall whitelist On a normal laptop, the lookup for one entry in a hash table with 39k keys and a total of 980k entries, takes around 1.1us	6 years ago
René Cannaò	7f6d64c3db	First commit for firewall whitelist	6 years ago
René Cannaò	6bed334db1	stats_history.history_mysql_query_digest #2368 Implementation of new table stats_history.history_mysql_query_digest #2368 Implemented command `SAVE MYSQL DIGEST TO DISK`. Save to disk with really minimal memory footprint compared to querying `stats_mysql_query_digest` or `stats_mysql_query_digest_reset` . Implemented new admin variable `admin-stats_mysql_query_digest_to_disk` to automatically save to disk.	7 years ago
René Cannaò	a0ea0c118c	V2.0.8 fast routing (#2309 ) * Implementation of unit tests related to mysql_query_rules_fast_routing * Implementation of rules_fast_routing with khash * Adding memory usage for new fast routing algo	7 years ago
Valentin Rakush	757684c6d9	[FR] Group Replication Timeout count	7 years ago
René Cannaò	2fec51c0a9	A lot of changes to make query digest map faster Extended SQLite3_result to support a mutex for concurrent rows insert. Extensively rewritten get_query_digests() and get_query_digests_reset() to support multiple threads, and concurrently generate a resultset. Function get_query_digests_reset() defer the deletion of objects from query digest map after generating the resultset for Admin. This drastically reduce the amount of time that query digest map is locked. Rewritten get_query_digests_total_size() to make it multi-threaded. Added 3 new functions for purging: - purge_query_digests() : wrapper - purge_query_digests_sync() : synchronous purge of query digest map, either single-threaded or multi-threaded - purge_query_digests_async() : asynchronous purge of query digest map, single-threaded only If table `stats_mysql_query_digest_reset` is queried but not `stats_mysql_query_digest`, only the first one is populated. If table `stats_mysql_query_digest_reset` is queried and also `stats_mysql_query_digest`, the second one is populated as a copy from the first. Implemented new SQL `TRUNCATE [TABLE]` for `[stats.]stats_mysql_query_digest[_reset]`: this will purge the query digest map as fast as possible without creating any table in SQLite. For testing/benchmark: - modified ProxySQL_Test___GenerateRandomQueryInDigestTable() to generate more random data and randomly switch `mysql_thread___query_digests_normalize_digest_text` Extended command `PROXYSQLTEST` with the following tests: - 1 : generates entries in digest map - 2 : gets all the entries from the digest map, but without writing to DB, with multiple threads - 3 : gets all the entries from the digest map and reset, but without writing to DB, with multiple threads - 4 : purges the digest map, synchronously, in single thread - 5 : purges the digest map, synchronously, in multiple threads - 6 : purges the digest map, asynchronously, in single thread (this seems the faster) Replaced some sprintf() with new function my_itoa() . Modified class QP_query_digest_stats() to include username, schemaname and client_address if no longer than 24 bytes. This makes storage requirement for QP_query_digest_stats() bigger, but reduces a lot of calls to memory management. Modified QP_query_digest_stats::get_row() to drastically reduce the number of malloc/free and string copy. Overall, this commit has the following effects: - it increases the memory usage for query digest map. On test workload, this results in 10% more memory usage - drastically reduces the calls to memory management - populating `stats_mysql_query_digest_reset` is 3 times faster - the amount of time in which query digest map is locked is reduced by 95%	7 years ago
René Cannaò	b12caa1d8c	Adding support for automated testing of Galera Using SQLite3 Server as backend	7 years ago
René Cannaò	58ff4ff892	Merge branch 'v2.0.4' into v2.0.1-aurora	7 years ago
René Cannaò	495c933624	Adding table stats.stats_mysql_free_connections This table exports metrics about connections in connection pool	7 years ago
René Cannaò	528d8cac38	Improved debugging Added new command `PROXYSQL INTERNAL SESSION` that clients can execute to receive internal information about their own connection in JSON format. Added JSON library. Recompiled SQLite3 to support JSON. Added new column `extended_info` in `stats_mysql_processlist`. Added new mysql variable `mysql-show_processlist_extended` that determine the content of `stats_mysql_processlist.extended_info`: - 0 : no info - 1 : JSON format - 2 : JSON format with pretty printing	7 years ago
René Cannaò	2e20854684	Second commit (not complete) to support Aurora	7 years ago
René Cannaò	b82a1c18ac	Merge pull request #1857 from sysown/v2.0.0-digestSplit V2.0.0 digest split	7 years ago
René Cannaò	782aae072e	Implement automatic vacuum of stats schema #1011	7 years ago
René Cannaò	47bf5faff7	Second large commit to support LDAP Functions add() and lookup() in MySQL_LDAP_Authentication have support for backend_username. Added mysql_ldap_mapping table. Created Admin::init_ldap() to be called after LDAP initialization. Added better LDAP caching. LOAD LDAP MAPPING TO RUNTIME cleans part of the cache (association to backend user). All queries will have a comment "proxysql-ldap-user=%s" to track original user	8 years ago
René Cannaò	d96108123d	Scheleton for LDAP support	8 years ago
René Cannaò	601f9b0862	Second commit to track MySQL errors	8 years ago
René Cannaò	b5d8c4d3ba	Added table mysql_query_rules_fast_routing This table is responsible for performing routing for very specific use case scenarios, like having tens of thousands of schemas	8 years ago
René Cannaò	f4a0c4a2b5	ProxySQL reads GTID information from proxysql_mysqlbinlog This is the first commit to pull data from proxysql_mysqlbinlog. It is still in Alpha phase, as it is missing a log of important logics, like error handling, retry mechanism, timeouts, etc	8 years ago
René Cannaò	ea9fa03ab1	Added table stats.stats_mysql_prepared_statements_info This new table exports global metrics on prepared statements	8 years ago
René Cannaò	785e66c928	Display interfaces in HTTP	8 years ago
René Cannaò	6fbd6f52ec	First implementation of web interface	9 years ago
René Cannaò	7dbcbc3e7a	Bug fix for #1188 * all commands expected in Admin are serialized * Cluster get mysql servers information directly from MyHGM * Before applying MySQL servers changes, Cluster fetch again the checksum to verify it didn't change	9 years ago
René Cannaò	46193a634c	Further development of SQLite3 Server Added variable for SQLite3 Server Added new command: * LOAD SQLITESERVER VARIABLES FROM MEMORY / TO RUNTIME * LOAD SQLITESERVER VARIABLES FROM DISK / TO MEMORY * SAVE SQLITESERVER VARIABLES FROM RUNTIME / TO MEMORY * SAVE SQLITESERVER VARIABLES FROM MEMORY / TO DISK Connections to SQLite3 Server uses the same MySQL users in `mysql_users` Fixed minor issues related to ClickHouse Serve Fixed also some Makefile errors	9 years ago
René Cannaò	0b0e6198d2	Added ClickHouse variables Set dynmic hostname/port New commands: * LOAD CLICKHOUSE VARIABLES FROM MEMORY / TO RUNTIME * LOAD CLICKHOUSE VARIABLES FROM DISK / TO MEMORY * SAVE CLICKHOUSE VARIABLES FROM RUNTIME / TO MEMORY * SAVE CLICKHOUSE VARIABLES FROM MEMORY / TO DISK	9 years ago
René Cannaò	62898623df	Add optional support for ClickHouse	9 years ago
René Cannaò	a703a9172c	Further implementation for ClickHouse Added --clickhouse-server variable Added Admin's table: * clickhouse_users * runtime_clickhouse_users * disk.clickhouse_users Added commands to handle ClickHouse users Extended number of commmands supported for ClickHouse	9 years ago
René Cannaò	e3c4cb9964	A lot of changes all related to Cluster In `MySQL_Cluster` class added functions to sync with remote node, like: - `pull_mysql_query_rules_from_peer()` - `pull_mysql_servers_from_peer()` - `pull_mysql_users_from_peer()` - `pull_proxysql_servers_from_peer()` Added 8 new global variables in Admin. 4 variables determine after how many different checks the remote configuration will be synced: - cluster_mysql_query_rules_diffs_before_sync - cluster_mysql_servers_diffs_before_sync - cluster_mysql_users_diffs_before_sync - cluster_proxysql_servers_diffs_before_sync 4 variables determine if after a remote sync the changes need to be written to disk: - cluster_mysql_query_rules_save_to_disk - cluster_mysql_servers_save_to_disk - cluster_mysql_users_save_to_disk - cluster_proxysql_servers_save_to_disk Table `proxysql_servers` is now automatically loaded from disk to memory and into runtime at startup. Added new Admin's command `LOAD PROXYSQL SERVERS FROM CONFIG` to load `proxysql_servers` from config file to memory (not runtime). Internal structures with credentials in MySQL_Authentication moved from `unsorted_map` to `map` : this to ensure the right order when generating the checksum. Config file supports both `address` and `hostname` for `mysql_servers` and `proxysql_servers` , #1091 `ProxySQL_Admin::load_proxysql_servers_to_runtime()` now has a lock or no lock option, to avoid deadlock For now, Cluster module is quite verbose.	9 years ago
René Cannaò	2a76f95e31	Further enhancement on Cluster Extended class ProxySQL_Checksum_Value() in ProxySQL_Cluster module to support further metrics Implemeted `SELECT GLOBAL_CHECKSUM()` and relative tracking of global checksums Added variable `admin-cluster_check_status_frequency` to check peer's global status at regular intervals	9 years ago
René Cannaò	ee8110a515	Enhancement on Cluster solution This commit introduces: 2 new tables: * `runtime_checksums_values` : stores checksums of configurations in runtime. For now for `mysql_query_rules`, `mysql_servers` and `mysql_users` * `stats_proxysql_servers_checksums` : when clustering is enabled, it collects all metrics from `runtime_checksums_values` from all its peers 3 new global variables that defines it checksum needs to be generated during `LOAD ... TO RUNTIME` * `admin-checksum_mysql_query_rules` * `admin-checksum_mysql_servers` * `admin-checksum_mysql_users` ProxySQL Cluster connections now have timeouts: * 1 second timeout for CONNECT and WRITE * 60 seconds timeout for READ (useful for long poll)	9 years ago
René Cannaò	f583982ace	Drafting support for ProxySQL Cluster	9 years ago

1 2 3

112 Commits (449f4eb65917d8939bf2e91cbf11684f475cee23)