proxysql

Commit Graph

Author	SHA1	Message	Date
René Cannaò	ca929946f8	Delete on stats_mysql_query_events without LIMIT SQLite amalgamation doesn't support DELETE with ORDER BY . Thus the DELETE with ORDER BY was replaced by a SELECT to retrieve the maximum id, followed by a DELETE up to that id.	1 year ago
Javier Jaramago Fernández	5816a7d915	Fix compilation for Centos 7 removing 'auto' reference Fix compilation for GCC 4.8.5, compiler is not being able to either handle reference initialization using brace-enclosed initializer lists.	1 year ago
Javier Jaramago Fernández	7c4445a44f	Add Prometheus metric for 'MySQL_Logger' metric 'totalGetAllEventsTimeMicros'	1 year ago
Javier Jaramago Fernández	c729b07544	Fix typo for 'MySQL_Logger' metric 'totalGetAllEventsTimeMicros'	1 year ago
Javier Jaramago Fernández	0a2cc5e5fc	Add Prometheus metrics support for 'MySQL_Logger' module	1 year ago
Javier Jaramago Fernández	f907e2d67a	Fix typo in 'circularBufferEventsAddedCount' metric name	1 year ago
René Cannaò	1ff3974e30	Export MySQL_Logger metrics	1 year ago
Rahim Kanji	cea3ed9c2c	CentOS7 Ubuntu16 build fix	1 year ago
René Cannaò	124fb2895f	Adding errno into in-memory query logging New `errno` column in tables: - stats_mysql_query_events - history_mysql_query_events Logging into `stats_mysql_errors` also errors generated by ProxySQL (no backend) Added initial TAP test to be extended	2 years ago
René Cannaò	234d4e6d59	Drafting handling of errors in MySQL Events log 2	2 years ago
René Cannaò	9eeed3c45b	Drafting handling of errors in MySQL Events log	2 years ago
René Cannaò	ede0e7f268	MySQL_Logger::getAllMetrics() Added function MySQL_Logger::getAllMetrics() to export all metrics related to eventlogs	2 years ago
René Cannaò	8f0324b9af	MySQL_Event log buffering enhancements Added enhancements: * added mysql-eventslog_table_memory_size * liste columns names in insertMysqlEventsIntoDb() * added counters and metrics (not exported yet)	2 years ago
René Cannaò	56eb2115d0	Added variable eventslog_buffer_max_query_length In MySQL_Event copy constructor, queries are copied up to eventslog_buffer_max_query_length bytes.	2 years ago
René Cannaò	23e3244cc5	Implementation of processEvents() and DUMP EVENTSLOG commands MySQL_Logger::processEvents() is responsible for dumping mysql query events from the circular buffer into SQLite tables. DUMP EVENTSLOG commands trigger a call to processEvents(): * DUMP EVENTSLOG FROM BUFFER TO MEMORY: copies events to the in-memory SQLite table * DUMP EVENTSLOG FROM BUFFER TO DISK: copies events to the on-disk SQLite table * DUMP EVENTSLOG FROM BUFFER TO BOTH: copies events to both in-memory and on-disk SQLite tables simultaneously	2 years ago
René Cannaò	efb95ca87e	First implementation of insertMysqlEventsIntoDb() MySQL_Logger::insertMysqlEventsIntoDb() populate table `stats_mysql_query_events` using mysql query events in the circular buffer	2 years ago
René Cannaò	754f1e52f2	Reimplementation of MySQL_Logger_CircularBuffer::get_all_events() Accept a reference to a vector instead of returning a deque	2 years ago
René Cannaò	61c189b8c3	Reimplementation of the circular buffer for Query Logging	2 years ago
Rene Cannao	d92e57f83a	Renamed mysql-eventslog_memory_history_size to mysql-eventslog_buffer_history_size	2 years ago
Rene Cannao	6a2d52f610	Adding a circular buffer to MySQL_Logger	2 years ago
Miro Stauder	b1dc534757	refactor out pktH	3 years ago
René Cannaò	b546d51d90	Fix uninitialized variable	3 years ago
René Cannaò	fcb2acd67c	Added TAP test for logging last_insert_id Also modified the Logger to log last_insert_id in JSON format only if not 0	3 years ago
René Cannaò	c0745b42c2	Adding last_insert_id and last_gtid to MySQL_Logger	3 years ago
René Cannaò	3837de61e2	Various minor fixes Fixes: * minor memory leak when running CHECKSUM command * minor memory leak in load_mysql_servers_to_runtime() * not initialized json in MySQL_Event (valgrind doesn't like it) * not initialized variable MySQL_Thread::shutdown in the constructor	3 years ago
Javier Jaramago Fernández	2ecbad3244	Fix warning due to catching of polymorphic type by value	4 years ago
Javier Jaramago Fernández	a661d33391	Fix invalid digest value in 'eventslog' for 'COM_STMT_EXECUTE' 'COM_STMT_EXECUTE' invalidly reported the 'digest' value from the previous query when preceded by a 'COM_QUERY'.	4 years ago
Javier Jaramago Fernández	41ef582fc5	Add new field 'stmt_id' to eventslog for 'STMT_PREPARE\|STMT_EXECUTE' events	4 years ago
Javier Jaramago Fernández	f993402c62	Fix crash when failed 'connection_reset' is tried to be logged	4 years ago
René Cannaò	aceeaf79ed	Disabled prometheus metrics dump , see #3571 This commit also fixes some minor memory corruptions	5 years ago
Rene Cannao	41b173a83c	Fixing a lot of compiler warnings	6 years ago
René Cannaò	e062f84c3e	Merge pull request #2399 from val214/v2.0.9-mem-leaks [WIP] V2.0.9 mem leaks fixes	6 years ago
peter	6bb057fbda	Fix add a mutex lock twice in the process of function call(log_audit_entry() -> write() -> write_auth()) Bug description: Since fix write event log and audit log coredump(#2427)， i think i left out a code that was not processed, resulting in coredump again in the program of proxysql , when i confige these parameters of auditlog_filename and auditlog_filesize in proxysql.cnf file or i set these parameters in proxysql-admin command line. the coredump contest: (gdb) bt #0 0x00002b224e9981f7 in raise () from /lib64/libc.so.6 #1 0x00002b224e9998e8 in abort () from /lib64/libc.so.6 #2 0x00002b224e991266 in __assert_fail_base () from /lib64/libc.so.6 #3 0x00002b224e991312 in __assert_fail () from /lib64/libc.so.6 #4 0x00000000005b2dc3 in main (argc=5, argv=0x7ffea714fd48) at main.cpp:1664 the proxysql.log contest: 2019-12-06 11:03:07 main.cpp:1659:main(): [ERROR] Watchdog: 8 threads missed a heartbeat 2019-12-06 11:03:10 main.cpp:1659:main(): [ERROR] Watchdog: 8 threads missed a heartbeat 2019-12-06 11:03:13 main.cpp:1659:main(): [ERROR] Watchdog: 8 threads missed a heartbeat 2019-12-06 11:03:16 main.cpp:1659:main(): [ERROR] Watchdog: 8 threads missed a heartbeat 2019-12-06 11:03:19 main.cpp:1659:main(): [ERROR] Watchdog: 8 threads missed a heartbeat 2019-12-06 11:03:22 main.cpp:1659:main(): [ERROR] Watchdog: 8 threads missed a heartbeat 2019-12-06 11:03:25 main.cpp:1659:main(): [ERROR] Watchdog: 8 threads missed a heartbeat 2019-12-06 11:03:28 main.cpp:1659:main(): [ERROR] Watchdog: 8 threads missed a heartbeat 2019-12-06 11:03:31 main.cpp:1659:main(): [ERROR] Watchdog: 8 threads missed a heartbeat 2019-12-06 11:03:34 main.cpp:1659:main(): [ERROR] Watchdog: 8 threads missed a heartbeat 2019-12-06 11:03:34 main.cpp:1663:main(): [ERROR] Watchdog: reached 10 missed heartbeats. Aborting! proxysql: main.cpp:1664: int main(int, const char**): Assertion `0' failed. Error: signal 6: proxysql(_Z13crash_handleri+0x25)[0x5b607c] /lib64/libc.so.6(+0x35270)[0x2b224e998270] /lib64/libc.so.6(gsignal+0x37)[0x2b224e9981f7] /lib64/libc.so.6(abort+0x148)[0x2b224e9998e8] /lib64/libc.so.6(+0x2e266)[0x2b224e991266] /lib64/libc.so.6(+0x2e312)[0x2b224e991312] proxysql(main+0xe3a)[0x5b2dc3] /lib64/libc.so.6(__libc_start_main+0xf5)[0x2b224e984c05] proxysql[0x5adaa5] 2019-12-06 11:03:34 main.cpp:1396:ProxySQL_daemonize_phase3(): [ERROR] ProxySQL crashed. Restarting! 2019-12-06 11:03:34 main.cpp:1397:ProxySQL_daemonize_phase3(): [INFO] ProxySQL version _DEBUG Impact of this fix： I analyzed the call relationship of write_auth function of MySQL_Logger.cpp file in the whole project of proxysql with the source insight ide. I found write_auth function that was only called by the write function of MySQL_Logger.cpp file in the whole project of proxysql, and this write function was only called by log_audit_entry function or log_request function of MySQL_Logger.cpp file in the whole project of proxysql. However, i've added a mutex lock separately in log_audit_entry function and log_request function. And so i think it should delete the code for the wrlock function call in write_auth function, and it doesn't have impact for this time fix. Bux fix: Fix add a mutex lock twice in the process of function call(log_audit_entry() -> write() -> write_auth())	7 years ago
bskllzh	d71d01cb63	Fix write event log and audit log coredump(the events.logfile&audit.logfile null pointer) Bug description: When i make a benchmark test with a sysbench(version: 1.0.17) in proxysql 2.0.8 + mysql 5.7.26, i get the follow error info: Error: signal 11: proxysql(_Z13crash_handleri+0x25)[0x5b607c] /lib64/libc.so.6(+0x35270)[0x2b673de34270] /lib64/libstdc++.so.6(_ZNSo6sentryC2ERSo+0xf)[0x2b673d67521f] /lib64/libstdc++.so.6(_ZNSo5writeEPKcl+0x31)[0x2b673d675621] proxysql(_ZN11MySQL_Event20write_query_format_1EPSt13basic_fstreamIcSt11char_traitsIcEE+0x27d)[0x6e2d81] proxysql(_ZN11MySQL_Event5writeEPSt13basic_fstreamIcSt11char_traitsIcEEP13MySQL_Session+0x79)[0x6e19e9] proxysql(_ZN12MySQL_Logger11log_requestEP13MySQL_SessionP17MySQL_Data_Stream+0x710)[0x6e5416] proxysql(_ZN13MySQL_Session8LogQueryEP17MySQL_Data_Stream+0x6d)[0x654055] proxysql(_ZN13MySQL_Session10RequestEndEP17MySQL_Data_Stream+0x35)[0x6540d7] proxysql(_ZN13MySQL_Session7handlerEv+0x4397)[0x6459ed] proxysql(_ZN12MySQL_Thread20process_all_sessionsEv+0x830)[0x623180] proxysql(_ZN12MySQL_Thread3runEv+0x2cbe)[0x62208a] proxysql(_Z24mysql_worker_thread_funcPv+0xbb)[0x5afb05] /lib64/libpthread.so.0(+0x7e25)[0x2b673cda8e25] /lib64/libc.so.6(clone+0x6d)[0x2b673def734d] 2019-12-02 09:52:26 main.cpp:1396:ProxySQL_daemonize_phase3(): [ERROR] ProxySQL crashed. Restarting! I analyze the coredump with proxysql 2.0.8 debug version very carefully, and i found the variable of events.logfile is null pointer in log_request function when it call write function(me.write(events.logfile, sess)). The following is the coredump（write (this=0x2b67402fd1f0, f=0x0, sess=0x2b67414a8c00)）: (gdb) bt #0 0x00002b673d67521f in std::ostream::sentry::sentry(std::ostream&) () from /lib64/libstdc++.so.6 #1 0x00002b673d675621 in std::ostream::write(char const, long) () from /lib64/libstdc++.so.6 #2 0x00000000006e2d81 in MySQL_Event::write_query_format_1 (this=0x2b67402fd1f0, f=0x0) at MySQL_Logger.cpp:265 #3 0x00000000006e19e9 in MySQL_Event::write (this=0x2b67402fd1f0, f=0x0, sess=0x2b67414a8c00) at MySQL_Logger.cpp:89 #4 0x00000000006e5416 in MySQL_Logger::log_request (this=0x2b673e419680, sess=0x2b67414a8c00, myds=0x2b6741346480) at MySQL_Logger.cpp:717 #5 0x0000000000654055 in MySQL_Session::LogQuery (this=0x2b67414a8c00, myds=0x2b6741346480) at MySQL_Session.cpp:6691 #6 0x00000000006540d7 in MySQL_Session::RequestEnd (this=0x2b67414a8c00, myds=0x2b6741346480) at MySQL_Session.cpp:6711 #7 0x00000000006459ed in MySQL_Session::handler (this=0x2b67414a8c00) at MySQL_Session.cpp:4086 #8 0x0000000000623180 in MySQL_Thread::process_all_sessions (this=0x2b6741301000) at MySQL_Thread.cpp:4392 #9 0x000000000062208a in MySQL_Thread::run (this=0x2b6741301000) at MySQL_Thread.cpp:4125 #10 0x00000000005afb05 in mysql_worker_thread_func (arg=0x2b673e461f30) at main.cpp:648 #11 0x00002b673cda8e25 in start_thread () from /lib64/libpthread.so.0 #12 0x00002b673def734d in clone () from /lib64/libc.so.6 And then i add a line of code in log_request function of MySQL_Logger.cpp file to check up whether events.logfile is null pointer ，as follow: void MySQL_Logger::log_request(MySQL_Session sess, MySQL_Data_Stream myds) { if (events.enabled==false) return; if (events.logfile==NULL) return; proxy_info("log_request: First: events.logfile is null %d\n", (events.logfile == NULL) ? 1 : 0); MySQL_Connection_userinfo ui=sess->client_myds->myconn->userinfo; And i make a benchmark test again with proxysql 2.0.8 debug version, the coredump of proxysql happen again, then i get the follow print info from proxysql.log [root@XXXXX proxysql]# grep -n -w 'log_request: First: events.logfile is null 1' proxysql.log 7358447:2019-12-02 09:52:26 MySQL_Logger.cpp:617:log_request(): [INFO] log_request: First: events.logfile is null 1 7358448:2019-12-02 09:52:26 MySQL_Logger.cpp:617:log_request(): [INFO] log_request: First: events.logfile is null 1 Bux fix: Fix write event log and audit log coredump, because the proxysql get the null pointer variable of events.logfile（audit.logfile）in a multithreaded environment.	7 years ago
Valentin Rakush	6a6873b342	Fix leaks in logger and admin	7 years ago
René Cannaò	ce4a4a2464	Code cleanup and speeding compile time	7 years ago
René Cannaò	45e701b54d	Code cleanup and speeding compile time	7 years ago
René Cannaò	80e92b144d	[WIP] Second commit for mysql firewall whitelist Improved also compiling time	7 years ago
René Cannaò	3ad899c758	Avoid crash in JSON library for invalid UTF8	7 years ago
René Cannaò	3b3def9f78	Fix unitialized file	7 years ago
René Cannaò	255d9247b8	Reduce locking on MySQL_Logger writes to disk	7 years ago
René Cannaò	0831740e45	Several enhancements on Query logger Enhancements: - added metrics rows_affected and rows_sent - added global variable mysql-eventslog_default_log : if 1 , logging is enabled for every query unless explicitly disabled in mysql_query_rules.log . Default is 0 - added global variable mysql-eventslog_format : default is 1 (legacy format). A value of 2 enables logging in JSON format. Issue #871 Changing value at runtime causes the current file to be closed and a new one created - fixed logging for prepared statements: till 2.0.5 only some percentage of prepared statements was correctly logged Extended tables stats_mysql_query_digest and stats_mysql_query_digest_reset to also include sum_rows_affected and sum_rows_sent Extended `eventslog_reader_sample.cpp` to support the new enhancements	7 years ago
René Cannaò	37d79ad23a	Fixed compiler warning: ISO C++ forbids converting a string constant to char pointer	7 years ago
René Cannaò	7ea33a1312	Completing initial implementation of Audit module	7 years ago
René Cannaò	9e2c1fdda1	Draft implementation of audit log	7 years ago
Nick Vyzas	28a9e3f984	Fixes - "Event log file is overwritten on ProxySQL service restart" #1201	9 years ago
René Cannaò	5e05348b22	Remove spinlock from MySQL Logger #977	9 years ago
René Cannaò	a6c53d0d12	Revert "Refactor header includes."	10 years ago
Maxim Bublis	30dbcc21a8	Refactor header includes. * reduce build time by 25%; * remove circular dependency on `cpp.h` and other headers. * fixes issue with `my_global.h` from mariadbclient which redefines `__attribute__`, that leads to broken STL and GCC intrinsics if `<algorithm>` wasn't included prior including this header; * fixes another issue with redefined `__attribute__` which leads to `_conn_exchange_t` not being aligned to cache line size; * removes `pthread_setstacksize` calls as `my_global.h` redefines `pthread_setstacksize` to be `pthread_dummy(0)` which led to stack size being never adjusted; * add missing include guards to some headers; * remove unused proxysql_hash.h header.	10 years ago
Maxim Bublis	043a321e55	Cleanup legacy codebase.	10 years ago

1 2

64 Commits (6a34f03be74114f75f7793f4903ffb025fb51fe2)