mirror of https://github.com/sysown/proxysql
You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
127 lines
6.1 KiB
127 lines
6.1 KiB
Monitor Module
|
|
==============
|
|
|
|
This section focus on Monitor v1.2.1, as it introduces multiple improved compared to v1.2.0
|
|
|
|
|
|
Variables removed as unused or deprecated:
|
|
* mysql-monitor_query_variables
|
|
* mysql-monitor_query_status
|
|
* mysql-monitor_timer_cached
|
|
|
|
Variables currently not in use:
|
|
* mysql-monitor_query_interval
|
|
* mysql-monitor_query_timeout
|
|
|
|
Overview
|
|
--------
|
|
Monitor Module is responsible for a series of check against the backends.
|
|
It currently supports 4 types of checks:
|
|
* **connect** : it connects to all the backends, and success/failure is logged in table `mysql_server_connect_log`;
|
|
* **ping** : it pings to all the backends, and success/failure is logged in table `mysql_server_ping_log` . In case of `mysql-monitor_ping_max_failures` missed heartbeat, sends a signal to MySQL_Hostgroups_Manager to kill all connections;
|
|
* **replication lag** : it checks `Seconds_Behind_Master` to all backends configured with `max_replication_lag` greater than 0, and check is logged in table `mysql_server_replication_lag_log`. If `Seconds_Behind_Master` > `max_replication_lag` the server is shunned until `Seconds_Behind_Master` < `max_replication_lag` ;
|
|
* **read only** : it checks `read_only` for all hosts in the hostgroups in table `mysql_replication_hostgroups`, and check is logged in table `mysql_server_read_only_log` . If `read_only=1` the host is copied/moved to the `reader_hostgroup`, while if `read_only=0` the host is copied/moved to the `writer_hostgroup` .
|
|
|
|
|
|
Variables
|
|
=========
|
|
General variables:
|
|
* *mysql-monitor_username*
|
|
|
|
Specifies the username that the Monitor module will use to connect to the backend. The user needs only `USAGE` privileges to connect, ping and check `read_only`. The user needs also `REPLICATION CLIENT` if it needs to monitor replication lag.
|
|
|
|
* *mysql-monitor_password*
|
|
|
|
Password for user *mysql-monitor_username*
|
|
|
|
* *mysql-monitor_enabled*
|
|
|
|
It enables or disables MySQL Monitor. Since MySQL Monitor can interfere with changed applied directly on the Admin interface, this variable allows to temporary disable it.
|
|
|
|
Connect variables:
|
|
* *mysql-monitor_connect_interval*
|
|
|
|
How frequently a connect check is performed, in milliseconds.
|
|
|
|
* *mysql-monitor_connect_timeout*
|
|
|
|
Connection timeout in milliseconds. The current implementation rounds this value to an integer number of seconds less or equal to the original interval, with 1 second as minimum. This lazy rouding is done because SSL connections are blocking calls.
|
|
|
|
Ping variables:
|
|
* *mysql-monitor_ping_interval*
|
|
|
|
How frequently a ping check is performed, in milliseconds.
|
|
|
|
* *mysql-monitor_ping_timeout*
|
|
|
|
Ping timeout in milliseconds.
|
|
|
|
* *mysql-monitor_ping_max_failures*
|
|
|
|
If a host misses *mysql-monitor_ping_max_failures* pings in a row, MySQL_Monitor informs MySQL_Hostgroup_Manager that the node is unreacheable and that should immediately kill all connections.
|
|
It is important to note that in case a connection to the backend is not available, MySQL_Monitor will first try to connect in order to ping, therefore the time to detect a node down could be one of the two:
|
|
* *mysql-monitor_ping_max_failures* * *mysql-monitor_connect_timeout*
|
|
* *mysql-monitor_ping_max_failures* * *mysql-monitor_ping_timeout*
|
|
|
|
Read only variables:
|
|
|
|
* *mysql-monitor_read_only_interval*
|
|
|
|
How frequently a read only check is performed, in milliseconds.
|
|
|
|
* *mysql-monitor_read_only_timeout*
|
|
|
|
Read only check timeout in milliseconds.
|
|
|
|
* *mysql-monitor_writer_is_also_reader*
|
|
|
|
When a node change its `read_only` value from 1 to 0, this variable determines if the node should be present in both hostgroups or not:
|
|
* *false* : node will be moved in `writer_hostgroup` and removed from `reader_hostgroup`
|
|
* *true* : node will be copied in `writer_hostgroup` and stay also in `reader_hostgroup`
|
|
|
|
Replication lag variables:
|
|
|
|
* *mysql-monitor_replication_lag_interval*
|
|
|
|
How frequently a replication lag check is performed, in milliseconds.
|
|
|
|
* *mysql-monitor_replication_lag_timeout*
|
|
|
|
Replication lag check timeout in milliseconds.
|
|
|
|
Other variables:
|
|
|
|
* *mysql-monitor_history*
|
|
|
|
To prevent that log tables grow without limit, Monitor Module will automatically purge records older than *mysql-monitor_history* milliseconds. Since ping checks relies on history table to determine if a node is missing heartbeats, the value of *mysql-monitor_history* is automatically adjusted to the follows if less than it:
|
|
* (*mysql-monitor_ping_max_failures* + 1 ) * *mysql-monitor_ping_timeout*
|
|
|
|
|
|
Main Threads
|
|
============
|
|
The Monitor Module has several internal threads. There are currently 5 main threads:
|
|
* Monitor: master thread, responsible to start and coordinate all the others
|
|
* monitor_connect_thread: main thread and scheduler for the connect checks
|
|
* monitor_ping_thread: main thread and scheduler for the ping checks
|
|
* monitor_read_only_thread: main thread and scheduler for the read only checks
|
|
* monitor_replication_lag_thread: main thread and scheduler for the replication lag checks
|
|
Up to version v1.2.0 the above threads but *Monitor* were also responsible to perform the checks
|
|
|
|
Thread Pool
|
|
===========
|
|
The implementation in v1.2.0 has a limitation with SSL implementation: with SSL, `connect()` is a blocking call, causing the threads to stall while performing the connect phase.
|
|
Version v1.2.1 tries to overcome this limitation with a new implementation. Now:
|
|
* *Monitor* initializes a Thread Pool of workers and creates a queue;
|
|
* *monitor_connect_thread*, *monitor_ping_thread*, *monitor_read_only_thread* and *monitor_replication_lag_thread* are producers that generate tasks and sent them to the workers using the queue;
|
|
* the workers process the tasks and perform the requires actions;
|
|
* if *Monitor* detects that the queue is growing too fast, it creates new temporary worker threads
|
|
|
|
Connection purging
|
|
==================
|
|
Monitor implements its own connection pool. Connections that are alive for more than 3 * `mysql_thread___monitor_ping_interval` milliseconds are automatically purged
|
|
|
|
wait_timeout
|
|
------------
|
|
|
|
To prevent that backends terminated connections, Monitor module automatically configures `wait_timeout` = `mysql_thread___monitor_ping_interval` * 10
|