From edb2f2626ee18d71a524319699ee051191b01972 Mon Sep 17 00:00:00 2001 From: Andrei Ismail Date: Fri, 21 Aug 2015 18:39:59 +0300 Subject: [PATCH 1/9] #351 Diagram with the 3 layers, explanation of the first 2 layers --- doc/configuration_system.md | 59 +++++++++++++++++++++++++++++++++++++ 1 file changed, 59 insertions(+) create mode 100644 doc/configuration_system.md diff --git a/doc/configuration_system.md b/doc/configuration_system.md new file mode 100644 index 000000000..8ad5efb8d --- /dev/null +++ b/doc/configuration_system.md @@ -0,0 +1,59 @@ +Configuration system +==================== + +ProxySQL has a simple to use configuration system suited to serve the following needs: +* allow easy automated updates to the configuration (this is because some ProxySQL users use it in larger setups with automated provisioning). There is a MySQL-compatible admin interface for this purpose +* allow as many configuration items as possible to be modified +at runtime, without restarting the daemon +* allow easy rollbacks of wrong configurations + +The 3 layers of the configuration system are described in the +picture below: + +``` ++-------------------------+ +| RUNTIME | ++-------------------------+ + /|\ | + | | + | | + | \|/ ++-------------------------+ +| MEMORY | ++-------------------------+ _ + /|\ | |\ + | | \ + | | \ + | \|/ \ ++-------------------------+ +-------------------------+ +| DISK | | CONFIG FILE | ++-------------------------+ +-------------------------+ + +``` + +__RUNTIME__ represents the in-memory data structures of ProxySQL used by the threads that are handling the requests. These contains the values of the global variables used, the list of backend servers grouped into hostgroups or the list of MySQL users that can connect to the proxy. Note that operators can never modify the contents of the __RUNTIME__ configuration section directly. They always have to go through the bottom layers. + +__MEMORY__ represents an in-memory SQLite3 database which is exposed to the outside via a MySQL-compatible interface. Users can connect with a MySQL client to this interface and query different tables and databases. The configuration tables +available through this interface are: +* mysql_servers -- the list of backend servers +* mysql_users -- the list of users and their credentials which can connect to ProxySQL. Note that ProxySQL will use these credentials to connect to the backend servers as well (TODO: check) +* mysql_query_rules -- the list of rules for routing traffic to the different backend servers. These rules can also cause a rewrite of the query, or caching of the result +* global_variables -- the list of global variables used throughout the proxy that can be tweaked at runtime. Examples of global variables: +``` +mysql> select * from global_variables limit 3; ++----------------------------------+----------------+ +| variable_name | variable_value | ++----------------------------------+----------------+ +| mysql-connect_retries_on_failure | 5 | +| mysql-connect_retries_delay | 1 | +| mysql-connect_timeout_server_max | 10000 | ++----------------------------------+----------------+ +``` +* mysql_collations -- the list of MySQL collations available for the proxy to work with. See [this](http://stackoverflow.com/questions/341273/what-does-character-set-and-collation-mean-exactly) StackOverflow answer for the difference between a collation and a charset. +* [only available in debug builds] debug_levels -- the list of types of debug statements that ProxySQL emits together with their verbosity levels. This allows us to easily configure at runtime what kind of statements we have in the log in order to debug different problems + +__DISK__ and __CONFIG FILE__ + +# Initial startup (--initial flag) for ProxySQL + +# Locations of config files \ No newline at end of file From f1fdc0a9dad17de4c91ee48e9d7c860603e00c2c Mon Sep 17 00:00:00 2001 From: Andrei Ismail Date: Fri, 21 Aug 2015 18:52:42 +0300 Subject: [PATCH 2/9] #351 Explaining disk and config file. Completing the line-up of sections --- doc/configuration_system.md | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/doc/configuration_system.md b/doc/configuration_system.md index 8ad5efb8d..7d4fb2d80 100644 --- a/doc/configuration_system.md +++ b/doc/configuration_system.md @@ -54,6 +54,26 @@ mysql> select * from global_variables limit 3; __DISK__ and __CONFIG FILE__ -# Initial startup (--initial flag) for ProxySQL +__DISK__ represents an on-disk SQLite3 database, with the default location at +`/tmp/proxysql.db`. Across restarts, the in-memory configs that were not +persisted will be gone. __CONFIG__ file is the classical config file, and we'll +see the relationship between it and the other configuration layers in the next +section. + +In the following sections, we'll describe the lifecycle of each of these layers +for the basic operations that the daemon goes through: starting up for the +first time, starting up, restarting, shutting down, etc. + +# Initial startup (or --initial flag) + +# Startup + +# Soft restart (via angel process) + +# Hard restart + +# Modifying config at runtime + +# Moving config between layers # Locations of config files \ No newline at end of file From e61965cf764b6ae782e6c029ddfc18b3b3e72c4d Mon Sep 17 00:00:00 2001 From: Andrei Ismail Date: Mon, 24 Aug 2015 14:50:51 +0300 Subject: [PATCH 3/9] #351 Describe SQL commands and how to use them. Describe start-up procedure in terms of configuration. --- doc/configuration_system.md | 107 +++++++++++++++++++++++++++++++++--- 1 file changed, 100 insertions(+), 7 deletions(-) diff --git a/doc/configuration_system.md b/doc/configuration_system.md index 7d4fb2d80..22ad808dd 100644 --- a/doc/configuration_system.md +++ b/doc/configuration_system.md @@ -16,15 +16,15 @@ picture below: +-------------------------+ /|\ | | | - | | + [1] | [2] | | \|/ +-------------------------+ | MEMORY | +-------------------------+ _ - /|\ | |\ - | | \ - | | \ - | \|/ \ + /|\ | |\ + | | \ + [3] | [4] | \ [5] + | \|/ \ +-------------------------+ +-------------------------+ | DISK | | CONFIG FILE | +-------------------------+ +-------------------------+ @@ -64,9 +64,19 @@ In the following sections, we'll describe the lifecycle of each of these layers for the basic operations that the daemon goes through: starting up for the first time, starting up, restarting, shutting down, etc. +# Startup + +At a normal start-up, ProxySQL initializes its configuration from the persisted +in-memory database. This is found in the "datadir" (usually the current working directory in which ProxySQL is ran). The filename for the database is named "proxysql.db". So, disk gets loaded into memory and then propagated towards the +runtime configuration. + # Initial startup (or --initial flag) -# Startup +At the initial start-up (or start-up done with --initial flag, which resets the +database by renaming the old one), the major difference from normal start-up is that the memory and runtime configuration gets populated from the config file. + +After this is done, the configuration is also persisted to the disk database, +which will be used for the next restarts. # Soft restart (via angel process) @@ -74,6 +84,89 @@ first time, starting up, restarting, shutting down, etc. # Modifying config at runtime +Modifying the config at runtime is done through the MySQL admin port of ProxySQL. After connecting to it, we are presented with a MySQL-compatible interface for querying several ProxySQL-related tables: +```mysql +mysql> show tables; ++-------------------+ +| tables | ++-------------------+ +| mysql_servers | +| mysql_users | +| mysql_query_rules | +| global_variables | +| mysql_collations | +| debug_levels | ++-------------------+ +6 rows in set (0.01 sec) +``` + +Each such table has a well defined role in the admin interface: +- `mysql_servers` contains the list of backend servers for ProxySQL to connect to +- `mysql_users` contains the list of users with which to authenticate to ProxySQL and the backend servers +- `mysql_query_rules` contains the rules for caching, routing or rewriting the SQL queries that pass through the proxy +- `global_variables` contains both the MySQL variables and the admin variables in a single table +- `debug_levels` is only used in the debug build of ProxySQL + +These tables represent the middle layer (in-memory database) from the diagram above and can be manipulated using standard SQL queries. In order to move the configuration from this layer upwards or downwards, see the next section. + # Moving config between layers -# Locations of config files \ No newline at end of file +In order to move configuration between the three layers, there are a set of different admin commands available through the admin interface. Once you understand what each of the three layers means, the semantics should be quite obvious. Together with the explanation of each command, there is a number written next to it. The number corresponds to the arrow in the diagram from above. + +For handling MySQL users: +* [1] LOAD MYSQL USERS FROM MEMORY / LOAD MYSQL USERS TO RUNTIME + * loads MySQL users from the in-memory database to the runtime data structures +* [2] SAVE MYSQL USERS TO MEMORY / SAVE MYSQL USERS FROM RUNTIME + * persists the MySQL users from the runtime data structures to the in-memory database +* [3] LOAD MYSQL USERS TO MEMORY / LOAD MYSQL USERS FROM DISK + * loads MySQL users from the on-disk database to the in-memory database +* [4] SAVE MYSQL USERS FROM MEMORY / SAVE MYSQL USERS TO DISK + * persists the MySQL users from the in-memory database to the on-disk database +* [5] LOAD MYSQL USERS FROM CONFIG + * loads from the configuration file the users into the in-memory database + +For handling MySQL servers: +* [1] LOAD MYSQL SERVERS FROM MEMORY / LOAD MYSQL SERVERS TO RUNTIME + * loads MySQL servers from the in-memory database to the runtime data structures +* [2] SAVE MYSQL SERVERS TO MEMORY / SAVE MYSQL SERVERS FROM RUNTIME + * persists the MySQL servers from the runtime data structures to the in-memory database +* [3] LOAD MYSQL SERVERS TO MEMORY / LOAD MYSQL SERVERS FROM DISK + * loads MySQL servers from the on-disk database to the in-memory database +* [4] SAVE MYSQL SERVERS FROM MEMORY / SAVE MYSQL SERVERS TO DISK + * persists the MySQL servers from the in-memory database to the on-disk database +* [5] LOAD MYSQL SERVERS FROM CONFIG + * loads from the configuration file the servers into the in-memory database + +For handling MySQL query rules: +* [1] LOAD MYSQL QUERY RULES FROM MEMORY / LOAD MYSQL QUERY RULES TO RUNTIME + * loads MySQL query rules from the in-memory database to the runtime data structures +* [2] SAVE MYSQL QUERY RULES TO MEMORY / SAVE MYSQL QUERY RULES FROM RUNTIME + * persists the MySQL query rules from the runtime data structures to the in-memory database +* [3] LOAD MYSQL QUERY RULES TO MEMORY / LOAD MYSQL QUERY RULES FROM DISK + * loads MySQL query rules from the on-disk database to the in-memory database +* [4] SAVE MYSQL QUERY RULES FROM MEMORY / SAVE MYSQL QUERY RULES TO DISK + * persists the MySQL query rules from the in-memory database to the on-disk database +* [5] LOAD MYSQL QUERY RULES FROM CONFIG + * loads from the configuration file the query rules into the in-memory database + +For handling MySQL variables: +* [1] LOAD MYSQL VARIABLES TO MEMORY / LOAD MYSQL VARIABLES FROM DISK + * loads MySQL variables from the on-disk database to the in-memory database +* [2] SAVE MYSQL VARIABLES FROM MEMORY / SAVE MYSQL VARIABLES TO DISK + * persists the MySQL variables from the in-memory database to the on-disk database +* [3] LOAD MYSQL VARIABLES FROM MEMORY / LOAD MYSQL VARIABLES TO RUNTIME + * loads MySQL variables from the in-memory database to the runtime data structures +* [4] SAVE MYSQL VARIABLES TO MEMORY / SAVE MYSQL VARIABLES FROM RUNTIME + * persists the MySQL variables from the runtime data structures to the in-memory database +* [5] LOAD MYSQL VARIABLES FROM CONFIG + * loads from the configuration file the variables into the in-memory database + +For handling admin variables: +* [1] LOAD ADMIN VARIABLES FROM MEMORY / LOAD ADMIN VARIABLES TO RUNTIME + * loads admin variables from the in-memory database to the runtime data structures +* [2] SAVE ADMIN VARIABLES TO MEMORY / SAVE ADMIN VARIABLES FROM RUNTIME + * persists the admin variables from the runtime data structures to the in-memory database +* [3] LOAD ADMIN VARIABLES TO MEMORY / LOAD ADMIN VARIABLES FROM DISK + * loads admin variables from the on-disk database to the in-memory database +* [4] SAVE ADMIN VARIABLES FROM MEMORY / SAVE ADMIN VARIABLES TO DISK + * persists the admin variables from the in-memory database to the on-disk database \ No newline at end of file From 92aefc3650cdb26a8c1b584e6ec032bece0c62e6 Mon Sep 17 00:00:00 2001 From: Andrei Ismail Date: Tue, 25 Aug 2015 00:06:21 +0300 Subject: [PATCH 4/9] #351 Explaining how the configuration is updated on soft/hard restart --- doc/configuration_system.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/doc/configuration_system.md b/doc/configuration_system.md index 22ad808dd..835435cf6 100644 --- a/doc/configuration_system.md +++ b/doc/configuration_system.md @@ -80,8 +80,12 @@ which will be used for the next restarts. # Soft restart (via angel process) +On soft restart, the configuration file isn't re-read, and the in-memory database is populated from the previously read version of the configuration file. + # Hard restart +On hard restart, the configuration file is re-read, because it's the same as killing the process and starting it again. + # Modifying config at runtime Modifying the config at runtime is done through the MySQL admin port of ProxySQL. After connecting to it, we are presented with a MySQL-compatible interface for querying several ProxySQL-related tables: From 98dd9608f8578a7dfe4dce0876a95dbce3dc1567 Mon Sep 17 00:00:00 2001 From: Andrei Ismail Date: Tue, 25 Aug 2015 00:06:52 +0300 Subject: [PATCH 5/9] #351 @renecannao confirmed that same credentials are used for frontend and backend servers --- doc/configuration_system.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/configuration_system.md b/doc/configuration_system.md index 835435cf6..540ae52fb 100644 --- a/doc/configuration_system.md +++ b/doc/configuration_system.md @@ -36,7 +36,7 @@ __RUNTIME__ represents the in-memory data structures of ProxySQL used by the thr __MEMORY__ represents an in-memory SQLite3 database which is exposed to the outside via a MySQL-compatible interface. Users can connect with a MySQL client to this interface and query different tables and databases. The configuration tables available through this interface are: * mysql_servers -- the list of backend servers -* mysql_users -- the list of users and their credentials which can connect to ProxySQL. Note that ProxySQL will use these credentials to connect to the backend servers as well (TODO: check) +* mysql_users -- the list of users and their credentials which can connect to ProxySQL. Note that ProxySQL will use these credentials to connect to the backend servers as well * mysql_query_rules -- the list of rules for routing traffic to the different backend servers. These rules can also cause a rewrite of the query, or caching of the result * global_variables -- the list of global variables used throughout the proxy that can be tweaked at runtime. Examples of global variables: ``` From 4d90d92e7c0235abbef9acc5b36614b670a1fea2 Mon Sep 17 00:00:00 2001 From: Andrei Ismail Date: Tue, 25 Aug 2015 00:25:26 +0300 Subject: [PATCH 6/9] #351 Added --reload explanation --- doc/configuration_system.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/doc/configuration_system.md b/doc/configuration_system.md index 540ae52fb..fb12244ee 100644 --- a/doc/configuration_system.md +++ b/doc/configuration_system.md @@ -70,13 +70,14 @@ At a normal start-up, ProxySQL initializes its configuration from the persisted in-memory database. This is found in the "datadir" (usually the current working directory in which ProxySQL is ran). The filename for the database is named "proxysql.db". So, disk gets loaded into memory and then propagated towards the runtime configuration. -# Initial startup (or --initial flag) +# Initial startup (or --initial/--reload flag) At the initial start-up (or start-up done with --initial flag, which resets the -database by renaming the old one), the major difference from normal start-up is that the memory and runtime configuration gets populated from the config file. +At the initial start-up (or start-up done with --initial flag, which resets the database by renaming the old one), the major difference from normal start-up is that the memory and runtime configuration gets populated from the config file. -After this is done, the configuration is also persisted to the disk database, -which will be used for the next restarts. +After this is done, the configuration is also persisted to the disk database, which will be used for the next restarts. + +The difference between --reload and --initial is that --reload does not create a backup of the old configuration file, while --initial does. # Soft restart (via angel process) From dac0997ddb77f5fb96a1863e7321044b31886848 Mon Sep 17 00:00:00 2001 From: Andrei Ismail Date: Tue, 25 Aug 2015 00:26:04 +0300 Subject: [PATCH 7/9] #351 Correctly explaining the default location of the config file --- doc/configuration_system.md | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/doc/configuration_system.md b/doc/configuration_system.md index fb12244ee..9c942878a 100644 --- a/doc/configuration_system.md +++ b/doc/configuration_system.md @@ -54,10 +54,7 @@ mysql> select * from global_variables limit 3; __DISK__ and __CONFIG FILE__ -__DISK__ represents an on-disk SQLite3 database, with the default location at -`/tmp/proxysql.db`. Across restarts, the in-memory configs that were not -persisted will be gone. __CONFIG__ file is the classical config file, and we'll -see the relationship between it and the other configuration layers in the next +__DISK__ represents an on-disk SQLite3 database, with the default location at `$(PWD)/proxysql.db`. Across restarts, the in-memory configs that were not persisted will be gone. __CONFIG__ file is the classical config file, and we'll see the relationship between it and the other configuration layers in the next section. In the following sections, we'll describe the lifecycle of each of these layers From bd83b4e6a311d77361d34f5a09e0725247f34c18 Mon Sep 17 00:00:00 2001 From: Andrei Ismail Date: Tue, 25 Aug 2015 00:28:41 +0300 Subject: [PATCH 8/9] #351 Cleaning up line endings --- doc/configuration_system.md | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/doc/configuration_system.md b/doc/configuration_system.md index 9c942878a..bc4e4cb1b 100644 --- a/doc/configuration_system.md +++ b/doc/configuration_system.md @@ -57,19 +57,15 @@ __DISK__ and __CONFIG FILE__ __DISK__ represents an on-disk SQLite3 database, with the default location at `$(PWD)/proxysql.db`. Across restarts, the in-memory configs that were not persisted will be gone. __CONFIG__ file is the classical config file, and we'll see the relationship between it and the other configuration layers in the next section. -In the following sections, we'll describe the lifecycle of each of these layers -for the basic operations that the daemon goes through: starting up for the -first time, starting up, restarting, shutting down, etc. +In the following sections, we'll describe the lifecycle of each of these layers for the basic operations that the daemon goes through: starting up for the first time, starting up, restarting, shutting down, etc. # Startup -At a normal start-up, ProxySQL initializes its configuration from the persisted -in-memory database. This is found in the "datadir" (usually the current working directory in which ProxySQL is ran). The filename for the database is named "proxysql.db". So, disk gets loaded into memory and then propagated towards the +At a normal start-up, ProxySQL initializes its configuration from the persisted in-memory database. This is found in the "datadir" (usually the current working directory in which ProxySQL is ran). The filename for the database is named "proxysql.db". So, disk gets loaded into memory and then propagated towards the runtime configuration. # Initial startup (or --initial/--reload flag) -At the initial start-up (or start-up done with --initial flag, which resets the At the initial start-up (or start-up done with --initial flag, which resets the database by renaming the old one), the major difference from normal start-up is that the memory and runtime configuration gets populated from the config file. After this is done, the configuration is also persisted to the disk database, which will be used for the next restarts. From 24113b9be1793b78628fc5a413b4f2510706d872 Mon Sep 17 00:00:00 2001 From: Andrei Ismail Date: Tue, 25 Aug 2015 00:31:21 +0300 Subject: [PATCH 9/9] #351 Add reference to separate document about admin tables --- doc/configuration_system.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/doc/configuration_system.md b/doc/configuration_system.md index bc4e4cb1b..8447fad42 100644 --- a/doc/configuration_system.md +++ b/doc/configuration_system.md @@ -107,6 +107,8 @@ Each such table has a well defined role in the admin interface: These tables represent the middle layer (in-memory database) from the diagram above and can be manipulated using standard SQL queries. In order to move the configuration from this layer upwards or downwards, see the next section. +For more details about these tables and their fields, see their dedicated document: TODO + # Moving config between layers In order to move configuration between the three layers, there are a set of different admin commands available through the admin interface. Once you understand what each of the three layers means, the semantics should be quite obvious. Together with the explanation of each command, there is a number written next to it. The number corresponds to the arrow in the diagram from above.