Verify the Boundary controller server is up and able to receive requests
Verify the Boundary controller and worker server status using health endpoints.
---
# Boundary health endpoints
Boundary provides health monitoring through the `/health` path using a listener
with the `"ops"` purpose. By default, a listener with that purpose runs on port
`9203`. See the [example configuration](#example-configuration) section for an
example listener stanza in a `config.hcl` file.
Boundary provides health monitoring through the `/health` path using a listener with the `"ops"` purpose. By default, a listener with that purpose runs on port `9203`. See the [example configuration](#example-configuration) section for an example listener stanza in a `config.hcl` file.
## Requirements
To enable the controller health endpoint, any Boundary instance must be
started with a controller. That is, a `controller` block and a `purpose = "api"`
listener must be defined in Boundary's configuration file. Additionally, a
`purpose = "ops"` listener must also be defined in Boundary's configuration
file. Under these conditions, the `ops` server (which hosts the controller health
api) will be exposed.
To enable the controller health endpoint, any Boundary instance must be started with a controller. That is, a `controller` block and a `purpose = "api"`
listener must be defined in Boundary's configuration file. Additionally, a `purpose = "ops"` listener must also be defined in Boundary's configuration file. Under these conditions, the `ops` server (which hosts the controller health api) will be exposed.
## Shutdown grace period
Optionally, when the controller health endpoint is enabled, Boundary can be
configured to change the controller health response to `503 Service Unavailable`
upon receiving a shutdown signal, and wait a configurable amount of time before
starting the shutdown process.
Optionally, when the controller health endpoint is enabled, you can configure the controller health response to `503 Service Unavailable` upon receiving a shutdown signal, and wait a configurable amount of time before starting the shutdown process.
This feature is designed to integrate with load balancers to reduce the risk of
an outgoing Boundary instance causing disruption to incoming requests.
This feature supports load balancing by reducing the risk of an outgoing Boundary instance causing disruption to incoming requests.
In this state, Boundary is still capable of processing requests as normal, but
will report as unhealthy through the controller health endpoint. In
load-balanced environments, this would cause this "unhealthy" instance to be
removed from the pool of instances eligible to handle requests, and thereby, reducing the
likelihood that it will receive a request to handle during shutdown.
In this state, Boundary is still capable of processing requests as normal, but will report as unhealthy through the controller health endpoint. In load-balanced environments, this would cause this "unhealthy" instance to be removed from the pool of instances eligible to handle requests, reducing the likelihood that it will receive a request to handle during shutdown.
This feature is disabled by default, even if the controller health endpoint is
enabled. You can set it up by defining `graceful_shutdown_wait_duration` in the
`controller` block of Boundary's configuration file. The value should be set to
This feature is disabled by default, even if the controller health endpoint is enabled. You can enable it by defining `graceful_shutdown_wait_duration` in the `controller` block of Boundary's configuration file. The value should be set to a string that is parseable by [ParseDuration](https://pkg.go.dev/time#ParseDuration).
## API
@ -55,13 +36,40 @@ The controller health service introduces a single read-only endpoint:
All responses return empty bodies. `GET /health` does not support any input.
## Check the health endpoint using `wget`
The Boundary Docker image includes `wget`. You can use it to check the health endpoint. Enterprise and Community edition users can check the health of controllers and workers. HCP Boundary users can check the health of their self-managed workers.
Use the following command to check a worker's status to determine if it is healthy:
- `-q` - Prints the response instead of saving it to a file.
- `-0` - Allows Boundary to pass the worker's URL.
If the command prints 1 to `stdout` or exits with exit code 0, it means the worker is healthy. If the exit code is greater than 0, the worker is unhealthy.
You can also use the following command without `grep -c 'READY'` to print a more verbose response that includes the worker's state, active session count, and connection state:
When you check the health endpoint using `curl`, Boundary returns a response with the following worker information:
```shell-session
$ curl "worker:9403/health?worker_info=1"
{
"worker_process_info":{
"state":"active",
@ -70,14 +78,12 @@ When you check the health endpoint, Boundary returns a response with the followi
}
}
```
The response contains the following fields:
- `state` - The operational state of the worker.
Possible worker states include active, shutdown, and unknown.
- `state` - The operational state of the worker. Possible worker states include active, shutdown, and unknown.
- `active_session_count` - The number of active sessions on the worker.
- `upstream_connection_state` - The connection state of the worker.
This value indicates whether the worker can connect to an upstream address or connection.
It can be any of the following states:
- `upstream_connection_state` - The connection state of the worker. This value indicates whether the worker can connect to an upstream address or connection. It can be any of the following states:
- `CONNECTING` - The channel is trying to make a connection and is waiting for name resolution or the connection establishment.
- `READY` - The channel has successfully established a connection, and any attempts to communicate have succeeded.
- `TRANSIENT_FAILURE` - The channel suffered a transient failure such as a time out or socket error.
@ -89,9 +95,7 @@ It can be any of the following states:
## Example configuration
Health checks are available for a controller defined with a `purpose = "ops"`
listener stanza. For details on what fields are allowed in this stanza, refer to
the documentation about [TCP Listener](/boundary/docs/configuration/listener/tcp).
Health checks are available for a controller defined with a `purpose = "ops"` listener stanza. For details on what fields are allowed in this stanza, refer to the documentation about [TCP Listener](/boundary/docs/configuration/listener/tcp).
An example listener stanza:
@ -114,8 +118,7 @@ listener "tcp" {
}
```
To enable a shutdown grace period, update the `controller` block with a defined
wait duration:
To enable a shutdown grace period, update the `controller` block with a defined wait duration:
```hcl
controller {
@ -127,6 +130,4 @@ controller {
}
```
A complete example can be viewed under the [Controller
In a future version Boundary will no longer automatically create roles when new scopes are created. This was implemented prior to multi-scope grants to ensure administrators and users had default permissions in new scopes. Since Boundary 0.15, initial roles created for new clusters provide these permissions by default to all scopes using multi-scope grants.
In a future version Boundary will no longer automatically create roles when new scopes are created. This was implemented prior to multi-scope grants to ensure administrators and users had default permissions in new scopes. Since Boundary 0.15, initial roles created for new clusters provide these permissions by default to all scopes using multi-scope grants.
</td>
</tr>
<tr>
<td style={{verticalAlign: 'middle'}}>
Docker image no longer contains <code>curl</code>
</td>
<td style={{verticalAlign: 'middle'}}>
As of version 0.16.3 and later, the <code>curl</code> binary is no longer included in the published Docker container image for Boundary. The image now includes <code>wget</code>, which you can alternatively use to check the health endpoint for a worker. If your workflow depends on having <code>curl</code> in the image, you can dynamically install it using <code>apk</code>.
<br /><br />
Learn more: <a href="#known-issues-and-breaking-changes">Known issues and breaking changes </a>
</td>
</tr>
</tbody>
</table>
## New features
@ -37,13 +64,13 @@ description: |-
GA
</td>
<td style={{verticalAlign: 'middle'}}>
A new resource was introduced in this release. Aliases let you associate a string with a Boundary target. You can then establish a session to the target by referencing its alias, instead of having to provide a target ID or target name and scope ID.
A new resource was introduced in this release. Aliases let you associate a string with a Boundary target. You can then establish a session to the target by referencing its alias, instead of having to provide a target ID or target name and scope ID.
A bug caused Boundary Desktop to incorrectly assume the controller was running an unsupported version. It would prevent users from being able to log in to the Desktop client.
<a href="/boundary/tutorials/self-managed-deployment/upgrade-version">Upgrade to the latest version of Boundary</a>
</td>
</tr>
<tr>
<td style={{verticalAlign: 'middle'}}>
0.16.0
@ -202,27 +249,69 @@ description: |-
</td>
</tr>
<tr>
<tr>
<td style={{verticalAlign: 'middle'}}>
0.16.3+
</td>
<td style={{verticalAlign: 'middle'}}>
Docker image no longer contains <code>curl</code>
</td>
<td style={{verticalAlign: 'middle'}}>
As of version 0.16.3 and later, the <code>curl</code> binary is no longer included in the published Docker container image for Boundary.
<br /><br />
The image now includes <code>wget</code>. You can use <code>wget</code> to check the health endpoint for workers.
<br /><br />
Learn more: <a href="/boundary/docs/operations/health#check-the-health-endpoint-using-wget">Check the health endpoint using <code>wget</code></a>
<br /><br />
If your workflow depends on having <code>curl</code> in the image, you can dynamically install it using <code>apk</code>. Refer to the following commands for examples of using <code>apk</code> to install <code>curl</code>:
A bug caused Boundary Desktop to incorrectly assume the controller was running an unsupported version. It would prevent users from being able to log in to the Desktop client.
If you tried to connect to an invalid alias, Boundary returned a message with the 401 status code. The 401 status code could cause users to believe that they did not properly authenticate, rather than realizing the alias resource was invalid.
<br /><br />
Learn more:
This issue is fixed in Boundary version 0.16.3. If you use an invalid alias, Boundary now properly returns a 404 status code, indicating that it could not find the alias resource.
<a href="/boundary/tutorials/self-managed-deployment/upgrade-version">Upgrade to the latest version of Boundary</a>
</td>
</tr>
<tr>
<td style={{verticalAlign: 'middle'}}>
0.16.0
<br /><br />(Fixed in 0.16.3)
</td>
<td style={{verticalAlign: 'middle'}}>
Session recording fails when you use Secure File Copy (SCP)
</td>
<td style={{verticalAlign: 'middle'}}>
If you tried to use SCP during a recorded session, the session recording details may shown the recording state as Failed, and you may not have been able to play back the session.
<br /><br />
This issue occurs when a recorded session file is too large. It can happen when you use SCP to transfer large files during a recorded session, In Boundary version 0.16.3, we have increased the maximum size for recorded session files to 5 GB. This issue should happen less often now, but be careful when you use SCP during a recorded session, because it can result in large recorded session files.
<a href="/boundary/tutorials/self-managed-deployment/upgrade-version">Upgrade to the latest version of Boundary</a>
</td>
</tr>
</tbody>
</table>
@ -242,7 +331,7 @@ description: |-
<code>boundary daemon</code> command
</td>
<td style={{verticalAlign: 'middle'}}>
The <code>boundary daemon</code> command has been deprecated in favor of the new <code>boundary cache</code> command. The functionality remains the same.
The <code>boundary daemon</code> command has been deprecated in favor of the new <code>boundary cache</code> command. The functionality remains the same.
The <code>include_terminated</code> field from the <code>list sessions</code> command has been deprecated and will be removed in an upcoming release. Boundary will return terminated sessions in all list session responses, unless they are filtered out using the <code>filter</code> field. For more information, refer to the filter resource documentation.
The <code>include_terminated</code> field from the <code>list sessions</code> command has been deprecated and will be removed in an upcoming release. Boundary will return terminated sessions in all list session responses, unless they are filtered out using the <code>filter</code> field. For more information, refer to the filter resource documentation.
<br /><br />
Learn more:
<a href="/boundary/docs/commands/sessions/list#command-options"><code>sessions list</code> command options</a> and <a href="/boundary/docs/concepts/filtering/resource-listing">Filter resource listings</a>
@ -266,7 +355,7 @@ description: |-
<code>grant_scope_id</code> field removed from roles
</td>
<td style={{verticalAlign: 'middle'}}>
The <code>grant_scope_id</code> field, which was deprecated in release 0.15.0, has been removed from roles. You can now manage roles using the <code>add-grant-scopes</code>, <code>remove-grant-scopes</code>, and <code>set-grant-scopes</code> commands.
The <code>grant_scope_id</code> field, which was deprecated in release 0.15.0, has been removed from roles. You can now manage roles using the <code>add-grant-scopes</code>, <code>remove-grant-scopes</code>, and <code>set-grant-scopes</code> commands.
<br /><br />
Learn more: <a href="/boundary/docs/commands/roles/add-grant-scopes"><code>add-grant-scopes</code></a>, <a href="/boundary/docs/commands/roles/remove-grant-scopes"><code>remove-grant-scopes</code></a>, and <a href="/boundary/docs/commands/roles/set-grant-scopes"><code>set-grant-scopes</code></a>
In a future version Boundary will no longer automatically create roles when new scopes are created. This was implemented prior to multi-scope grants to ensure administrators and users had default permissions in new scopes. Since Boundary 0.15, initial roles created for new clusters provide these permissions by default to all scopes using multi-scope grants.
</td>
</tr>
<tr>
<td style={{verticalAlign: 'middle'}}>
Docker image no longer contains <code>curl</code>
</td>
<td style={{verticalAlign: 'middle'}}>
As of version 0.17.1 and later, the <code>curl</code> binary is no longer included in the published Docker container image for Boundary. The image now includes <code>wget</code>, which you can alternatively use to check the health endpoint for a worker. If your workflow depends on having <code>curl</code> in the image, you can dynamically install it using <code>apk</code>.
<br /><br />
Learn more: <a href="#known-issues-and-breaking-changes">Known issues and breaking changes </a>
</td>
</tr>
</tbody>
</table>
## New features
<table>
@ -31,13 +64,13 @@ description: |-
GA
</td>
<td style={{verticalAlign: 'middle'}}>
Prior to this version, if you wanted to edit or update worker tags, you had to do it using the worker configuration file, the CLI, or the API. Now, you can edit worker tags directly in the Boundary UI.
Prior to this version, if you wanted to edit or update worker tags, you had to do it using the worker configuration file, the CLI, or the API. Now, you can edit worker tags directly in the Boundary UI.
As of version 0.17.1 and later, the <code>curl</code> binary is no longer included in the published Docker container image for Boundary.
<br /><br />
The image now includes <code>wget</code>. You can use <code>wget</code> to check the health endpoint for workers.
<br /><br />
Learn more: <a href="/boundary/docs/operations/health#check-the-health-endpoint-using-wget">Check the health endpoint using <code>wget</code></a>
<br /><br />
If your workflow depends on having <code>curl</code> in the image, you can dynamically install it using <code>apk</code>. Refer to the following commands for examples of using <code>apk</code> to install <code>curl</code>:
If you tried to connect to an invalid alias, Boundary returned a message with the 401 status code. The 401 status code could cause users to believe that they did not properly authenticate, rather than realizing the alias resource was invalid.
<br /><br />
This issue is fixed in Boundary version 0.17.1. If you use an invalid alias, Boundary now properly returns a 404 status code, indicating that it could not find the alias resource.
<a href="/boundary/tutorials/self-managed-deployment/upgrade-version">Upgrade to the latest version of Boundary</a>
</td>
</tr>
<tr>
<td style={{verticalAlign: 'middle'}}>
0.17.0 (Fixed in 0.17.1)
</td>
<td style={{verticalAlign: 'middle'}}>
Session recording fails when you use Secure File Copy (SCP)
</td>
<td style={{verticalAlign: 'middle'}}>
If you tried to use SCP during a recorded session, the session recording details may shown the recording state as Failed, and you may not have been able to play back the session.
<br /><br />
This issue occurs when a recorded session file is too large. It can happen when you use SCP to transfer large files during a recorded session, In Boundary version 0.17.1, we have increased the maximum size for recorded session files to 5 GB. This issue should happen less often now, but be careful when you use SCP during a recorded session, because it can result in large recorded session files.
hc-github-team-secure-boundary
2 years ago
committed by
GitHub