* docs: Document API rate limiting
* docs: Fix default setting
* addss rate limiting examples to complete configuration example
* use # for comments for consistency
* Update website/content/docs/api-clients/api.mdx
Co-authored-by: Robin Beck <stellarsquall@users.noreply.github.com>
* Apply batch suggestions from code review
Co-authored-by: Michael Li <michael.li@hashicorp.com>
* docs: Updates from review feedback
* docs: Updates from review feedback
* docs: Add new defaults and sub-headings
* Update website/content/docs/api-clients/go-sdk.mdx
Co-authored-by: Timothy Messier <tim.messier@gmail.com>
* docs: Remove remaining comments from code blocks
* docs: Update code blocks
* fixes tabs
* Update website/content/docs/api-clients/api.mdx
Co-authored-by: Michael Li <michael.li@hashicorp.com>
---------
Co-authored-by: stellarsquall <stellarsquall@protonmail.ch>
Co-authored-by: Robin Beck <stellarsquall@users.noreply.github.com>
Co-authored-by: Michael Li <michael.li@hashicorp.com>
Co-authored-by: Timothy Messier <tim.messier@gmail.com>
@ -25,7 +25,9 @@ Boundary's current API version is 1; all API paths begin with `/v1/`.
- `403`: Boundary returns `403` if a provided token was valid but does not have the grants required to perform the requested action.
- `404`: Boundary returns `404` if a resource cannot be found. Note that this happens _prior_ to authentication/authorization checking in nearly all cases as the resource information (such as its scope, available actions, etc.) is a required part of that check. As a result, an action against a resource that does not exist will return a `404` instead of a `401` or `403`. While this could be considered an information leak, since IDs are randomly generated and this only discloses whether an ID is valid, it's tolerable as it allows for far simpler and more robust client implementation.
- `405`: Boundary returns a `405` to indicate that the method (HTTP verb or custom action) is not implemented for the given resource.
- `429`: Boundary returns a `429` if any of the API rate limit quotas have been exhausted for the resource and action. It includes the `Retry-After` header so that the client knows how long to wait before making a new request.
- `500`: Boundary returns `500` if an error occurred that is not (directly) tied to invalid user input. If a `500` is generated, information about the error will be logged to Boundary's server log but is not generally provided to the client.
- `503`: Boundary returns a `503` if it is unable to store a quota due to the API rate limit being exceeded. It includes the `Retry-After` header so that the client knows how long to wait before making a new request.
## Path layout
@ -79,3 +81,197 @@ The following method conventions are used within Boundary's API:
### DELETE
`DELETE` is used for deleting a specific resource, and is only used against a particular resource path.
## Rate limiting
If your controllers try to process every API request, it can lead to situations when the controllers are overwhelmed with requests.
The controllers may either run out of resources or overwhelm the database server and exhaust its resources.
API rate limiting lets you configure limits on the rate of API requests to help manage your resources and prevent them from being overwhelmed.
### Quotas
Boundary creates quotas to track the number of requests in a given time period.
By default, Boundary tracks the number of requests by auth token, IP address, and the overall total.
Boundary reserves a portion of its memory for tracking quotas to ensure that it does not consume too much memory if there is a sudden burst of requests.
If Boundary is unable to store a quota, it limits the request with a 503 HTTP status code.
You can configure the maximum number of quotas Boundary allows using the `api_rate_limit_max_quotas` variable.
There are also two metrics that allow you to monitor quota tracking:
There are separate configurable limits for each combination of resource and action.
By default, the limits for `list` actions are:
- 150 requests per 30 seconds per auth token
- 1,500 requests per 30 seconds per IP address
- 1,500 requests per 30 seconds in total
The default limits for all other actions are:
- 3,000 requests per 30 seconds per auth token
- 30,000 requests per 30 seconds per IP address
- 30,000 requests per 30 seconds in total
You can override the default settings and configure other specific limitations using the `api_rate_limit` stanza in the controller configuration.
### HTTP headers
Clients that make requests to the controller API can inspect HTTP response headers to understand the configured limits and current usage.
Each response contains the following headers:
- `RateLimit` - Provides the current limit, number of remaining requests, and the time at which the quota will reset for the limit that is closest to being exhausted for the requested resource and action.
- `RateLimit-Policy` - Describes the limits for the requested resource and action.
If the request is limited, Boundary sends the client a 429 HTTP status code with a `Retry-After` header.
The `Retry-After` header contains the number of seconds the client should wait before it sends the request again.
### More information
Refer to the [controller stanza](/boundary/docs/configuration/controller#api_rate_limit) documentation for the specific `api_rate_limit` configuration options.
Some example configurations are listed below.
### Rate limiting configuration examples
The following example shows a simple configuration where the same limits are applied to all resources and examples:
```hcl
controller {
# Total limit for all resources and actions
api_rate_limit {
resources = ["*"]
actions = ["*"]
per = "total"
limit = 500
period = "1s"
}
# Limit for all IP addresses to all resources and actions to prevent a
# malicious host that is fabricating tokens or spamming unauthed endpoints
api_rate_limit {
resources = ["*"]
actions = ["*"]
per = "ip-address"
limit = 100
period = "1s"
}
# Limit for all authed requests to prevent one user
# from consuming all of the total limit
api_rate_limit {
resources = ["*"]
actions = ["*"]
per = "auth-token"
limit = 100
period = "1s"
}
}
```
The following example shows a configuration that disables rate limiting.
You may want to disable rate limiting if you already use an external system like a reverse proxy to apply rate limiting:
```hcl
controller {
api_rate_limit_disable = true
}
```
The following example uses the default settings for most endpoints, but configures a single override:
```hcl
controller {
api_rate_limit {
resources = ["target"]
actions = ["list"]
per = "auth-token"
limit = 10
period = "1s"
}
}
```
The following example is more complex.
Initially it sets some defaults to apply to all resources and actions.
Then it configures some specific endpoints with different limits.
```hcl
controller {
# Total limit for all resources and actions
api_rate_limit {
resources = ["*"]
actions = ["*"]
per = "total"
limit = 500
period = "1s"
}
# Total limit for all list operations,
# sets an overall cap on the list action since it is relatively expensive
api_rate_limit {
resources = ["*"]
actions = ["list"]
per = "total"
limit = 200
period = "1s"
}
# Limit for IP address to all resources and actions
# to prevent a malicious host that is fabricating tokens
# or spamming unauthed endpoints
api_rate_limit {
resources = ["*"]
actions = ["*"]
per = "ip-address"
limit = 50
period = "1s"
}
# Limit of all authed requests, this essentially sets a default
# for all authed requests to prevent one user from consuming
# all of the total list limit set above. Any limits that follow
# can override the default for specific sets of resources and actions
api_rate_limit {
resources = ["*"]
actions = ["*"]
per = "auth-token"
limit = 100
period = "1s"
}
# Limit of all list operations by auth token to set a sane default
# since they are generally more expensive
api_rate_limit {
resources = ["*"]
actions = ["list"]
per = "auth-token"
limit = 50
period = "1s"
}
# Limit for targets and sessions list by auth token. These are resources
# that clients will want to list frequently, but might be more expensive
# if there is a large number of them for each user. Having a lower limit
# encourages the use of refresh tokens and caching.
Boundary has a Go SDK that sports full coverage of Boundary's API. This SDK is mostly auto-generated, so the patterns are predictable from package to package. For the most part, browsing [pkg.go.dev](https://pkg.go.dev/github.com/hashicorp/boundary/api) is a great way to get started.
<Note>
The Boundary API is configured with [rate limiting](/boundary/docs/api-clients/api#rate-limiting), by default.
If the Boundary SDK client is unable to process a request due to a rate limit being met, it sends the API a `429` or `523` [status code](/boundary/docs/api-clients/api#status-codes) and limits the request.
By default the client will wait for the amount of time specified in the `Retry-After` header and then try again for a maximum of 2 retries.
</Note>
Below, an example walks through using the SDK to authenticate against an auth method or perform recovery workflows. The patterns for creating a resource-typed client are the same across packages.
## Authenticating to Boundary with the Go SDK
@ -64,7 +72,11 @@ Now let's create an auth method client using the base client from above:
amClient := authmethods.NewClient(client)
```
~> **Note:** This creates a shallow copy of the base client. Modifications made to the client via `am.ApiClient()` will not be reflected in the base client.
<Note>
This creates a shallow copy of the base client. Modifications made to the client via `am.ApiClient()` will not be reflected in the base client.
</Note>
The last thing you'll need is the ID of the auth method in Boundary. You can get this on the CLI
are anything specified by Go's [ParseDuration()](https://golang.org/pkg/time/#ParseDuration) method. Only
used when an `ops` listener is set and the Controller is present. Default is 0 seconds.
- `api_rate_limit` - Sets limits on the rate of requests for controller API endpoints.
This setting can help prevent resources from being overwhelmed with too many requests at a time.
The `api_rate_limit` configuration stanza contains the following fields:
- `resources` - Specifies the Boundary resource you want to limit rates for.
The resource can be `target` or `credential-library`, for example.
You can include all resources by using the wildcard `"*"`.
- `actions` - Specifies the actions you want to limit on the resource.
The actions could be `create`, `list`, or `authorize-session`, for example.
You can include all actions by using the wildcard `"*"`.
- `per` - Specifies how the limit is allocated.
You can choose from the following values:
- `total` - Counts all requests, regardless of auth token or IP address.
- `ip-address` - Counts requests per IP address.
This value lets you limit requests per IP address.
- `auth-token` - Counts requests per the user's auth token.
This value lets you limit requests per auth token.
You can also use the wildcard `"*"` to include all values, but only when `unlimited` is also configured.
- `limit` - Specifies the number of requests that are allowed within the `period`.
- `period` - Specifies the time window for the `limit`.
The limit resets after this period of time has passed.
- `unlimited` - Indicates that the corresponding resources and actions should not be rate limited.
If you set this value to `true`, you should not specify values for the `limit` and `period` or you will receive an error.
For more information about how API rate limiting works, refer to the [API rate limiting](/boundary/docs/api-clients/api#rate-limiting) documentation.
- `api_rate_limit_disable` - Disables API rate limiting, if set to `true`.
If `api_rate_limit_disable` is set to `true`, and you have provided any `api_rate_limit` stanzas, you will receive an error.
- `api_rate_limit_max_quotas` - Specifies the maximum number of API rate limiting quotas that Boundary allows.
## Signals
The `SIGHUP` signal causes a controller to reload its configuration file to pick up any updates to the `database url` value. Any other updated values are ignored.
| `boundary_controller_api_http_request_duration_seconds` | Histogram of latencies for HTTP requests. |
| `boundary_controller_api_http_request_size_bytes` | Histogram of request sizes for HTTP requests. |
| `boundary_controller_api_http_response_size_bytes` | Histogram of response sizes for HTTP requests. |
| `boundary_controller_api_ratelimiter_quota_storage_capacity` | A gauge of storage capacity for API rate limiting quotas. |
| `boundary_controller_api_ratelimiter_quota_storage_usage` | A gauge of storage usage for API rate limiting quotas. |
| `boundary_controller_cluster_grpc_request_duration_seconds` | Histogram of latencies for requests made to the gRPC service running on the cluster listener. |
Dan Heath
2 years ago
committed by
GitHub