From 8b2ae5201be6788059ed340ca4dff6ebd4853343 Mon Sep 17 00:00:00 2001 From: Dan Heath <76443935+Dan-Heath@users.noreply.github.com> Date: Tue, 28 Jan 2025 21:14:43 +0000 Subject: [PATCH] backport of commit 1252481645090ad9899b700e40418cd3e9ef4e8a --- .../docs/concepts/host-discovery/gcp.mdx | 478 ++++++++++++++---- .../docs/concepts/host-discovery/index.mdx | 6 +- 2 files changed, 370 insertions(+), 114 deletions(-) diff --git a/website/content/docs/concepts/host-discovery/gcp.mdx b/website/content/docs/concepts/host-discovery/gcp.mdx index 891e8d1c15..7d7f3f135e 100644 --- a/website/content/docs/concepts/host-discovery/gcp.mdx +++ b/website/content/docs/concepts/host-discovery/gcp.mdx @@ -5,38 +5,262 @@ description: >- Use dynamic host catalogs to automatically discover GCP Compute Engine VM instances and add them as hosts. Create a host catalog and host set for GCP resources. --- # GCP dynamic host catalogs + Boundary uses dynamic host catalogs to automatically discover GCP Compute Engine VM instances and add them as hosts. +You can authenticate to GCP using a service account, service impersonation, or the GCP Application Default Credentials (ADC): + + + + +To authenticate using the service account, create a service account in GCP and download the private key file. + +The service account should have the following roles: + + - `roles/compute.viewer`: (Required) To list Compute Engine VM instance in the project. + - `roles/iam.serviceAccountKeyAdmin`: (Optional) To rotate the service account key if `disable_credential_rotation` is set to `false`. + + + + + +Service account impersonation allows Boundary to authenticate with GCP using a service account that impersonates another service account. +Impersonation is useful when you want to restrict the permissions of the service account Boundary uses. + +You must create two service accounts to authenticate using service impersonation: + +- `Base service account`: The service account Boundary uses to authenticate with GCP. The service account should have the following roles: + - `roles/iam.serviceAccountTokenCreator`: (Required) To create a token for the target service account. + - `roles/iam.serviceAccountKeyAdmin`: (Optional) To rotate the service account key if `disable_credential_rotation` is set to `false`. + +- `Target service account`: The service account to impersonate. The service account should have the following role: + - `roles/compute.viewer`: (Required) To list Compute Engine VM instance in the project. + + + + + +If you run Boundary on a GCP VM instance, you can use Application Default Credentials (ADC) to authenticate with GCP. +Boundary uses the service account associated with the VM instance to authenticate with GCP. + + + + + + +If you downloaded the private key from GCP, it may contain extra `/n` characters that cause an error. +You can use `jq` to remove the extra `/n` characters. + + + ## Create a host catalog to connect with GCP -Boundary uses plugins to integrate with a variety of providers. To use -a dynamic host catalog to integrate with GCP, you create a host catalog of the `plugin` type -and set the `plugin-name` value to `gcp`. You must also provide the specific -fields needed for Boundary to authenticate with GCP. -### Authentication with service account -Create a service account in GCP and download the private key file. The service account should have the following roles: -- `roles/compute.viewer` (Required): To list Compute Engine VM instance in the project. -- `roles/iam.serviceAccountKeyAdmin` (Optional): - To rotate the service account key if `disable_credential_rotation` is set to `false`. +Boundary uses plugins to integrate with a variety of providers. +To use a dynamic host catalog to integrate with GCP, you create a host catalog of the `plugin` type and set the `plugin-name` value to `gcp`. + +You must also provide the specific fields needed for Boundary to authenticate with GCP. +The required fields depend on whether you authenticate using the service account, service impersonation, or the GCP Application Default Credentials (ADC). -Pass the following fields to Boundary for authentication: +Complete the following steps to create a dynamic host catalog for GCP: - - -```shell-session -$ boundary host-catalogs create plugin \ - -scope-id $BOUNDARY_PROJECT_ID \ - -plugin-name gcp \ - -attr disable_credential_rotation=true \ - -attr zone=us-central1-a \ - -attr project_id=env://GCP_PROJECT_ID \ - -attr client_email=env://CLIENT_EMAIL \ - -secret private_key_id=env://PRIVATE_KEY_ID \ - -secret private_key=file://$PRIVATE_KEY_FILE_PATH -``` + + +1. Log in to Boundary. +1. Select the org, and then select the project you want to create a host catalog for. +1. Select **Host Catalogs**. +1. Select **New Host Catalog**. +1. Complete the following fields: + - **Name**: (Optional) An optional description of the host catalog for identification purposes. + If you enter a name, it must be unique. + - **Description**: (Optional) An optional description of the host catalog for identification purposes. + - **Type**: (Required) Select **Dynamic** to create a dynamic host catalog. + - **Provider**: (Required) Select **GCP** to create a dynamic host catalog for your GCP resources. + + + + + - **Project ID**: (Required) The project ID of any instances that you want to add to the host catalog. + - **Zone**: (Required) The GCP zone of the instances that you want to add to the host catalog. + - **Client Email**: The unique email address that is used to identify the service account. + It is required when you authenticate using the service account. + - **Target Service Account ID**: Skip this field when you configure authentication using the service account. + It is only used when you authenticate using service account impersonation. + - **Private Key ID**: The unique identifier of the private key. + It is required when you authenticate using the service account. + - **Private Key**: The private key used to obtain an OAuth 2.0 access token. + The key must be PEM encoded. + It is required when you authenticate using the service account. + + If you downloaded the private key from GCP, it may contain extra `/n` characters that cause an error. + You can use `jq` to remove the extra `/n` characters. + + + + + + - **Project ID**: (Required) The project ID of any instances that you want to add to the host catalog. + - **Zone**: (Required) The GCP zone of the instances that you want to add to the host catalog. + - **Client Email**: The unique email address that is used to identify the base service account. + It is required when you authenticate using service account impersonation. + - **Target Service Account ID**: The unique identifier for the service account that is impersonated. + It is required when you authenticate using service account impersonation. + - **Private Key ID**: The unique identifier of the private key for the base service account. + It is required when you authenticate using service account impersonation. + - **Private Key**: The private key used to obtain an OAuth 2.0 access token. + The key must be PEM encoded. + It is required when you authenticate using service account impersonation. + + If you downloaded the private key from GCP, it may contain extra `/n` characters that cause an error. + You can use `jq` to remove the extra `/n` characters. + + + + + + - **Project ID**: (Required) The project ID of any instances that you want to add to the host catalog. + - **Zone**: (Required) The GCP zone of the instances that you want to add to the host catalog. + - **Client Email**: (Optional) The unique email address that is used to identify the base service account. + - **Target Service Account ID**: Skip this field when you configure authentication using the ADC. + It is only used when you authenticate using service account impersonation. + - **Private Key ID**: (Optional) The unique identifier of the private key for the base service account. + - **Private Key**: (Optional) The private key used to obtain an OAuth 2.0 access token. + The key must be PEM encoded. + + If you downloaded the private key from GCP, it may contain extra `/n` characters that cause an error. + You can use `jq` to remove the extra `/n` characters. + + + + + - **Worker Filter**: (Optional) An optional filter to route requests to a designated worker. + - **Disable credential rotation**: (Optional) When enabled, Boundary does not rotate the credentials with GCP automatically. +1. Select **Save**. + + + + + + + + +1. Log in to Boundary. +1. Use the following command to create a dynamic host catalog for GCP: + + ```shell-session + $ boundary host-catalogs create plugin \ + -scope-id $BOUNDARY_PROJECT_ID \ + -plugin-name gcp \ + -attr disable_credential_rotation=true \ + -attr zone=us-central1-a \ + -attr project_id=env://GCP_PROJECT_ID \ + -attr client_email=env://CLIENT_EMAIL \ + -secret private_key_id=env://PRIVATE_KEY_ID \ + -secret private_key=file://$PRIVATE_KEY_FILE_PATH + ``` + + The `scope-id` and `plugin-name` fields are required when you create a dynamic host catalog. + + The fields following the `attr` and `secret` flags are specific to GCP and are used by Boundary for authentication: + + - `disable_credential_rotation`: (Optional) When set to `true`, Boundary does not rotate the credentials automatically. + - `zone`: (Required) The GCP zone of the instances that you want to add to the host catalog. + - `project_id`: (Required) The project ID of any instances that you want to add to the host catalog. + - `client_email`: The unique email address that is used to identify the service account. + It is required when you authenticate using the service account. + - `private_key_id`: The unique identifier of the private key. + It is required when you authenticate using the service account. + - `private_key`: The private key used to obtain an OAuth 2.0 access token. + The key must be PEM encoded. + It is required when you authenticate using the service account. + + If you downloaded the private key from GCP, it may contain extra `/n` characters that cause an error. + You can use `jq` to remove the extra `/n` characters. + + Refer to [the domain model documentation](/boundary/docs/concepts/domain-model/host-catalogs) for additional fields that you can use when you create host catalogs. + + + + + +1. Log in to Boundary. +1. Use the following command to create a dynamic host catalog for GCP: + + ```shell-session + $ boundary host-catalogs create plugin \ + -scope-id $BOUNDARY_PROJECT_ID \ + -plugin-name gcp \ + -attr disable_credential_rotation=true \ + -attr zone=us-central1-a \ + -attr project_id=env://GCP_PROJECT_ID \ + -attr client_email=env://BASE_SERVICE_ACCOUNT_EMAIL \ + -attr target_service_account_id=env://TARGET_SERVICE_ACCOUNT_EMAIL \ + -secret private_key_id=env://BASE_SERVICE_ACCOUNT_PRIVATE_KEY_ID \ + -secret private_key=file://$BASE_SERVICE_ACCOUNT_PRIVATE_KEY_FILE_PATH + ``` + + The `scope-id` and `plugin-name` fields are required when you create a dynamic host catalog. + + The fields following the `attr` and `secret` flags are specific to GCP and are required by Boundary for authentication: + + - `disable_credential_rotation`: (Optional) When set to `true`, Boundary does not rotate the credentials with GCP automatically. + - `zone`: (Required) The deployment area within a region. + All host sets in this catalog are configured for this zone. + - `project_id`: (Required) The project ID associated with the service account. + - `client_email`: The email address used to uniquely identify the base service account. + It is required when you authenticate using service account impersonation. + - `target_service_account_id`: The email address of the service account to impersonate. + It is required when you authenticate using service account impersonation. + - `private_key_id`: The ID of the private key for the base service account to use with this host catalog. + It is required when you authenticate using service account impersonation. + - `private_key`: The private key for the base service account to use with this host catalog. + It is required when you authenticate using service account impersonation. + + If you downloaded the private key from GCP, it may contain extra `/n` characters that cause an error. + You can use `jq` to remove the extra `/n` characters. + + Refer to [the domain model documentation](/boundary/docs/concepts/domain-model/host-catalogs) for additional fields that you can use when you create host catalogs. + + + + + +1. Log in to Boundary. +1. Use the following command to create a dynamic host catalog for GCP: + + ```shell-session + $ boundary host-catalogs create plugin \ + -scope-id $BOUNDARY_PROJECT_ID \ + -plugin-name gcp \ + -attr disable_credential_rotation=true \ + -attr zone=us-central1-a \ + -attr project_id=env://GCP_PROJECT_ID + ``` + + The `scope-id` and `plugin-name` fields are required when you create a dynamic host catalog. + + The fields following the `attr` flag are specific to GCP and are required by Boundary for authentication: + + - `disable_credential_rotation`: (Optional) When set to `true`, Boundary does not rotate the credentials with GCP automatically. + - `zone`: (Required) The deployment area within a region. + All host sets in this catalog are configured for this zone. + - `project_id`: (Required) The project ID associated with the service account. + + Refer to [the domain model documentation](/boundary/docs/concepts/domain-model/host-catalogs) for additional fields that you can use when you create host catalogs. + + + - + + + + + + +Refer to the [Boundary Terraform provider documentation](https://registry.terraform.io/providers/hashicorp/boundary/latest/docs) to learn about the requirements for the following example attributes. + +Apply the following Terraform policy: ```hcl resource "boundary_host_catalog_plugin" "gcp_host_catalog" { @@ -56,40 +280,33 @@ $ boundary host-catalogs create plugin \ } ``` - - - -### Authentication with service account impersonation -Service account impersonation allows Boundary to authenticate with GCP using a service account that impersonates another service account. -Impersonation is useful when you want to restrict the permissions of the service account Boundary uses. Two service accounts are required: + The `scope_id` and `plugin_name` fields are required when you create a dynamic host catalog. -- `Base service account`: The service account Boundary uses to authenticate with GCP. The service account should have the following roles: - - `roles/iam.serviceAccountTokenCreator` (Required) - To create a token for the target service account. - - `roles/iam.serviceAccountKeyAdmin` (Optional) - To rotate the service account key if `disable_credential_rotation` is set to `false`. + The fields following `attributes_json` and `secrets_json` are specific to GCP and are used by Boundary for authentication: -- `Target service account`: The service account to impersonate. The service account should have the following roles: - - `roles/compute.viewer` (Required): To list Compute Engine VM instance in the project. + - `zone`: (Required) The GCP zone of the instances that you want to add to the host catalog. + - `project_id`: (Required) The project ID of any instances that you want to add to the host catalog. + - `client_email`: The unique email address that is used to identify the service account. + It is required when you authenticate using the service account. + - `disable_credential_rotation`: (Optional) When set to `true`, Boundary does not rotate the credentials automatically. + - `private_key_id`: The unique identifier of the private key. + It is required when you authenticate using the service account. + - `private_key`: The private key used to obtain an OAuth 2.0 access token. + The key must be PEM encoded. + It is required when you authenticate using the service account. -Pass the following fields to Boundary for authentication: + If you downloaded the private key from GCP, it may contain extra `/n` characters that cause an error. + You can use `jq` to remove the extra `/n` characters. - - - -```shell-session -$ boundary host-catalogs create plugin \ - -scope-id $BOUNDARY_PROJECT_ID \ - -plugin-name gcp \ - -attr disable_credential_rotation=true \ - -attr zone=us-central1-a \ - -attr project_id=env://GCP_PROJECT_ID \ - -attr client_email=env://BASE_SERVICE_ACCOUNT_EMAIL \ - -attr target_service_account_id=env://TARGET_SERVICE_ACCOUNT_EMAIL \ - -secret private_key_id=env://BASE_SERVICE_ACCOUNT_PRIVATE_KEY_ID \ - -secret private_key=file://$BASE_SERVICE_ACCOUNT_PRIVATE_KEY_FILE_PATH -``` +Refer to [the domain model documentation](/boundary/docs/concepts/domain-model/host-catalogs) for additional fields that you can use when you create host catalogs. - + + + +Refer to the [Boundary Terraform provider documentation](https://registry.terraform.io/providers/hashicorp/boundary/latest/docs) to learn about the requirements for the following example attributes. + +Apply the following Terraform policy: ```hcl resource "boundary_host_catalog_plugin" "gcp_host_catalog" { @@ -110,29 +327,35 @@ $ boundary host-catalogs create plugin \ } ``` - - +The `scope_id` and `plugin_name` fields are required when you create a dynamic host catalog. -### Authentication with Application Default Credentials (ADC) -If you run Boundary on a GCP VM instance, you can use Application Default Credentials (ADC) to authenticate with GCP. -Boundary uses the service account associated with the VM instance to authenticate with GCP. +The fields following `attributes_json` and `secrets_json` are specific to GCP and are required by Boundary for authentication: -Pass the following fields to Boundary for authentication: +- `zone`: (Required) The deployment area within a region. All host sets in this + catalog are configured for this zone. +- `project_id`: (Required) The project ID associated with the service account. +- `client_email`: The email address used to uniquely identify the base service account. +It is required when you authenticate using service account impersonation. +- `target_service_account_id`: The email address of the service account to impersonate. +It is required when you authenticate using service account impersonation. +- `disable_credential_rotation`: (Optional) When set to `true`, Boundary does not rotate the credentials with GCP automatically. +- `private_key_id`: The ID of the private key for the base service account to use with this host catalog. +It is required when you authenticate using service account impersonation. +- `private_key`: The private key for the base service account to use with this host catalog. +It is required when you authenticate using service account impersonation. + + If you downloaded the private key from GCP, it may contain extra `/n` characters that cause an error. + You can use `jq` to remove the extra `/n` characters. - - - -```shell-session -$ boundary host-catalogs create plugin \ - -scope-id $BOUNDARY_PROJECT_ID \ - -plugin-name gcp \ - -attr disable_credential_rotation=true \ - -attr zone=us-central1-a \ - -attr project_id=env://GCP_PROJECT_ID -``` +Refer to [the domain model documentation](/boundary/docs/concepts/domain-model/host-catalogs) for additional fields that you can use when you create host catalogs. - + + + +Refer to the [Boundary Terraform provider documentation](https://registry.terraform.io/providers/hashicorp/boundary/latest/docs) to learn about the requirements for the following example attributes. + +Apply the following Terraform policy: ```hcl resource "boundary_host_catalog_plugin" "gcp_host_catalog" { @@ -148,47 +371,83 @@ $ boundary host-catalogs create plugin \ } ``` - - - -The `scope-id` and `plugin-name` fields are required when you create a - dynamic host catalog. +The `scope_id` and `plugin_name` fields are required when you create a dynamic host catalog. -The fields following the `attr` and `secret` flags are specific to GCP and are required by - Boundary for authentication. +The fields following `attributes_json` are specific to GCP and are required by Boundary for authentication: -- `zone` (Required): The deployment area within a region. All host sets in this - catalog are configured for this zone. -- `project_id` (Required): The project ID associated with the service account. -- `disable_credential_rotation` (Optional): When set to `true`, Boundary does not rotate the credentials with GCP automatically. -- `client_email` (Optional): Boundary uses this email address associated with the service account. - The email address used to uniquely identify the service account. -- `target_service_account_id` (Optional): Boundary uses this email address of the service account to impersonate. - This is used for authentication with service account impersonation. -- `private_key_id` (Optional): The ID of the private key for the service account to use with this host catalog. -- `private_key` (Optional): The private key for the service account to use with this host catalog. +- `zone`: (Required) The GCP zone of the instances that you want to add to the host catalog. +- `project_id`: (Required) The project ID of any instances that you want to add to the host catalog. +- `disable_credential_rotation`: (Optional) When set to `true`, Boundary does not rotate the credentials with GCP automatically. Refer to [the domain model documentation](/boundary/docs/concepts/domain-model/host-catalogs) for additional fields that you can use when you create host catalogs. + + + + + + ## Create a host set to connect with GCP -[Host sets](/boundary/docs/concepts/domain-model/host-sets) specify which GCP - filters Boundary should use to identify which discovered hosts that should be added as members. -Create a host set using the following command: +[Host sets](/boundary/docs/concepts/domain-model/host-sets) specify which GCP filters Boundary should use to identify which discovered hosts that should be added as members. + +Complete the following steps to create a host set: + + +1. Log in to Boundary. +1. Select the org, and then select the project you want to create a host set for. +1. Select **Host Catalogs**. +1. Select the dynamic host catalog to which you want to add a host set. +1. Click the **Host Sets** tab, and then click **New**. +1. Complete the following fields: + + - **Name**: (Optional) An optional name for identification purposes. + If you enter a name, it must be unique. + - **Description**: (Optional) An optional description of the host catalog for identification purposes. +1. Click **Save**. + + + -```shell-session -$ boundary host-sets create plugin \ - -host-catalog-id $BOUNDARY_HOST_CATALOG_ID \ - -attr filters=labels.env=prod \ - -attr filters=labels.app=web -``` +1. Log in to Boundary. +1. Use the following command to create a host set: + + ```shell-session + $ boundary host-sets create plugin \ + -host-catalog-id $BOUNDARY_HOST_CATALOG_ID \ + -attr filters=labels.env=prod \ + -attr filters=labels.app=web + ``` + + The `host-catalog-id` value is a required field that specifies in which host catalog to create this host set. + + Like with the host catalog, the fields passed in after the `attr` flag are specific to GCP. + + The `filters` field contains string filters in the format key=val1. + The key corresponds to a filter option. + For a list of filter options, refer to the [GCP API documentation](https://cloud.google.com/compute/docs/reference/rest/v1/instances/list#filter). + When you apply multiple filters, the plugin applies a logical `AND` operation to combine the filters. + + If there is no explicit instance status filter, the default status is set to `status="RUNNING"`. + The default status ensures that Boundary only filters running instances, saving time when processing results. + + We recommend using labels as filters because they are more flexible and can filter on multiple values as in the following examples: + + - `-attr filters=labels.env=dev` + - `-attr filters=labels.env=prod AND labels.app=web` + + For more fields that you can use when creating host sets, refer to [the domain model documentation](/boundary/docs/concepts/domain-model/host-sets). +Refer to the [Boundary Terraform provider documentation](https://registry.terraform.io/providers/hashicorp/boundary/latest/docs) to learn about the requirements for the following example attributes. + +Apply the following Terraform policy: + ```hcl resource "boundary_host_set_plugin" "gcp_host_set" { name = "GCP Host Set" @@ -199,27 +458,24 @@ resource "boundary_host_set_plugin" "gcp_host_set" { } ``` - - +The `host_catalog_id` value is a required field that specifies in which host catalog to create this host set. -The `host-catalog-id` value is a required field that specifies in which host catalog to - create this host set. +Like with the host catalog, the fields passed in after `attributes_json` flag are specific to GCP. -Like with the host catalog, the fields passed in after the `attr` flag are - specific to GCP. +The `filters` field contains string filters in the format key=val1. +The key corresponds to a filter option. +For a list of filter options, refer to the [GCP API documentation](https://cloud.google.com/compute/docs/reference/rest/v1/instances/list#filter). +When you apply multiple filters, the plugin applies a logical `AND` operation to combine the filters. -The `filters` field contains string filters in the format key=val1. The key corresponds to - a filter option. For a list of filter options, refer to the - [GCP API documentation](https://cloud.google.com/compute/docs/reference/rest/v1/instances/list#filter). - When you apply multiple filters, the plugin applies a logical `AND` operation to combine the filters. +If there is no explicit instance status filter, the default status is set to `status="RUNNING"`. +The default status ensures that Boundary only filters running instances, saving time when processing results. - If there is no explicit instance status filter, the default status is set to `status="RUNNING"`. - This ensures that Boundary only filters running instances, saving time when processing results. +We recommend using labels as filters because they are more flexible and can filter on multiple values as in the following examples: - We recommend using labels as filters because they are more flexible and can filter on multiple values - as in the following examples: - `-attr filters=labels.env=dev` - `-attr filters=labels.env=prod AND labels.app=web` -For more fields that you can use when creating host sets, refer to - [the domain model documentation](/boundary/docs/concepts/domain-model/host-sets). +For more fields that you can use when creating host sets, refer to [the domain model documentation](/boundary/docs/concepts/domain-model/host-sets). + + + diff --git a/website/content/docs/concepts/host-discovery/index.mdx b/website/content/docs/concepts/host-discovery/index.mdx index 0b085ad659..a1a57dbf16 100644 --- a/website/content/docs/concepts/host-discovery/index.mdx +++ b/website/content/docs/concepts/host-discovery/index.mdx @@ -54,12 +54,12 @@ external resources should be ingested into the catalog by with an attributes filter. These filters specify which discovered hosts should be members of the host set. -Boundary currently supports dynamic host catalog for AWS and -Azure and we will continue to grow this ecosystem to support additional providers. +Boundary currently supports dynamic host catalog for AWS, Azure, and GCP. +We will continue to grow this ecosystem to support additional providers. ## Next steps To get started with dynamic host catalogs, refer to the following topics: - + - [AWS dynamic host catalogs](/boundary/docs/concepts/host-discovery/aws) - [Azure dynamic host catalogs](/boundary/docs/concepts/host-discovery/azure) - [GCP dynamic host catalogs](/boundary/docs/concepts/host-discovery/gcp)