From 9f3dd5b72a1fa558ba61a3d75ce562d8b88242dc Mon Sep 17 00:00:00 2001 From: Lucas Bajolet Date: Mon, 13 May 2024 17:28:19 -0400 Subject: [PATCH] website: update install plugins docs Since we're changing how packer manages plugin installation with 1.11.0, we reflect those changes to the website documentation. Now, we only describe the packer init and packer plugins install commands, along with the `--path` flag for installing from a local source. The explanations of how packer discovers and picks which version of a plugin to load are also included, along with the list of constraints that determine whether a plugin can be considered or not to be loadable. --- .../content/docs/plugins/install-plugins.mdx | 240 +++++++++--------- 1 file changed, 119 insertions(+), 121 deletions(-) diff --git a/website/content/docs/plugins/install-plugins.mdx b/website/content/docs/plugins/install-plugins.mdx index 2413d025f..fad39b7d9 100644 --- a/website/content/docs/plugins/install-plugins.mdx +++ b/website/content/docs/plugins/install-plugins.mdx @@ -8,10 +8,7 @@ page_title: Install Plugins Packer plugins are separate, standalone applications that perform tasks during each build. -You do not need to install the components that ship with the Packer binary. -Packer automatically knows how to find and launch these built-in plugins. - -This page explains how to install custom external plugins. Refer to [External Plugins](/packer/integrations) for a list of available plugins and their documentation. +This page explains how to install external plugins. Refer to [Integrations](/packer/integrations) for a list of available plugins and their documentation. Depending on the template type you're using (HCL2 or legacy JSON), the methods for installing plugins may differ. @@ -22,50 +19,81 @@ command, and those requirements are explicitly documented in the template. be repeated for as many plugins as you need. We recommend this for JSON as the template cannot contain the information about the required plugins. -Finally, you can manually install any plugin. This is mostly useful if you're in an environment with -restricted internet access, or if you're installing non-final versions of plugins. +While `packer plugins install` is able to install plugins remotely, you can use +the `--path` flag to install locally-sourced plugin binaries. +This is typically useful in two general cases: + +1. You are in an environment with restricted access to a plugin's source. +1. You are trying to install a development version of a plugin for testing. Refer to the [Installation Guides](#installation-guides) section of this page for information about -each, including usage examples. +each method, including usage examples. -The remainder of this document will serve as documentation on how Packer interacts with plugins. +The remainder of this document documents how Packer interacts with plugins. We encourage you to read this to get familiar with this process, as it will help you troubleshoot -your builds if you encounter problems with that. +builds if you encounter problems with plugins. ## Source Addresses -A plugin's source address is its global identifier. It also tells Packer where -to download it. +A plugin's source address is conceptually where a plugin binary can be downloaded from. +Packer also uses this source as a global unique identifier. +This source translates directly to the file hierarchy in which the plugin binaries will be +installed. + +Source addresses are a URL, without a scheme (e.g. `"https://"`), query (e.g. `?arg=val`), or +fragment (e.g. `#anchor`). + +Example: `github.com/hashicorp/happycloud` -Source addresses consist of three parts delimited by slashes (`/`), as -follows: +#### Github sources -`//` +Github sources all follow this convention: `//`. -- **Hostname:** The hostname of the location/service that - distributes the plugin. Currently, the only valid "hostname" is github.com, - but we plan to eventually support plugins downloaded from other domains. +- **Hostname:** The hostname of the location/service that distributes the plugin. - **Namespace:** An organizational namespace within the specified host. - This often is the organization that publishes the plugin. + This often is the organization that maintains the plugin. - **Type:** A short name for the platform or system the plugin manages. The type is usually the plugin's preferred local name. -For example, the fictional `myawesomecloud` plugin could belong to the +For example, the fictional `happycloud` plugin could belong to the `hashicorp` namespace on `github.com`, so its `source` could be -`github.com/hashicorp/myawesomecloud`, +`github.com/hashicorp/happycloud`, --> Note: the actual _repository_ that myawesomecloud comes from must always have -the name format `github.com/hashicorp/packer-plugin-myawesomecloud`, but the -`required_plugins` block omits the redundant `packer-plugin-` repository prefix -for brevity. +-> Note: the actual _repository_ that happycloud comes from must always have +the name format `github.com/hashicorp/packer-plugin-happycloud`, but the +source voluntarily omits the redundant `packer-plugin-` prefix for brevity. -The source address with all three components given explicitly is called the -plugin's _fully-qualified address_. You will see fully-qualified address in +The source address with all the components given explicitly is called the +plugin's _fully-qualified address_. You will see fully-qualified addresses in various outputs, like error messages. -## Plugin Loading Workflow +### Source and local hierarchy + +As mentioned above, each source URL will directly impact how Packer will install +a plugin to the local plugin directory. +Each component of the path described will become a subdirectory, and finally, +plugin binaries are installed at the end of that hierarchy. +Multiple versions can be installed along one another. + +In order for the binaries installed in this directory to be discovered and used +by Packer, they must follow this set of rules: + +1. The name of the binary must be in the `packer-plugin-__x__(.exe)?` format +1. The binary must be accompanied by a SHA256SUM file, which must contain the SHA256 hash of the binary +1. The `describe` command of the binary must return informations that match the plugin's name, namely: + + a. The plugin version must match the name + + b. The API version must match the name's + +1. The version must be canonical: e.g. 1.0.1 (valid) vs. 01.000.01 (invalid) +1. The version must be valid semver (e.g. 1.0.0) +1. Pre-releases are accepted, but must be `-dev` only. +1. Metadata shall not be in the name of the plugin binary, only metadata-free versions will be considered valid for discovery purposes. + +## Plugin Discovery Workflow At initialization, Packer attempts to discover the plugins installed locally. The logic follows what's described in Configuring Packer's @@ -80,31 +108,60 @@ with `PACKER_LOG=1` enabled, where you can find log lines similar to the followi [INFO] Discovered potential plugin: [...] ``` -This logic however is ignored when plugins are defined in `required_plugins` blocks; -instead, for every plugin required in this way, Packer will only consider them if they're -installed in Packer's plugin directory, under a directory hierarchy that matches the -source, with the plugin name respecting a convention. +Once this discovery step has finished, each discovered plugin will be ready to use +by Packer, on the highest available version. -For example, if we install the `github.com/hashicorp/amazon` plugin in version 1.2.8 through -either `packer init` or `packer plugins install`, this will yield the following (in a -Linux x86_64 environment): +Example: ```shell └── github.com └── hashicorp - └── amazon - ├── packer-plugin-amazon_v1.2.8_x5.0_linux_amd64 - └── packer-plugin-amazon_v1.2.8_x5.0_linux_amd64_SHA256SUM + └── happycloud + ├── packer-plugin-happycloud_v1.1.8_x5.0_linux_amd64 + ├── packer-plugin-happycloud_v1.1.8_x5.0_linux_amd64_SHA256SUM + ├── packer-plugin-happycloud_v1.2.8_x5.0_linux_amd64 + └── packer-plugin-happycloud_v1.2.8_x5.0_linux_amd64_SHA256SUM +``` + +In this example, the `happycloud` plugin will be the one and used to serve +happycloud-related components, in version 1.2.8. + +### HCL2 and `required_plugins` + +The aforementioned logic is all that can apply to legacy JSON templates. +HCL2 however introduces the concept of `required_plugins` blocks. +These blocks allow you to declare what plugins your template needs in order +to be able to build an artifact, along with version constraints. + +Example: + +```hcl2 +packer{ + required_plugins { + happycloud = { + source = "github.com/hashicorp/happycloud" + version = "= 1.1.8" + } + } +} ``` -Both the plugin's binary, and the related SHA256SUM file must be placed alongside -each other for Packer to consider them for a `required_plugins` constraint. +With this constraint specified, Packer will not only be able to remotely install +the plugin you need with `packer init` (provided it is hosted on a remote that the command +supports), but also restrict which versions can be used for building the template. + +Using the aforementioned plugin hierarchy, a constraint like the one above will +take precedence over the usual discovery process, this time yielding version 1.1.8 +of the happycloud plugin for serving its components. ## Installation Guides - - +### Packer init + +-> Using `packer init` requires using HCL2 templates. If you are using JSON and + want to start using HCL2 templates in order to leverage this feature, please + refer to our [JSON to HCL](/packer/docs/templates/json_to_hcl) guide. In order to use `packer init` for managing installation of your plugins, there are two steps required. @@ -121,13 +178,13 @@ Here is an example `required_plugins` block: ```hcl packer { required_plugins { - myawesomecloud = { + coolcloud = { version = ">= 2.7.0" - source = "github.com/azr/myawesomecloud" + source = "github.com/hashicorp/coolcloud" } happycloud = { version = ">= 1.1.3" - source = "github.com/azr/happycloud" + source = "github.com/hashicorp/happycloud" } } } @@ -137,65 +194,18 @@ Once your template contains those `required_plugins`, run [`packer init`](/packer/docs/commands/init) to install all missing plugin binaries. Given the above example, Packer will try to look for a GitHub -repository owned by user or organization `azr` named +repository owned by user or organization `hashicorp` named `packer-plugin-myawesomecloud` and `packer-plugin-happycloud`. -## Names and Addresses - -Each plugin has two identifiers: - -- A `source` address, which is where the plugin is downloaded from. -- A unique **local name**, which is used everywhere else in a Packer configuration. - -## Local Names - -Local names allow you to access the components of a plugin and must be unique -per configuration. +-> Note: `packer init` tries to install plugins matching the version constraints required by the template. + Running `packer init` multiple times on the same template will result in no changes if all matching plugins + have already been installed. If you want to update a plugin or force re-installation, you may use the + `--update` or `--force` arguments. -This is best explained using an example. In the above `required_plugins` block, -we declared the local name "myawesomecloud" for the plugin `azr/myawesomecloud`. -If the "myawesomecloud" plugin contains both an "ebs" builder and an "import" -post-processor, then the builder will be accessed in a source block by using: - -```hcl -source "myawesomecloud-ebs" "example" { - // builder configuration... -} -``` - -similarly, the import post-processor would be accessed by declaring the -post-processor block: - -```hcl -post-processor "myawesomecloud-import" { - // post-processor configuration... -} -``` - -If we change the required_plugins block to use a different local name "foo": - -```hcl - required_plugins { - foo = { - version = ">= 2.7.0" - source = "github.com/azr/myawesomecloud" - } - } -``` - -Then we'd instead access that builder using the source: - -```hcl -source "foo-ebs" "example" { - // builder configuration... -} -``` - - - +### Packer plugins install Plugin installation via `packer plugins install` works similar to that of the `packer init` command, but -no `required_plugins` block are required, and thus can be used with both legacy JSON and HCL2 templates. +no `required_plugins` block are required. ```shell packer plugins install github.com/hashicorp/vagrant @@ -210,32 +220,20 @@ The command will install the plugin in the `PACKER_CONFIG_DIR` set, or its default location, which depends on the OS/environment, as documented in [Configuring Packer](/packer/docs/configure#packer-s-plugin-directory). - - - -If you have obtained or built a plugin binary for your OS/Architecture and want to -use it with Packer, you can install it manually. For Packer to load the plugin, -it must be named with the convention `packer-plugin-NAME`, and placed in Packer's plugin -directory, as documented in -[Configuring Packer](/packer/docs/configure#packer-s-plugin-directory). - -For example, if your configuration directory is located in `~/.config/packer`, -you can copy the binary to `~/.config/packer/plugins/packer-plugin-NAME`, and -Packer will be able to load it afterwards. +#### Using packer plugins install to install a local copy of a binary -If you have a `required_plugins` for the plugin you're manually installing, make sure -it respects the constraints described in the [Plugin loading workflow](#plugin-loading-workflow) -section, otherwise Packer will not be able to load it. +If the plugin you want to install cannot be installed remotely, you can use the +`--path` argument for `packer plugins install` in order to use the provided +plugin binary as source. -Starting with v1.10.0 of Packer, you can also use `packer plugins install` with the -`--path` flag to install a plugin from a binary, following the layout that is required to -work with `required_plugins` block. +Example: ```shell -packer plugins install --path github.com/hashicorp/vagrant +$ ls -l packer-plugin-happycloud +-rwxrwxr-x 1 root root 44745210 Jan 01 1979 packer-plugin-happycloud +$ PACKER_PLUGIN_PATH=/root/plugins packer plugins install --path ./packer-plugin-happycloud "github.com/hashicorp/happycloud" +Successfully installed plugin github.com/hashicorp/happycloud from /root/packer-plugin-happycloud to /root/plugins/github.com/hashicorp/happycloud/packer-plugin-happycloud_v1.2.8-dev_x5.0_linux_amd64 ``` --> packer plugins install --path only works with release versions of plugins. - - - +-> Note: the version does not need to be specified, Packer will automatically + determine which version to install based on the plugin's `describe` output.