From 9986834a6260875e6c9d396a0ece62a23343a4a2 Mon Sep 17 00:00:00 2001 From: Wilken Rivera Date: Tue, 9 Feb 2021 12:13:26 -0500 Subject: [PATCH 1/6] website/commands/init: Add blurb on plugin selection This change adds text around how plugin selection for plugins defined in a require_plugins block work. It also adds a small call out to users that third party plugins are not verified by HashiCorp. --- website/content/docs/commands/init.mdx | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/website/content/docs/commands/init.mdx b/website/content/docs/commands/init.mdx index 89a9b0864..57a9813fa 100644 --- a/website/content/docs/commands/init.mdx +++ b/website/content/docs/commands/init.mdx @@ -8,20 +8,20 @@ sidebar_title: init # `init` Command -> **Note:** Packer init does not work with legacy JSON templates. You can -upgrade your JSON config files to HCL using the hcl2ugprade command. +upgrade your JSON config files to HCL using the [hcl2_upgrade](/docs/commands/hcl2_upgrade) command. -> **Note:** Packer init will only work with multiplugins -- that is plugins -that are named `packer-plugin-*` -- to install a single plugin binary -- that is +that are named `packer-plugin-*`. To install a single plugin binary -- that is `packer-provisioner-*`, `packer-builder-*`, etc. -- nothing changes, you will have to [install the plugin manually](/docs/plugins#installing-plugins). The `packer init` command is used to download Packer plugin binaries. This is the first command that should be executed when working with a new or existing template. This command is always safe to run multiple times. Though subsequent -runs may give errors, this command will never delete anything. +runs may give errors, this command will never delete anything. Packer does not currently have the notion of a state like Terraform has. In other words, -currently `packer init` is only in charge of installing packer plugins. +currently `packer init` is only in charge of installing packer plugins. Currently, `packer init` can only fetch binaries from public projects on **GitHub**. GitHub's public API, [limits the number of unauthenticated requests per hour one IP can @@ -53,6 +53,13 @@ packer { } ``` +HashiCorp does not officially verify third party Packer plugins, plugins not under the HashiCorp namespace `hashicorp/*`; as with all open source tools, please do your own due diligence when using a new tool. + +## Plugin Selection +Plugin selection depends on the source and version constraints defined within the `required_plugins` block. +For each of the required plugins Packer will query the source repository `azr/myawesomecloud` whose fully qualified address +is `https://github.com/azr/packer-plugin-myawesomecloud` for a plugin matching the version constraints for the host Os. + Packer init will install the latest found version matching the version selection in the `required_plugins` section. Make sure to set a correct [version constraint @@ -60,9 +67,11 @@ string](/docs/templates/hcl_templates/blocks/packer#version-constraints). The plugins will be installed in the [Plugin Directory](/docs/configure#packer-s-plugin-directory). - +See [Installing Plugins](/docs/plugins#installing-plugins) for more information on how plugin installation works. ## Options - `-upgrade` - On top of installing missing plugins, update installed plugins to the latest available version, if there is a new higher one. Note that this still takes into consideration the version constraint of the config. + + From 25b2ab90824426072ac0bb4cbf63c57381d4c156 Mon Sep 17 00:00:00 2001 From: Wilken Rivera Date: Tue, 9 Feb 2021 14:23:44 -0500 Subject: [PATCH 2/6] guides/1-7-plugin-upgrade: Update migration guide * Add text to call out the packer-sdk-migrator tool. * Add text around the naming conventions for registering multi-plugins --- .../guides/1.7-plugin-upgrade/index.mdx | 29 ++++++++++++++----- 1 file changed, 21 insertions(+), 8 deletions(-) diff --git a/website/content/guides/1.7-plugin-upgrade/index.mdx b/website/content/guides/1.7-plugin-upgrade/index.mdx index e011df7c7..2f56ac92d 100644 --- a/website/content/guides/1.7-plugin-upgrade/index.mdx +++ b/website/content/guides/1.7-plugin-upgrade/index.mdx @@ -18,11 +18,12 @@ Packer is currently backwards compatible with the old API because the plugin API ### How to update plugins to use the packer-plugin-SDK -In a best-case scenario, all you'll have to do is update the packer imports to use the packer-plugin-sdk import path +In a best-case scenario, all you'll have to do is update the packer imports to use the packer-plugin-sdk import path. Replacing `github.com/hashicorp/packer` with `github.com/hashicorp/packer-plugin-sdk`. -`github.com/hashicorp/packer` with `github.com/hashicorp/packer-plugin-sdk`. +To help with the migration of existing plugins the [packer-sdk-migrator](https://github.com/hashicorp/packer-sdk-migrator#packer-sdk-migrator) CLI tool is available to automatically rewrite all of the import paths to use to `packer-plugin-sdk` module. The migration CLI provides an [eligibility check](https://github.com/hashicorp/packer-sdk-migrator#packer-sdk-migrator-check-check-eligibility-for-migration) that can be used to determine if a plugin can be migrated with the tool. -But some of the import paths have changed more than that because we've refactored the SDK some to make it easier to discover and use helpful modules. Below are a few common import paths. For a full list of available imports see [Packer Plugin SDK Docs](https://pkg.go.dev/github.com/hashicorp/packer-plugin-sdk/) + +With the creation of the SDK the import paths may not map directly to the paths within Packer core, as they have been refactored to make it easier to discover and use helpful modules. Below are a few common import paths. For a full list of available imports see [Packer Plugin SDK Docs](https://pkg.go.dev/github.com/hashicorp/packer-plugin-sdk/) | Old Path | New Path | @@ -111,10 +112,11 @@ func main() { ``` -With this single-plugin binary you'd install it by putting it into the plugin directory with the name `packer-provisioner-foo`. +With this single-plugin binary you'd install it by putting it into the plugin directory with the name +`packer-provisioner-foo`. -Now, you'll want to use the NewSet() function to create a server. so, +In the new multi-plugin binary you'll want to use the NewSet() function to create a server. ```golang package main @@ -138,11 +140,22 @@ func main() { ``` -So the implementation is extremely similar, but now we use the "NewSet" function, and call Run() instead of Serve(). +The implementation is extremely similar, but now we use the "NewSet" function to create the plugin Server, and call Run() instead of Serve(). + +With the multi-plugin binary example, it is important to not that the name of the binary must be in the form of `packer-pluign-bar`. Otherwise Packer will not register it as a multiple plugin. + +```go +go build -o packer-plugin-bar +``` + +Then you'd install it by putting it into the plugin directory with the name `packer-plugin-bar`. + +With the multi-plugin binary you would then refer to your newly installed plugin as `bar-foo` within your Packer template. Whereas, in the single binary setup, you'd have used the name "bar" in your Packer template. + +At this point it is important to note that multi-plugins are to Packer, what providers are to Terraform. Multi-plugin binaries are meant to provide one or more plugin types to Packer. See [Registering multiple plugins](#registering-multiple-plugins) -With this example, you'd install it by putting it into the plugin directory with the name `packer-plugin-bar`. +If your plugin providers a single plugin type please continue reading for instruction on using the Set architecture for single plugins. -In the single binary setup, this you'd have used the name "bar" in your packer template. Now, you need to use the name "bar-foo" in the template. ## Using the Set Architecture for Single Plugins From bb03a321f256ca512e3437a7fcb4b479b3fdc264 Mon Sep 17 00:00:00 2001 From: Wilken Rivera Date: Tue, 9 Feb 2021 15:14:29 -0500 Subject: [PATCH 3/6] Add distributing migrated plugins section --- website/content/guides/1.7-plugin-upgrade/index.mdx | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/website/content/guides/1.7-plugin-upgrade/index.mdx b/website/content/guides/1.7-plugin-upgrade/index.mdx index 2f56ac92d..defcc089a 100644 --- a/website/content/guides/1.7-plugin-upgrade/index.mdx +++ b/website/content/guides/1.7-plugin-upgrade/index.mdx @@ -152,7 +152,7 @@ Then you'd install it by putting it into the plugin directory with the name `pac With the multi-plugin binary you would then refer to your newly installed plugin as `bar-foo` within your Packer template. Whereas, in the single binary setup, you'd have used the name "bar" in your Packer template. -At this point it is important to note that multi-plugins are to Packer, what providers are to Terraform. Multi-plugin binaries are meant to provide one or more plugin types to Packer. See [Registering multiple plugins](#registering-multiple-plugins) +At this point it is important to note that multi-plugins are to Packer, what providers are to Terraform. Multi-plugin binaries are meant to provide one or more plugin types to Packer. See [registering multiple plugins](#registering-multiple-plugins) for details on how to register more than one plugin type. If your plugin providers a single plugin type please continue reading for instruction on using the Set architecture for single plugins. @@ -200,4 +200,7 @@ And then reference your plugin in the Packer template using "vsphere-iso" and "v You can still use the `DEFAULT_NAME` constant with a set implementation; in this example, whatever you register with the DEFAULT_NAME will be referenced in the Packer template as "vsphere". You can only register one plugin per plugin set using the DEFAULT_NAME -We will soon write a follow-up guide explaining how to register you new plugin set with HashiCorp. +## Distributing migrated plugins +Once a plugin has been migrated to use the `packer-plugin-sdk` it can be released as it normally would and used by Packer by [installing the plugin](/docs/plugins#installing-plugins) manually into the respective Packer plugin directory. + +If however, you would like to take advantage of Packer's automated plugin installation process via `packer init` -- the preferred method of installation -- you will need to make the plugin available on GitHub and in a repository named after the multi-plugin `https://github.com//packer-plugin-`. See [Creating a GitHub Release](/docs/plugins/creation#creating-a-github-release) for details on the recommended practice for releasing Packer plugins on GitHub. From 29beb051c5ce4cf271b29f36b3ce5a7366071ef5 Mon Sep 17 00:00:00 2001 From: Wilken Rivera Date: Tue, 9 Feb 2021 15:37:03 -0500 Subject: [PATCH 4/6] Fix borken URL --- website/content/docs/builders/community-supported.mdx | 2 +- website/content/docs/plugins/creation/custom-datasources.mdx | 4 ++-- website/content/docs/post-processors/community-supported.mdx | 2 +- website/content/docs/provisioners/community-supported.mdx | 2 +- website/content/docs/templates/hcl_templates/datasources.mdx | 2 +- 5 files changed, 6 insertions(+), 6 deletions(-) diff --git a/website/content/docs/builders/community-supported.mdx b/website/content/docs/builders/community-supported.mdx index 8f9fc26e3..a2d64c0a8 100644 --- a/website/content/docs/builders/community-supported.mdx +++ b/website/content/docs/builders/community-supported.mdx @@ -10,6 +10,6 @@ sidebar_title: Community-Supported The following builders are developed and maintained by various members of the Packer community, not by HashiCorp. For more information on how to use community -builders, see our docs on [extending Packer](/docs/extending). +builders, see our docs on [extending Packer](/docs/plugins/creation). @include 'builders/community_builders.mdx' diff --git a/website/content/docs/plugins/creation/custom-datasources.mdx b/website/content/docs/plugins/creation/custom-datasources.mdx index 8ef063ae5..b9f1055fd 100644 --- a/website/content/docs/plugins/creation/custom-datasources.mdx +++ b/website/content/docs/plugins/creation/custom-datasources.mdx @@ -10,10 +10,10 @@ sidebar_title: Custom Data Sources Packer Data Sources are the components of Packer that allow data to be fetched for use within the configuration. Use of data sources allows a build to use information defined outside of Packer. An example of -data source is the [amazon-ami data source](/docs/datasources/amazon-ami), which outputs the data of a fetched Amazon AMI. +data source is the [amazon-ami data source](/docs/datasources/amazon/ami), which outputs the data of a fetched Amazon AMI. Prior to reading this page, it is assumed you have read the page on [plugin -development basics](/docs/extending/plugins). +development basics](/docs/plugins). Data Source plugins implement the `packersdk.Datasource` interface and are registered within a plugin Set with `set.RegisterDatasource(...)` function and served using the `set.Run()`. diff --git a/website/content/docs/post-processors/community-supported.mdx b/website/content/docs/post-processors/community-supported.mdx index 4db4e556d..e0b2bbeb3 100644 --- a/website/content/docs/post-processors/community-supported.mdx +++ b/website/content/docs/post-processors/community-supported.mdx @@ -12,6 +12,6 @@ sidebar_title: Community-Supported The following post-processors are developed and maintained by various members of the Packer community, not by HashiCorp. For more information on how to use community -post-processors, see our docs on [extending Packer](/docs/extending). +post-processors, see our docs on [extending Packer](/docs/plugins/creation). @include 'post-processors/community_post-processors.mdx' diff --git a/website/content/docs/provisioners/community-supported.mdx b/website/content/docs/provisioners/community-supported.mdx index 1be85c0f6..c1b58a53d 100644 --- a/website/content/docs/provisioners/community-supported.mdx +++ b/website/content/docs/provisioners/community-supported.mdx @@ -10,6 +10,6 @@ sidebar_title: Community-Supported The following provisioners are developed and maintained by various members of the Packer community, not by HashiCorp. For more information on how to use community -provisioners, see our docs on [extending Packer](/docs/extending). +provisioners, see our docs on [extending Packer](/docs/plugins/creation). @include 'provisioners/community_provisioners.mdx' diff --git a/website/content/docs/templates/hcl_templates/datasources.mdx b/website/content/docs/templates/hcl_templates/datasources.mdx index f0a6359bb..6c6dc4eec 100644 --- a/website/content/docs/templates/hcl_templates/datasources.mdx +++ b/website/content/docs/templates/hcl_templates/datasources.mdx @@ -79,4 +79,4 @@ source "amazon-ebs" "basic-example" { - The list of available data sources can be found in the [data sources](/docs/datasources) section. -- Create your own [custom data source](/docs/extending/custom-datasources) ! +- Create your own [custom data source](/docs/plugins/creation/custom-datasources) ! From 926e287025ce19ae4035a17205c9eebbad98b961 Mon Sep 17 00:00:00 2001 From: Wilken Rivera Date: Wed, 10 Feb 2021 10:33:01 -0500 Subject: [PATCH 5/6] Apply suggestions from code review Co-authored-by: Megan Marsh Co-authored-by: Adrien Delorme --- website/content/guides/1.7-plugin-upgrade/index.mdx | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/website/content/guides/1.7-plugin-upgrade/index.mdx b/website/content/guides/1.7-plugin-upgrade/index.mdx index defcc089a..73ef48002 100644 --- a/website/content/guides/1.7-plugin-upgrade/index.mdx +++ b/website/content/guides/1.7-plugin-upgrade/index.mdx @@ -142,17 +142,17 @@ func main() { The implementation is extremely similar, but now we use the "NewSet" function to create the plugin Server, and call Run() instead of Serve(). -With the multi-plugin binary example, it is important to not that the name of the binary must be in the form of `packer-pluign-bar`. Otherwise Packer will not register it as a multiple plugin. +With the multi-plugin binary example, it is important to note that the name of the binary must be in the form of `packer-pluign-name`. Otherwise Packer will not register it as a multi-plugin, nor to download it. ```go go build -o packer-plugin-bar ``` -Then you'd install it by putting it into the plugin directory with the name `packer-plugin-bar`. +Then you'd install it by putting it into the plugin directory with the name `packer-plugin-name`. With the multi-plugin binary you would then refer to your newly installed plugin as `bar-foo` within your Packer template. Whereas, in the single binary setup, you'd have used the name "bar" in your Packer template. -At this point it is important to note that multi-plugins are to Packer, what providers are to Terraform. Multi-plugin binaries are meant to provide one or more plugin types to Packer. See [registering multiple plugins](#registering-multiple-plugins) for details on how to register more than one plugin type. +At this point it is important to note that multi-plugins are to Packer, what providers are to Terraform. Multi-plugin binaries are meant to provide one or more plugin types to Packer which depend on the same cloud or other APIs. See [registering multiple plugins](#registering-multiple-plugins) for details on how to register more than one plugin type. If your plugin providers a single plugin type please continue reading for instruction on using the Set architecture for single plugins. From 4e2d2961b6bc5df6b03ba1be8788c51e59ee783b Mon Sep 17 00:00:00 2001 From: Wilken Rivera Date: Wed, 10 Feb 2021 10:37:24 -0500 Subject: [PATCH 6/6] Update with suggestion from review --- website/content/docs/commands/init.mdx | 2 +- website/content/guides/1.7-plugin-upgrade/index.mdx | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/website/content/docs/commands/init.mdx b/website/content/docs/commands/init.mdx index 57a9813fa..2e631a931 100644 --- a/website/content/docs/commands/init.mdx +++ b/website/content/docs/commands/init.mdx @@ -58,7 +58,7 @@ HashiCorp does not officially verify third party Packer plugins, plugins not und ## Plugin Selection Plugin selection depends on the source and version constraints defined within the `required_plugins` block. For each of the required plugins Packer will query the source repository `azr/myawesomecloud` whose fully qualified address -is `https://github.com/azr/packer-plugin-myawesomecloud` for a plugin matching the version constraints for the host Os. +is `https://github.com/azr/packer-plugin-myawesomecloud` for a plugin matching the version constraints for the host operating system. Packer init will install the latest found version matching the version selection in the `required_plugins` section. Make sure to set a correct [version diff --git a/website/content/guides/1.7-plugin-upgrade/index.mdx b/website/content/guides/1.7-plugin-upgrade/index.mdx index 73ef48002..0fde46b34 100644 --- a/website/content/guides/1.7-plugin-upgrade/index.mdx +++ b/website/content/guides/1.7-plugin-upgrade/index.mdx @@ -152,7 +152,7 @@ Then you'd install it by putting it into the plugin directory with the name `pac With the multi-plugin binary you would then refer to your newly installed plugin as `bar-foo` within your Packer template. Whereas, in the single binary setup, you'd have used the name "bar" in your Packer template. -At this point it is important to note that multi-plugins are to Packer, what providers are to Terraform. Multi-plugin binaries are meant to provide one or more plugin types to Packer which depend on the same cloud or other APIs. See [registering multiple plugins](#registering-multiple-plugins) for details on how to register more than one plugin type. +At this point it is important to note that multi-plugins are to Packer, what providers are to Terraform. Multi-plugin binaries provide one or more plugin types to Packer which depend on the same cloud or other APIs. See [registering multiple plugins](#registering-multiple-plugins) for details on how to register more than one plugin type. If your plugin providers a single plugin type please continue reading for instruction on using the Set architecture for single plugins. @@ -203,4 +203,4 @@ You can still use the `DEFAULT_NAME` constant with a set implementation; in this ## Distributing migrated plugins Once a plugin has been migrated to use the `packer-plugin-sdk` it can be released as it normally would and used by Packer by [installing the plugin](/docs/plugins#installing-plugins) manually into the respective Packer plugin directory. -If however, you would like to take advantage of Packer's automated plugin installation process via `packer init` -- the preferred method of installation -- you will need to make the plugin available on GitHub and in a repository named after the multi-plugin `https://github.com//packer-plugin-`. See [Creating a GitHub Release](/docs/plugins/creation#creating-a-github-release) for details on the recommended practice for releasing Packer plugins on GitHub. +If however, you would like to take advantage of Packer's automated plugin installation process via `packer init` -- the preferred method of installation -- you will need to make the plugin available on GitHub and in a repository named after the multi-plugin `https://github.com//packer-plugin-name`. See [Creating a GitHub Release](/docs/plugins/creation#creating-a-github-release) for details on the recommended practice for releasing Packer plugins on GitHub.