755 Commits (ebdee64c643ae5ad22c4853ec6c64731bae5f310)

Author	SHA1	Message	Date
Lucas Bajolet	eb9e1a4795	packer: remove implicit required plugins ... Since this feature is no longer something we plan to activate later, as it contradicts with our efforts to remove bundled plugins, and encouraging users to move to either manually installing plugins, or managing them through `packer init', we clean-up the code for this feature.	3 years ago
hashicorp-copywrite[bot]	19055df3ec	[COMPLIANCE] License changes (#12568) ... * Updating the license from MPL to Business Source License Going forward, this project will be licensed under the Business Source License v1.1. Please see our blog post for more details at https://hashi.co/bsl-blog, FAQ at https://hashi.co/license-faq, and details of the license at www.hashicorp.com/bsl. * Update copyright file headers to BUSL-1.1 --------- Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com>	3 years ago
Wilken Rivera	99c79ecbd6	Add build tags to skip amd64 tests on darwin/arm64 (#12549)	3 years ago
Lucas Bajolet	3512a175a6	command: change wording for init's postitional arg ... `packer init' is meant to be invoked on a template file or a directory. This was described in the command-line help message as `[template.pkr.hcl|folder]'. Because this is encapsulated between square brackets, this could be taken as an option instead of a positional, mandatory argument. This commit changes the wording to adopt a similar convention as `packer build', so the two are consistent in how they are invoked, and to avoid mistaking this argument as optional.	3 years ago
Lucas Bajolet	d0198c8a17	packer: track bundled plugins usage on telemetry	3 years ago
Lucas Bajolet	b522edca36	hcl2_upgrade: add required_plugins in out template ... When a legacy JSON template references built-in plugins, we add them to the resulting template as a `required_plugins' block in HCL2.	3 years ago
Lucas Bajolet	a2930bda4f	packer: warn on bundled plugins usage ... Since bundled plugins will be removed in an upcoming version of Packer, this commit adds a new warning message whenever a template uses one such plugin. This warning has been implemented on build, validate, console and the inspect subcommands. In addition to warning about the upcoming change and potential issue this will cause, this warning message proposes solutions to the user so they know what they'll have to do in order not to rely on those bundled plugins later.	3 years ago
Wilken Rivera	8982d4f168	Migrate external Parallels plugin to Parallels/packer-plugin-parallels (#12476) ... * Update external plugin documentation source The Parallels plugin for Packer is now maintained by the Parallels team, under their respective GitHub org. This changes updates the source address for the external plugin that should be used for pulling new plugin documentation. * Removed packer-plugin-parallels for list of vendored plugins	3 years ago
Wilken Rivera	d67bacc8d3	Remove community plugins from the vendored plugins list (#12436) ... This step removes community plugins from the Packer binary release. These plugins are being maintained and released independently of Packer and have become out of date. To ensure users are using the latest version of the plugins they are being removed in favor of the external plugin installation processes.	3 years ago
Wilken Rivera	06fe087097	Merge pull request #12380 from hashicorp/bump-go-1.20 ... Bump Go tooling to version 1.20	3 years ago
Wilken Rivera	65f6428fce	Remove external archived provisioner plugins (#12374) ... This change removes a set of plugins that have not been updated in a while. These plugins have been archived for some time now. Users wishing to continue using these plugins should use the `packer plugins install` or `packer init` commands to install the external plugin.	3 years ago
Wilken Rivera	13a24db0de	Remove IONOS plugins from vendored plugins (#12385) ... * Remove profitbricks plugin * Remove oneandone plugin The following plugins have been unmaintained for some time now, and their upstream cloud provider has consolidate the services. These plugins will continue to be available to Packer via direct installation using packer init or the packer plugins install command. But they will no longer be bundled with Packer.	3 years ago
Wilken Rivera	eca75a6847	Run make fmt to fix checks	3 years ago
Wilken Rivera	ddd7ee98b6	Bump go.mod to 1.20 ... * Fix go vet issue * Bump Go tooling to version 1.20 * Update workflows to use go-version file	3 years ago
Wilken Rivera	08d1dfe36c	Remove external plugin for Digital Ocean (#12376) ... The Digital Ocean plugin is now maintained and released by the Digital Ocean team. To ensure users are downloading the latest available version the plugin will nolonger be bundled with Packer.	3 years ago
Lucas Bajolet	31921f0c9f	packer: track template type in spans ... The template type for a builder, provisioner, post-processor or datasource was not tracked with telemetry data. This commit adds the infrastructure to track this, bumping the schema for non-crashes to `beta/packer/6' at the same time.	3 years ago
Wilken Rivera	e0a5286645	cmd/console: Add config-type flag to command help (#12360) ... It is not clear that the console command runs in legacy JSON mode by default and that users can specify the -config-type flag to toggle HCL2 mode. This change adds the config-type flag to the command's help text to alert users of the gotcha. In the next minor Packer release the console command will be updated to default to HCL2 mode. Relates to: #12359 Relates to: #10603	3 years ago
Wilken Rivera	f678c06396	Migrate external UCloud plugin to ucloud/packer-plugin-ucloud (#12335) ... * Update external plugin documentation source The UCloud plugin for Packer is now maintained by the UCloud team, under their respective GitHub org. This changes updates the source address for the external plugin that should be used for pulling new plugin documentation. * Remove packer-plugin-ucloud from vendored plugins.	3 years ago
Wilken Rivera	20c2d48df7	Migrate external Linode plugin to linode/packer-plugin-linode (#12329) ... * Update external plugin documentation source The Linode plugin for Packer is now maintained by the Linode team, under their respective GitHub org. This changes updates the source address for the external plugin that should be used for pulling new plugin documentation. * Removed packer-plugin-linode for list of vendored plugins The Linode plugin for Packer is now maintained by the Linode team, under their respective GitHub org. This changes removes Linode from the vendored plugins lists.	3 years ago
hashicorp-copywrite[bot]	b7df3ca36f	[COMPLIANCE] Add Copyright and License Headers (#12254) ... Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com>	3 years ago
Wilken Rivera	095959ed6d	Fix crash when variables block is undefined ... This change fixes a crash that occurs when trying to add undeclared variables into an empty variables map. The variables block when defined will always be non-nil but in the case where variables are declared and initialized from a legacy JSON variables file the map must be initialized first.	3 years ago
Lucas Bajolet	ec1d2e68f5	hcp: generate fingerprints on each new build ... Fingerprints are how we link a packer build to an iteration on HCP. These are computed automatically from the Git SHA in the current state, and are unique to the bucket/iteration. The main problem with this approach is that while sound in theory, it quickly falls apart when users want to run the same build configuration twice, but expect a new image to be created. With the current model, this fails, as the iteration with the current SHA already exists. While this is solvable through environment variables, or by committing a change to the repository, we think this is not clear enough, and causes an extra step to what should otherwise be a simple process. Therefore, to lower the barrier of entry into HCP, we change this behaviour with this commit. Now, fingerprints are randomly generated ULIDs instead of a git SHA, and a new one is always generated, unless one is already specified in the environment. This makes continuation of an existing iteration a conscious choice rather than something automatic, and virtually eliminates conflicts such as the ones described above.	3 years ago
Lucas Bajolet	4d559e23b7	hcl2: don't warn on excluded post-processor (#12187) ... * hcl2template: use && for chaining conditions For deciding if we need to run an error provisioner block, we checked using a nested if that we didn't need to avoid running it. This is unnecessary, and adds a superfluous level of inbrication to the code, so we chain both conditions with &&. * hcl2: don't warn on excluded post-processor When a post-processor is excluded through the -except command-line argument for packer build, we used to print a warning on HCL2 templates if this except statement did not match a build. However, since we can use the except option to avoid running post-processors, we should account for this as well when printing out a warning that there were no matches. This commit adds this counter to the except checks for the post-processors as well.	3 years ago
Lucas Bajolet	ca197afe9b	hcp: remove superfluous return value on GetBuilds ... This commit irons out one of the pain points of the HCP rework by introducing a HCPPublisher interface, implemented both by the JSON Core, and the HCL2 PackerConfig, which keeps a map of the build names used by Packer to the build names pushed on HCP. This in turn lets us go back to the old implementation of the GetBuilds function, which returns a list of (filtered) builds, and eventually an error if something went wrong while processing.	3 years ago
Wilken Rivera	9002ac263f	Update BuildBlock with valid HCL2Ref (#12167) ... While looking into #11932 I found that Packer will throw an error when an unknown source is referenced from within the sources attribute for a build block. The hcl.Diagnostics error includes a Subject, which should highlight where in the HCL2 configuration file the unknown source resides. But when creating the BuildBlock no HCL2Ref data is copied over, thus the Subject is displayed with with zero value for an hcl.Range type. This change updates the build creation logic to copy the HCL2Ref data from the build block at decode time so that it can be properly referenced downstream. Closes #11932 Failure due to change in output ``` --- FAIL: TestValidateCommand_ShowLineNumForMissing (0.00s) --- FAIL: TestValidateCommand_ShowLineNumForMissing/test-fixtures/validate-invalid/missing_build_block.pkr.hcl (0.00s) validate_test.go:377: Unexpected output: ( """ Error: Unknown source file.cho - on line 0: + on test-fixtures/validate-invalid/missing_build_block.pkr.hcl line 6: (source code not available) ... // 4 identical lines """ ) validate_test.go:379: FAIL FAIL github.com/hashicorp/packer/command 1.002s ```	3 years ago
Wilken Rivera	3b9274aa04	command/validate: Add `-evaluate-datasources` flag to command help text (#12152) ... ``` Usage: packer validate [options] TEMPLATE Checks the template is valid by parsing the template and also checking the configuration with the various builders, provisioners, etc. If it is not valid, the errors will be shown and the command will exit with a non-zero exit status. If it is valid, it will exit with a zero exit status. Options: -syntax-only Only check syntax. Do not verify config of the template. -except=foo,bar,baz Validate all builds other than these. -only=foo,bar,baz Validate only these builds. -machine-readable Produce machine-readable output. -var 'key=value' Variable for templates, can be used multiple times. -var-file=path JSON or HCL2 file containing user variables, can be used multiple times. -no-warn-undeclared-var Disable warnings for user variable files containing undeclared variables. -evaluate-datasources Evaluate data sources during validation (HCL2 only, may incur costs); Defaults to false. ```	3 years ago
Wilken Rivera	dbeeab448a	cmd/hcl2_upgrade: Generate variable block for all referenced user input variables (#12136) ... Currently the HCL2 upgrade command generates a valid variable block for all variables within the JSON variables property. However JSON templates in Packer support variable interpolation for input variables defined within a variable definition file without it being declared within the variables property. When upgrading a JSON template to HCL2 the user variable reference gets converted to var.<var_name>. The upgraded template ultimately fails on validation or build execution with undefined variable errors, with this change the upgrade command will now create a variable block for all input variables referenced within the build template to ensure all required variables been defined.	3 years ago
Lucas Bajolet	bb2384c56a	command/validate: add option to eval datasources ... When packer validate is invoked, it does not try to evaluate the datasources before attempting to decide if the template is valid. In many cases, this works, but sometimes it will fail as the value is unknown by the validation code. Since the validation code for all the elements of a Packer template is left to be implemented by plugins, we cannot rely on checking for unknown values everywhere, especially since the unknown references are replaced automatically by a value of the right type for the configuration expected. So, in order for such configurations to be validable, we add an extra option to packer validate, that will let users evaluate the datasources from a template.	3 years ago
Wilken Rivera	57cbe4e203	core: Update validation options for undeclared variables (#12104) ... * Update validation options for undeclared variables In an effort to help users move from JSON to HCL2 templates the support for variable definitions files are being updated to ignore undeclared variable warnings on build execution. For legacy JSON templates builds no warnings are displayed when var-files contain undeclared variables. Since preferred mode HCL2 templates is to be explicit with variable declarations - they must be declared to be used - validation for undeclared variables still warns when running `packer validate`. A new flag has been added to the validate command that can be used to disable undeclared variable warnings. * Update validation test for unused variables Example Run ``` ~> go run . validate -no-warn-undeclared-var -var-file command/test-fixtures/validate/var-file-tests/undeclared.pkrvars.hcl command/test-fixtures/validate/var-file-tests/basic.pkr.hcl The configuration is valid. ~> go run . validate -var-file command/test-fixtures/validate/var-file-tests/undeclared.pkrvars.hcl command/test-fixtures/validate/var-file-tests/basic.pkr.hcl Warning: Undefined variable The variable "unused" was set but was not declared as an input variable. To declare variable "unused" place this block in one of your .pkr.hcl files, such as variables.pkr.hcl variable "unused" { type = string default = null } The configuration is valid. ~> go run . build -var-file command/test-fixtures/validate/var-file-tests/undeclared.pkrvars.hcl command/test-fixtures/validate/var-file-tests/basic.pkr.hcl file.chocolate: output will be in this color. Build 'file.chocolate' finished after 744 microseconds. ==> Wait completed after 798 microseconds ==> Builds finished. The artifacts of successful builds are: --> file.chocolate: Stored file: chocolate.txt ``` * Rename Strict field to WarnOnUndeclaredVar The field name Strict is a bit vague since it is only used for checking against undeclared variables within a var-file definition. To mitigate against potential overloading of this field it is being renamed to be more explicit on its usage. * command/build: Add warn-on-undeclared-var flag Now that the default behaviour is to not display warnings for undeclared variables an optional flag has been added to toggle the old behaviour. ``` ~> go run . build -warn-on-undeclared-var -var-file command/test-fixtures/validate/var-file-tests/undeclared.pkrvars.hcl command/test-fixtures/validate/var-file-tests/basic.pkr.hcl Warning: Undefined variable The variable "unused" was set but was not declared as an input variable. To declare variable "unused" place this block in one of your .pkr.hcl files, such as variables.pkr.hcl variable "unused" { type = string default = null } file.chocolate: output will be in this color. Build 'file.chocolate' finished after 762 microseconds. ==> Wait completed after 799 microseconds ==> Builds finished. The artifacts of successful builds are: --> file.chocolate: Stored file: chocolate.txt ```	3 years ago
Wilken Rivera	606e6c48f1	internal: reorganise registry/HCP code ... This commit reorganises the code for both the registry/API and the Orchestrator/Registry. The main difference with the previous version is how stuff is exposed. Now we only expose a Registry interface to the outside (previously named Orchestrator), which has several implementations: null is the default, and is returned if HCP is not enabled. The other implementations being HCL/JSON, both private to the hcp sub-package. The api (previously `registry') is the set of functionality that abstracts and calls the HCP API. It was meant to be merged with the `hcp' package, but because of a dependency loop with the datasources, both are separated for now.	3 years ago
Lucas Bajolet	1cee460d0d	hcp: extract all HCP-related code to hcp package ... As part of the work to displace everything related to HCP from scattered places around the Packer code, we move it all to an hcp package. This in turn reduces the amount of code that the commands have to integrate, and leaves the HCP details to its own enclave.	3 years ago
Lucas Bajolet	dad07c6097	hcp: keep map of build names for hcp tracking ... To be able to track HCP builds during a Packer build, we need to propagate the name used for registering the build to HCP. This works out-of-the-box for JSON templates, as the build's name is always the builder's, so there's no confusion on that. For HCP templates however, there's a possibility for a template to define a name for the build, which is returned by a CoreBuild's Name() function, in addition to the source's Type/Name. The builds are registered to HCP with the source.String() function however, which does not contain the build's name. This prevents any build containing a name to work, as their name/source.String does not match. The name we registered the build with is stored in a CoreBuild as the Type attribute, but cannot be safely accessed, as builds are types within the command as packersdk.Build, which only exposes Name() for this purpose. In order to circumvent this problem, and as a way to present a prototype of solution that will likely have to be discussed before we merge it, this commit hotfixes the issue.	3 years ago
Lucas Bajolet	f3e4c2c3bb	tmp: move HCP logic to unwrap builds/post	3 years ago
Lucas Bajolet	68ddb2f89d	hcl2_upgrade: add share_image_gallery workaround (#12087) ... In azure templates, the shared_image_gallery was mistakenly considered an attribute while this is supposed to be a block. This is due to a heuristic we use for deciding whether a JSON object is to be translated to an attribute or a block that fell short as the shared_image_gallery does not contain complex types. This cannot be fixed trivially for the general case, so we add this entity to the list of workarounds until we can implement something more robust.	3 years ago
Jenna Goldstrich	0bac9b6f63	Remove a skip for darwin arm64 (m1/m2 macs) for exec tests. (#12088)	3 years ago
Lucas Bajolet	296562614f	hcl2_upgrade: fix liniting issues with fmt.Sprintf ... Some literal strings without format-string directives nor arguments were littered around the hcl2_upgrade command's code, which is pointed out by the linters. Since they serve no purpose, we remove them.	3 years ago
Lucas Bajolet	3c249c0d42	hcl2_upgrade: replace passthroughs by pre-visit ... In previous versions of hcl2_upgrade, we referenced a list of passthrough statements from the original JSON template to be replaced by their equivalent Go-template in HCL2. This works but requires us to explicitely accept every possible templated variable in order to not ever encounter <no value> in our generated HCL2 templates. This is suboptimal, hence this commit changes approach by pre-visting the AST from the original Go Template, escaping every sequence that is not covered by the list of functions we can migrate. Pipes are also excluded from this function.	3 years ago
Wilken Rivera	7cb0c98925	Merge pull request #12033 from hashicorp/error_messages_rework ... [HPR-575] command/build: rework HCP-related error messages	3 years ago
Zach Shilton	879fd9035c	fix: update redirected and broken links (#12060) ... * website: fix broken links on /docs/templates * fix: redirected install-plugins link * fix: debugging link * fix: secrets manager link in docs * fix: secrets manager link in source * fix: amazon ami plugin link in docs * fix: amazon ami plugin link in source * fix: extending plugins link * fix: plugins/builders/amazon links * fix: various builders links * fix: various amazon builder links * fix: redirected terminology link * fix: custom-provisioners link * fix: docker-push redirected plugin link * fix: googlecompute plugin links * fix: hyperv iso plugin links * website: update link to hcl upgrade guide Co-authored-by: Wilken Rivera <wilken@hashicorp.com> Co-authored-by: Wilken Rivera <wilken@hashicorp.com>	3 years ago
Lucas Bajolet	61c810e720	command: rework HCP-related error messages ... The HCP error messages were sometimes a bit terse in terms of the information it bubbles up to the user. This made them useful for people who already knew what to look for, but newcomers would almost certainly be lost because of the lack of information in those. To improve on this situation, we reword most of those error messages in this commit.	3 years ago
Lucas Bajolet	35496e533d	packer: include builder type in HCP name for JSON ... JSON templates used only to report the builder's type in HCP builds, even if the name was specified. This commit changes this behaviour to include both if they're available, in a similar fashion as what is done on the HCL2 templates.	3 years ago
Wilken Rivera	12e9316f94	Add documenation for HCPConfigMode type	3 years ago
Wilken Rivera	148d5e3a2d	Add initialize method to Iteration ... Fingerprint initialize was previously occurring during the creation of a bucket. We want to be able to initialize a bucket and defer the setting of a fingerprint to a later point. * Update test to reflect new function signatures for Bucket and Iteration	3 years ago
Wilken Rivera	7f7f397124	Rename withHCLBucketConfiguration function	3 years ago
Wilken Rivera	a4a97d5c05	Move datasource data into its own bucketConfiguratioOpts func	3 years ago
Wilken Rivera	cf9304266e	Consolidate bucket configuration logic	3 years ago
Lucas Bajolet	b3188637bf	registry: load bucket slug in command/registry ... Loadgin the slug from the environment within the Bucket implied that the Bucket was successfully created, which may fail outside of a Git directory when the iteration fingerprint is not set through environment variables. To make it possible to print as much information regarding the environment to the users at once rather than stopping immediately when this step fails, we move the logic to read the default value from the environment to the HCP setup code.	3 years ago
Lucas Bajolet	9267d933bc	command: report multiple errors in bad HCP config ... When HCP is detected to be enabled, but some configuration is missing, we returned immediately on the first error. This commit changes this behaviour by reporting every error at once, so users will know immediately if something is wrong when they invoke Packer with HCP support, and one or more environment variables is not defined as we'd expect them.	3 years ago
Lucas Bajolet	ae15ed339c	registry: add function to detect explicit HCP ... The HCP_PACKER_REGISTRY environment variable had its behaviour changed recently, as prior versions of Packer expected the attribute to be forcefully set to a value that is neither "off" nor "0" in order to get HCP integration to work, or that a "hcp_packer_registry" block was defined in an HCL template. Now, this environment variable defaults to not explicitely enable, but instead to explicitely disable HCP integration, and the feature switch fall upon HCP_PACKER_BUCKET_NAME. As an extra feature, we keep the prior behaviour alive when it is explicitely defined as a value to enable it. That way we can report errors if the rest is not defined, rather than silently ignore it. This function we add to env is the first stone to enable this behaviour.	3 years ago
Lucas Bajolet	c0e7e7bd3c	hcl2: report error on build without sources ... When a template describes a build block without a source reference, the build should be considered invalid as we won't have a CoreBuild produced as a result of the need to have both. In current versions of Packer, this will produce an error message hinting that nothing will happen because of the lack of either build or source block. This commit takes the defined block, and points out to it as missing a source block as being the reason why nothing is happening, making it clearer what is required for an HCL2 build to be processed.	3 years ago