packer

Commit Graph

Author	SHA1	Message	Date
Wilken Rivera	f4522fa783	backport of commit `6c9e653120`	3 years ago
Lucas Bajolet	79e989d03e	hcl2: don't warn on excluded post-processor (#12187 ) * hcl2template: use && for chaining conditions For deciding if we need to run an error provisioner block, we checked using a nested if that we didn't need to avoid running it. This is unnecessary, and adds a superfluous level of inbrication to the code, so we chain both conditions with &&. * hcl2: don't warn on excluded post-processor When a post-processor is excluded through the -except command-line argument for packer build, we used to print a warning on HCL2 templates if this except statement did not match a build. However, since we can use the except option to avoid running post-processors, we should account for this as well when printing out a warning that there were no matches. This commit adds this counter to the except checks for the post-processors as well.	3 years ago
Lucas Bajolet	b2a98664c9	hcp: remove superfluous return value on GetBuilds This commit irons out one of the pain points of the HCP rework by introducing a HCPPublisher interface, implemented both by the JSON Core, and the HCL2 PackerConfig, which keeps a map of the build names used by Packer to the build names pushed on HCP. This in turn lets us go back to the old implementation of the GetBuilds function, which returns a list of (filtered) builds, and eventually an error if something went wrong while processing.	3 years ago
Wilken Rivera	76f7922288	Update BuildBlock with valid HCL2Ref (#12167 ) While looking into #11932 I found that Packer will throw an error when an unknown source is referenced from within the sources attribute for a build block. The hcl.Diagnostics error includes a Subject, which should highlight where in the HCL2 configuration file the unknown source resides. But when creating the BuildBlock no HCL2Ref data is copied over, thus the Subject is displayed with with zero value for an hcl.Range type. This change updates the build creation logic to copy the HCL2Ref data from the build block at decode time so that it can be properly referenced downstream. Closes #11932 Failure due to change in output ``` --- FAIL: TestValidateCommand_ShowLineNumForMissing (0.00s) --- FAIL: TestValidateCommand_ShowLineNumForMissing/test-fixtures/validate-invalid/missing_build_block.pkr.hcl (0.00s) validate_test.go:377: Unexpected output: ( """ Error: Unknown source file.cho - on line 0: + on test-fixtures/validate-invalid/missing_build_block.pkr.hcl line 6: (source code not available) ... // 4 identical lines """ ) validate_test.go:379: FAIL FAIL github.com/hashicorp/packer/command 1.002s ```	3 years ago
Wilken Rivera	3b9274aa04	command/validate: Add `-evaluate-datasources` flag to command help text (#12152 ) ``` Usage: packer validate [options] TEMPLATE Checks the template is valid by parsing the template and also checking the configuration with the various builders, provisioners, etc. If it is not valid, the errors will be shown and the command will exit with a non-zero exit status. If it is valid, it will exit with a zero exit status. Options: -syntax-only Only check syntax. Do not verify config of the template. -except=foo,bar,baz Validate all builds other than these. -only=foo,bar,baz Validate only these builds. -machine-readable Produce machine-readable output. -var 'key=value' Variable for templates, can be used multiple times. -var-file=path JSON or HCL2 file containing user variables, can be used multiple times. -no-warn-undeclared-var Disable warnings for user variable files containing undeclared variables. -evaluate-datasources Evaluate data sources during validation (HCL2 only, may incur costs); Defaults to false. ```	3 years ago
Wilken Rivera	dbeeab448a	cmd/hcl2_upgrade: Generate variable block for all referenced user input variables (#12136 ) Currently the HCL2 upgrade command generates a valid variable block for all variables within the JSON variables property. However JSON templates in Packer support variable interpolation for input variables defined within a variable definition file without it being declared within the variables property. When upgrading a JSON template to HCL2 the user variable reference gets converted to var.<var_name>. The upgraded template ultimately fails on validation or build execution with undefined variable errors, with this change the upgrade command will now create a variable block for all input variables referenced within the build template to ensure all required variables been defined.	3 years ago
Lucas Bajolet	bb2384c56a	command/validate: add option to eval datasources When packer validate is invoked, it does not try to evaluate the datasources before attempting to decide if the template is valid. In many cases, this works, but sometimes it will fail as the value is unknown by the validation code. Since the validation code for all the elements of a Packer template is left to be implemented by plugins, we cannot rely on checking for unknown values everywhere, especially since the unknown references are replaced automatically by a value of the right type for the configuration expected. So, in order for such configurations to be validable, we add an extra option to packer validate, that will let users evaluate the datasources from a template.	3 years ago
Wilken Rivera	57cbe4e203	core: Update validation options for undeclared variables (#12104 ) * Update validation options for undeclared variables In an effort to help users move from JSON to HCL2 templates the support for variable definitions files are being updated to ignore undeclared variable warnings on build execution. For legacy JSON templates builds no warnings are displayed when var-files contain undeclared variables. Since preferred mode HCL2 templates is to be explicit with variable declarations - they must be declared to be used - validation for undeclared variables still warns when running `packer validate`. A new flag has been added to the validate command that can be used to disable undeclared variable warnings. * Update validation test for unused variables Example Run ``` ~> go run . validate -no-warn-undeclared-var -var-file command/test-fixtures/validate/var-file-tests/undeclared.pkrvars.hcl command/test-fixtures/validate/var-file-tests/basic.pkr.hcl The configuration is valid. ~> go run . validate -var-file command/test-fixtures/validate/var-file-tests/undeclared.pkrvars.hcl command/test-fixtures/validate/var-file-tests/basic.pkr.hcl Warning: Undefined variable The variable "unused" was set but was not declared as an input variable. To declare variable "unused" place this block in one of your .pkr.hcl files, such as variables.pkr.hcl variable "unused" { type = string default = null } The configuration is valid. ~> go run . build -var-file command/test-fixtures/validate/var-file-tests/undeclared.pkrvars.hcl command/test-fixtures/validate/var-file-tests/basic.pkr.hcl file.chocolate: output will be in this color. Build 'file.chocolate' finished after 744 microseconds. ==> Wait completed after 798 microseconds ==> Builds finished. The artifacts of successful builds are: --> file.chocolate: Stored file: chocolate.txt ``` * Rename Strict field to WarnOnUndeclaredVar The field name Strict is a bit vague since it is only used for checking against undeclared variables within a var-file definition. To mitigate against potential overloading of this field it is being renamed to be more explicit on its usage. * command/build: Add warn-on-undeclared-var flag Now that the default behaviour is to not display warnings for undeclared variables an optional flag has been added to toggle the old behaviour. ``` ~> go run . build -warn-on-undeclared-var -var-file command/test-fixtures/validate/var-file-tests/undeclared.pkrvars.hcl command/test-fixtures/validate/var-file-tests/basic.pkr.hcl Warning: Undefined variable The variable "unused" was set but was not declared as an input variable. To declare variable "unused" place this block in one of your .pkr.hcl files, such as variables.pkr.hcl variable "unused" { type = string default = null } file.chocolate: output will be in this color. Build 'file.chocolate' finished after 762 microseconds. ==> Wait completed after 799 microseconds ==> Builds finished. The artifacts of successful builds are: --> file.chocolate: Stored file: chocolate.txt ```	3 years ago
Wilken Rivera	606e6c48f1	internal: reorganise registry/HCP code This commit reorganises the code for both the registry/API and the Orchestrator/Registry. The main difference with the previous version is how stuff is exposed. Now we only expose a Registry interface to the outside (previously named Orchestrator), which has several implementations: null is the default, and is returned if HCP is not enabled. The other implementations being HCL/JSON, both private to the hcp sub-package. The api (previously `registry') is the set of functionality that abstracts and calls the HCP API. It was meant to be merged with the `hcp' package, but because of a dependency loop with the datasources, both are separated for now.	3 years ago
Lucas Bajolet	1cee460d0d	hcp: extract all HCP-related code to hcp package As part of the work to displace everything related to HCP from scattered places around the Packer code, we move it all to an hcp package. This in turn reduces the amount of code that the commands have to integrate, and leaves the HCP details to its own enclave.	3 years ago
Lucas Bajolet	dad07c6097	hcp: keep map of build names for hcp tracking To be able to track HCP builds during a Packer build, we need to propagate the name used for registering the build to HCP. This works out-of-the-box for JSON templates, as the build's name is always the builder's, so there's no confusion on that. For HCP templates however, there's a possibility for a template to define a name for the build, which is returned by a CoreBuild's Name() function, in addition to the source's Type/Name. The builds are registered to HCP with the source.String() function however, which does not contain the build's name. This prevents any build containing a name to work, as their name/source.String does not match. The name we registered the build with is stored in a CoreBuild as the Type attribute, but cannot be safely accessed, as builds are types within the command as packersdk.Build, which only exposes Name() for this purpose. In order to circumvent this problem, and as a way to present a prototype of solution that will likely have to be discussed before we merge it, this commit hotfixes the issue.	3 years ago
Lucas Bajolet	f3e4c2c3bb	tmp: move HCP logic to unwrap builds/post	3 years ago
Lucas Bajolet	68ddb2f89d	hcl2_upgrade: add share_image_gallery workaround (#12087 ) In azure templates, the shared_image_gallery was mistakenly considered an attribute while this is supposed to be a block. This is due to a heuristic we use for deciding whether a JSON object is to be translated to an attribute or a block that fell short as the shared_image_gallery does not contain complex types. This cannot be fixed trivially for the general case, so we add this entity to the list of workarounds until we can implement something more robust.	3 years ago
Jenna Goldstrich	0bac9b6f63	Remove a skip for darwin arm64 (m1/m2 macs) for exec tests. (#12088 )	3 years ago
Lucas Bajolet	296562614f	hcl2_upgrade: fix liniting issues with fmt.Sprintf Some literal strings without format-string directives nor arguments were littered around the hcl2_upgrade command's code, which is pointed out by the linters. Since they serve no purpose, we remove them.	3 years ago
Lucas Bajolet	3c249c0d42	hcl2_upgrade: replace passthroughs by pre-visit In previous versions of hcl2_upgrade, we referenced a list of passthrough statements from the original JSON template to be replaced by their equivalent Go-template in HCL2. This works but requires us to explicitely accept every possible templated variable in order to not ever encounter <no value> in our generated HCL2 templates. This is suboptimal, hence this commit changes approach by pre-visting the AST from the original Go Template, escaping every sequence that is not covered by the list of functions we can migrate. Pipes are also excluded from this function.	3 years ago
Wilken Rivera	7cb0c98925	Merge pull request #12033 from hashicorp/error_messages_rework [HPR-575] command/build: rework HCP-related error messages	3 years ago
Zach Shilton	879fd9035c	fix: update redirected and broken links (#12060 ) * website: fix broken links on /docs/templates * fix: redirected install-plugins link * fix: debugging link * fix: secrets manager link in docs * fix: secrets manager link in source * fix: amazon ami plugin link in docs * fix: amazon ami plugin link in source * fix: extending plugins link * fix: plugins/builders/amazon links * fix: various builders links * fix: various amazon builder links * fix: redirected terminology link * fix: custom-provisioners link * fix: docker-push redirected plugin link * fix: googlecompute plugin links * fix: hyperv iso plugin links * website: update link to hcl upgrade guide Co-authored-by: Wilken Rivera <wilken@hashicorp.com> Co-authored-by: Wilken Rivera <wilken@hashicorp.com>	3 years ago
Lucas Bajolet	61c810e720	command: rework HCP-related error messages The HCP error messages were sometimes a bit terse in terms of the information it bubbles up to the user. This made them useful for people who already knew what to look for, but newcomers would almost certainly be lost because of the lack of information in those. To improve on this situation, we reword most of those error messages in this commit.	3 years ago
Lucas Bajolet	35496e533d	packer: include builder type in HCP name for JSON JSON templates used only to report the builder's type in HCP builds, even if the name was specified. This commit changes this behaviour to include both if they're available, in a similar fashion as what is done on the HCL2 templates.	3 years ago
Wilken Rivera	12e9316f94	Add documenation for HCPConfigMode type	3 years ago
Wilken Rivera	148d5e3a2d	Add initialize method to Iteration Fingerprint initialize was previously occurring during the creation of a bucket. We want to be able to initialize a bucket and defer the setting of a fingerprint to a later point. * Update test to reflect new function signatures for Bucket and Iteration	3 years ago
Wilken Rivera	7f7f397124	Rename withHCLBucketConfiguration function	3 years ago
Wilken Rivera	a4a97d5c05	Move datasource data into its own bucketConfiguratioOpts func	3 years ago
Wilken Rivera	cf9304266e	Consolidate bucket configuration logic	3 years ago
Lucas Bajolet	b3188637bf	registry: load bucket slug in command/registry Loadgin the slug from the environment within the Bucket implied that the Bucket was successfully created, which may fail outside of a Git directory when the iteration fingerprint is not set through environment variables. To make it possible to print as much information regarding the environment to the users at once rather than stopping immediately when this step fails, we move the logic to read the default value from the environment to the HCP setup code.	3 years ago
Lucas Bajolet	9267d933bc	command: report multiple errors in bad HCP config When HCP is detected to be enabled, but some configuration is missing, we returned immediately on the first error. This commit changes this behaviour by reporting every error at once, so users will know immediately if something is wrong when they invoke Packer with HCP support, and one or more environment variables is not defined as we'd expect them.	3 years ago
Lucas Bajolet	ae15ed339c	registry: add function to detect explicit HCP The HCP_PACKER_REGISTRY environment variable had its behaviour changed recently, as prior versions of Packer expected the attribute to be forcefully set to a value that is neither "off" nor "0" in order to get HCP integration to work, or that a "hcp_packer_registry" block was defined in an HCL template. Now, this environment variable defaults to not explicitely enable, but instead to explicitely disable HCP integration, and the feature switch fall upon HCP_PACKER_BUCKET_NAME. As an extra feature, we keep the prior behaviour alive when it is explicitely defined as a value to enable it. That way we can report errors if the rest is not defined, rather than silently ignore it. This function we add to env is the first stone to enable this behaviour.	3 years ago
Lucas Bajolet	c0e7e7bd3c	hcl2: report error on build without sources When a template describes a build block without a source reference, the build should be considered invalid as we won't have a CoreBuild produced as a result of the need to have both. In current versions of Packer, this will produce an error message hinting that nothing will happen because of the lack of either build or source block. This commit takes the defined block, and points out to it as missing a source block as being the reason why nothing is happening, making it clearer what is required for an HCL2 build to be processed.	3 years ago
Lucas Bajolet	61c56e161c	hcl2: add example for undefined variable set warn When a variable is set in a pkrvars file, but isn't defined, an error message is output, but does not deliver an example of what is expected by Packer in order to complete a build. To remedy that, we improve the error message by giving an example of variable block to include in the build template.	3 years ago
Lucas Bajolet	73af77aeed	command: enhance error message on no build to run	3 years ago
Lucas Bajolet	e94b8766cb	build: exit immediately if no builds and diags When a template with some builds to run ends its GetBuilds with an error, and no builds produced, we can exit immediately without printing more errors.	3 years ago
Lucas Bajolet	3f6c0a5f23	command: move func to extract stdout/err in tests In order to test the output from a test command, the commandOutput function exists in the command_test.go file. Since its behaviour is linked to the test meta that we produce in the test_utils.go file, we move it there, and rename it to `GetStdoutAndErrFromTestMeta' to make it clearer what to expect from it.	3 years ago
Eng Zer Jun	f5ec0f1c57	test: use `T.Setenv` to set env vars in tests This commit replaces `os.Setenv` with `t.Setenv` in tests. The environment variable is automatically restored to its original value when the test and all its subtests complete. Reference: https://pkg.go.dev/testing#T.Setenv Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>	3 years ago
Lucas Bajolet	779a31a25f	hcp: use HCP_PACKER_REGISTRY to disable HCP The current behaviour for HCP integration is based on the presence of the HCP_PACKER_REGISTRY environment variable, where if it is either not "0" or "off", the HCP integration is enabled. This commit changes the behaviour to only use this variable to explicitely disable HCP, and the HCP_PACKER_BUCKET_NAME variable will condition if the HCP integration is enabled for a build or not in JSON templates. For HCL templates, the integration can also be disabled if the HCP_PACKER_REGISTRY variable is set to "0" or "off", and will be considered enabled if the HCP_PACKER_BUCKET_NAME is set, or there is a "hcp_packer_registry" block in a build.	3 years ago
Wilken Rivera	11e71729f1	Remove Oracle plugin from the list of vendored plugins (#11983 ) The latest release of the Oracle plugin for Packer removed Solaris from its list of supported platforms. Since Packer still releases binaries for Solaris the Oracle plugin will no longer be bundled and distributed within the Packer binary. Practitioners relying on the plugin can continue using the plugin by installing it manually using either the `packer init` or `packer plugins install` commands.	3 years ago
Lucas Bajolet	a298617ecb	command/build: warn and exit when nothing to build When a `packer build' is run on a config that does not define anything to be built, the process would run normally, and only state that no artifacts were produced after builds were run successfully. This is misleading as we reach the end of the command without having a proper warning that nothing was even attempted to be built. This commit adds a warning when that happens, and leaves immediately after.	3 years ago
Lucas Bajolet	7cd095062a	parser: extract HCP logic out of the HCL parser Before this commit, some HCP-related logic was embedded in the parser, making all of it common to all commands. This complexifies the logic as some parts rely on the template being in a HCP-valid environment, and the datasources need to be evaluated in order for some operations to be executed. To simplify this and avoid those pitfalls, we move this logic to its own set of functions that will be performed after the parsing is done, on any valid HCL2 or JSON template.	4 years ago
Lucas Bajolet	4be7e38474	command: move GetConfig methods to meta Since the GetConfig methods are common to all the commands of Packer that rely on a config to work, we move them from build.go to meta.go in order to make the intent clearer.	4 years ago
Wilken Rivera	e3ad29d2aa	Update third-party plugin imports (#11912 ) * packer-plugin-digitalocean: Update module import path * packer-plugin-outscale: Remove bundled plugin * packer-plugin-scaleway: Remove bundled plugin	4 years ago
teddylear	805225a113	feat: add http datasource (#11658 )	4 years ago
Wilken Rivera	a7c37ee34d	Add tests for the packer plugins subcommand (#11660 ) * Add tests for plugins install command * Add tests for plugins remove command * Add PACKER_GITHUB_API_TOKEN to all test env * Update dir hashes for non-darwin os * Update tests with feedback from review	4 years ago
Wilken Rivera	1c398a73b4	Set default file extension for binary removal on Windows (#11627 ) Related to #11625	4 years ago
Wilken Rivera	ea3c73a480	Set default file extension for binary installations on Windows (#11625 ) This change sets the default file extension for binary installations on Windows. Results before change ``` $ packer plugins install github.com/hashicorp/amazon 9 errors occurred: * ignoring invalid remote binary packer-plugin-amazon_v1.0.8_x5.0_freebsd_arm64.zip: wrong system, expected windows_amd64 * ignoring invalid remote binary packer-plugin-amazon_v1.0.8_x5.0_darwin_amd64.zip: wrong system, expected windows_amd64 * ignoring invalid remote binary packer-plugin-amazon_v1.0.8_x5.0_freebsd_amd64.zip: wrong system, expected windows_amd64 * ignoring invalid remote binary packer-plugin-amazon_v1.0.8_x5.0_linux_amd64.zip: wrong system, expected windows_amd64 * ignoring invalid remote binary packer-plugin-amazon_v1.0.8_x5.0_darwin_arm64.zip: wrong system, expected windows_amd64 * ignoring invalid remote binary packer-plugin-amazon_v1.0.8_x5.0_windows_arm64.zip: wrong system, expected windows_amd64 * ignoring invalid remote binary packer-plugin-amazon_v1.0.8_x5.0_windows_386.zip: wrong system, expected windows_amd64 * ignoring invalid remote binary packer-plugin-amazon_v1.0.8_x5.0_linux_arm.zip: wrong system, expected windows_amd64 * could not find a packer-plugin-amazon_v1.0.8_x5.0_windows_amd64.zip file in zipfile exit status 1 ``` Results after change ``` $ packer plugins install github.com/hashicorp/amazon Installed plugin github.com/hashicorp/amazon v1.0.8 in "C:/Users/Packer/AppData/Roaming/packer.d/plugins/github.com/hashicorp/amazon/packer-plugin-amazon_v1.0.8_x5.0_windows_amd64.exe" ```	4 years ago
Adrien Delorme	0362a3ec10	all shells: Pass env vars through a key value store (#11569 ) * allowing to set shell env vars from a key/value map. * tests This new map will work alongside the `environment_vars` but it allows using a datasource value seamlessly. At validation, and because `environment_vars` was an array of strings containing `KEY=value`, using datsources would not work, because these values are not known yet and would evaluate to `<unknown>`. With this, the value and the key can be unknown and will validate correctly for datasources.	4 years ago
Adrien Delorme	e03ad29ca0	Sanitize var code (#11566 ) * evaluateLocalVariables: modify code for readability and some (not benchmarked) perfs * Make default input variable type the DynamicPseudoType This should be the default, and avoids a panic. This type can represent situations where a type is not yet known. Its meaning is undefined in cty. * do not take Empty types from default value * Update types.variables.go Co-authored-by: Wilken Rivera <wilken@hashicorp.com>	4 years ago
Adrien Delorme	9f4a1281dd	Add `packer plugins` command and subcommands to interact with plugins (#11553 ) * add basic docs for plugins command * refactor docs Co-Authored-By: Wilken Rivera <1749304+nywilken@users.noreply.github.com> * add plugins command * add plugins subcommands they do nothing for now * add plugins installed command + tests * add plugins install command * add remove plugin command * better docs for the plugins install command * remove duplicate content * better output for installed plugins * add plugins required command * Update plugins_install.go * add newline after `Usage:` * Update plugins_remove.go * Update plugins_required.go * Update plugins_remove.go * Update plugins_installed.go * Update plugins_install.go * add docs * Update plugins_install.go * fix typos * Update plugins_test.go * fix typos Co-Authored-By: Wilken Rivera <1749304+nywilken@users.noreply.github.com> * Update core_wrapper.go Co-Authored-By: Wilken Rivera <1749304+nywilken@users.noreply.github.com> * Update website/content/docs/commands/plugins/remove.mdx Co-authored-by: Wilken Rivera <wilken@hashicorp.com> * Update website/content/docs/commands/plugins/required.mdx Co-authored-by: Wilken Rivera <wilken@hashicorp.com> * Update website/content/docs/commands/plugins/required.mdx Co-authored-by: Wilken Rivera <wilken@hashicorp.com> * Update plugins_required.go * Update install.mdx * Update required.mdx * plugins requirement, warn when no plugin was found * Update website/content/docs/commands/plugins/required.mdx Co-authored-by: Wilken Rivera <wilken@hashicorp.com> Co-authored-by: Wilken Rivera <1749304+nywilken@users.noreply.github.com> Co-authored-by: Wilken Rivera <wilken@hashicorp.com>	4 years ago
teddylear	1d01ad3651	feat: print all locals errors when there is a circular error (#11527 )	4 years ago
Wilken Rivera	58fa1f96e2	Temporarily revert the use of indirect redirects for external plugins (#11430 ) The auto installation of external plugins via packer init only work for HCL templates. Since we have not officially removed all plugins (slated for 1.8.0) I am reverting this test change. Once 1.7.9 is released we can begin removing plugins and optimizing the redirect experience for JSON-only users.	4 years ago
Adrien Delorme	3278c1a817	Packer SVC acc tests (#11274 )	4 years ago

1 2 3 4 5 ...

734 Commits (v1.8.6)