packer

Commit Graph

Author	SHA1	Message	Date
Lucas Bajolet	434a1637f8	hcl2template: fix func to get vars from a config The previous implementation of the GetVarsByType function worked only on top-level attributes, ignoring the nested blocks in the structure. This implies that if a datasource depends on another through an expression within a nested block, we may not execute it first, and then executing this datasource before its dependent is possible, resulting in an error in the end. This commit is an attempt at making this more reliable for HCL configs, but only works on configs lifted from HCL files for now. We need to make this more reliable for later iterations.	3 years ago
Lucas Bajolet	51ec78622d	hcl2template: simplify startDatasource function	3 years ago
Lucas Bajolet	d9259d6d12	hcl2template: extract attr filter code from ds Datasources use their attribute's expressions to determine whether or not they depend on another datasource, in order to get the list of dependencies and execute them before executing a datasource. This code may be useful later on for figuring out the dependencies for any block, so we move this code to the utils.go file, and use this for datasources.	3 years ago
Lucas Bajolet	14b6ad10fe	hcl2template: report localtion for cycle detection When a datasource fails to be evaluated because a cycle has been detected, we point out one of the links of the chain now so that users have a better idea of what to look at.	3 years ago
Lucas Bajolet	2a6ccaccab	hcl2template: simplify datasource evaluation Since datasources are recursively evaluated depending on their dependencies, we don't need to pre-execute those that depend on nothing, as the recursive traversal of the datasources will take care of that for us.	3 years ago
Lucas Bajolet	8438d81d4d	hcl2template: remove unused shouldContinue bool Not sure why this was defined and returned, but the value was set, but never used, as such this is not useful to keep in the code, so let's simplify this now.	3 years ago
guoguangwu	7eb6a45cd4	chore: remove refs to deprecated io/ioutil Signed-off-by: guoguangwu <guoguangwu@magic-shield.com>	3 years ago
guoguangwu	324e628b86	chore: unnecessary use of fmt.Sprintf Signed-off-by: guoguangwu <guoguangwu@magic-shield.com>	3 years ago
Lucas Bajolet	318529f373	hcl2template: remove value validation for locals Local variables can't have a validation block in their definition, so this step in not useful and should be removed. Besides, since the validation was done on the local variables before evaluation, it did nothing at all, as the PackerConfig.LocalVariables collection gets populated during evaluation, so this is essentially a no-op, and can be safely removed.	3 years ago
Lucas Bajolet	eb9e1a4795	packer: remove implicit required plugins Since this feature is no longer something we plan to activate later, as it contradicts with our efforts to remove bundled plugins, and encouraging users to move to either manually installing plugins, or managing them through `packer init', we clean-up the code for this feature.	3 years ago
hashicorp-copywrite[bot]	19055df3ec	[COMPLIANCE] License changes (#12568 ) * Updating the license from MPL to Business Source License Going forward, this project will be licensed under the Business Source License v1.1. Please see our blog post for more details at https://hashi.co/bsl-blog, FAQ at https://hashi.co/license-faq, and details of the license at www.hashicorp.com/bsl. * Update copyright file headers to BUSL-1.1 --------- Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com>	3 years ago
Wilken Rivera	9ccec0d4d5	Removed repetitive statement from HCP debug logs (#12552 ) * Remove repetitive log statement The log being removed is duplicated each time evalContext is called, which seems like an unneeded debug log.	3 years ago
Lucas Bajolet	a2930bda4f	packer: warn on bundled plugins usage Since bundled plugins will be removed in an upcoming version of Packer, this commit adds a new warning message whenever a template uses one such plugin. This warning has been implemented on build, validate, console and the inspect subcommands. In addition to warning about the upcoming change and potential issue this will cause, this warning message proposes solutions to the user so they know what they'll have to do in order not to rely on those bundled plugins later.	3 years ago
Lucas Bajolet	456bc2397b	hcl2template: only warn once for missing plugins In HCL2, if a template contains a `required_plugins' block, but the required plugins are not installed, we print a warning. Prior to this commit, this was reported as an error once per missing plugin, which was redundant. This commit changes that by gathering all the missing plugins once, and printing them as a bullet list, so the message isn't redundant anymore.	3 years ago
Wilken Rivera	eca75a6847	Run make fmt to fix checks	3 years ago
Lucas Bajolet	f1a41b488f	packer: add HCL2 support for telemetry Packer's telemetry package captures the field names from a configuration on legacy JSON templates, but did not contain any code for handling HCL2 template configurations. This commit adds a routine to extract the field names from a HCL2 configuration once flattened, and adds some glue code to propagate the configurations to the telemetry structures.	3 years ago
hashicorp-copywrite[bot]	b7df3ca36f	[COMPLIANCE] Add Copyright and License Headers (#12254 ) Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com>	3 years ago
Lucas Bajolet	4d559e23b7	hcl2: don't warn on excluded post-processor (#12187 ) * hcl2template: use && for chaining conditions For deciding if we need to run an error provisioner block, we checked using a nested if that we didn't need to avoid running it. This is unnecessary, and adds a superfluous level of inbrication to the code, so we chain both conditions with &&. * hcl2: don't warn on excluded post-processor When a post-processor is excluded through the -except command-line argument for packer build, we used to print a warning on HCL2 templates if this except statement did not match a build. However, since we can use the except option to avoid running post-processors, we should account for this as well when printing out a warning that there were no matches. This commit adds this counter to the except checks for the post-processors as well.	3 years ago
Lucas Bajolet	ca197afe9b	hcp: remove superfluous return value on GetBuilds This commit irons out one of the pain points of the HCP rework by introducing a HCPPublisher interface, implemented both by the JSON Core, and the HCL2 PackerConfig, which keeps a map of the build names used by Packer to the build names pushed on HCP. This in turn lets us go back to the old implementation of the GetBuilds function, which returns a list of (filtered) builds, and eventually an error if something went wrong while processing.	3 years ago
Wilken Rivera	9002ac263f	Update BuildBlock with valid HCL2Ref (#12167 ) While looking into #11932 I found that Packer will throw an error when an unknown source is referenced from within the sources attribute for a build block. The hcl.Diagnostics error includes a Subject, which should highlight where in the HCL2 configuration file the unknown source resides. But when creating the BuildBlock no HCL2Ref data is copied over, thus the Subject is displayed with with zero value for an hcl.Range type. This change updates the build creation logic to copy the HCL2Ref data from the build block at decode time so that it can be properly referenced downstream. Closes #11932 Failure due to change in output ``` --- FAIL: TestValidateCommand_ShowLineNumForMissing (0.00s) --- FAIL: TestValidateCommand_ShowLineNumForMissing/test-fixtures/validate-invalid/missing_build_block.pkr.hcl (0.00s) validate_test.go:377: Unexpected output: ( """ Error: Unknown source file.cho - on line 0: + on test-fixtures/validate-invalid/missing_build_block.pkr.hcl line 6: (source code not available) ... // 4 identical lines """ ) validate_test.go:379: FAIL FAIL github.com/hashicorp/packer/command 1.002s ```	3 years ago
Wilken Rivera	57cbe4e203	core: Update validation options for undeclared variables (#12104 ) * Update validation options for undeclared variables In an effort to help users move from JSON to HCL2 templates the support for variable definitions files are being updated to ignore undeclared variable warnings on build execution. For legacy JSON templates builds no warnings are displayed when var-files contain undeclared variables. Since preferred mode HCL2 templates is to be explicit with variable declarations - they must be declared to be used - validation for undeclared variables still warns when running `packer validate`. A new flag has been added to the validate command that can be used to disable undeclared variable warnings. * Update validation test for unused variables Example Run ``` ~> go run . validate -no-warn-undeclared-var -var-file command/test-fixtures/validate/var-file-tests/undeclared.pkrvars.hcl command/test-fixtures/validate/var-file-tests/basic.pkr.hcl The configuration is valid. ~> go run . validate -var-file command/test-fixtures/validate/var-file-tests/undeclared.pkrvars.hcl command/test-fixtures/validate/var-file-tests/basic.pkr.hcl Warning: Undefined variable The variable "unused" was set but was not declared as an input variable. To declare variable "unused" place this block in one of your .pkr.hcl files, such as variables.pkr.hcl variable "unused" { type = string default = null } The configuration is valid. ~> go run . build -var-file command/test-fixtures/validate/var-file-tests/undeclared.pkrvars.hcl command/test-fixtures/validate/var-file-tests/basic.pkr.hcl file.chocolate: output will be in this color. Build 'file.chocolate' finished after 744 microseconds. ==> Wait completed after 798 microseconds ==> Builds finished. The artifacts of successful builds are: --> file.chocolate: Stored file: chocolate.txt ``` * Rename Strict field to WarnOnUndeclaredVar The field name Strict is a bit vague since it is only used for checking against undeclared variables within a var-file definition. To mitigate against potential overloading of this field it is being renamed to be more explicit on its usage. * command/build: Add warn-on-undeclared-var flag Now that the default behaviour is to not display warnings for undeclared variables an optional flag has been added to toggle the old behaviour. ``` ~> go run . build -warn-on-undeclared-var -var-file command/test-fixtures/validate/var-file-tests/undeclared.pkrvars.hcl command/test-fixtures/validate/var-file-tests/basic.pkr.hcl Warning: Undefined variable The variable "unused" was set but was not declared as an input variable. To declare variable "unused" place this block in one of your .pkr.hcl files, such as variables.pkr.hcl variable "unused" { type = string default = null } file.chocolate: output will be in this color. Build 'file.chocolate' finished after 762 microseconds. ==> Wait completed after 799 microseconds ==> Builds finished. The artifacts of successful builds are: --> file.chocolate: Stored file: chocolate.txt ```	3 years ago
Wilken Rivera	606e6c48f1	internal: reorganise registry/HCP code This commit reorganises the code for both the registry/API and the Orchestrator/Registry. The main difference with the previous version is how stuff is exposed. Now we only expose a Registry interface to the outside (previously named Orchestrator), which has several implementations: null is the default, and is returned if HCP is not enabled. The other implementations being HCL/JSON, both private to the hcp sub-package. The api (previously `registry') is the set of functionality that abstracts and calls the HCP API. It was meant to be merged with the `hcp' package, but because of a dependency loop with the datasources, both are separated for now.	3 years ago
Lucas Bajolet	1cee460d0d	hcp: extract all HCP-related code to hcp package As part of the work to displace everything related to HCP from scattered places around the Packer code, we move it all to an hcp package. This in turn reduces the amount of code that the commands have to integrate, and leaves the HCP details to its own enclave.	3 years ago
Lucas Bajolet	dad07c6097	hcp: keep map of build names for hcp tracking To be able to track HCP builds during a Packer build, we need to propagate the name used for registering the build to HCP. This works out-of-the-box for JSON templates, as the build's name is always the builder's, so there's no confusion on that. For HCP templates however, there's a possibility for a template to define a name for the build, which is returned by a CoreBuild's Name() function, in addition to the source's Type/Name. The builds are registered to HCP with the source.String() function however, which does not contain the build's name. This prevents any build containing a name to work, as their name/source.String does not match. The name we registered the build with is stored in a CoreBuild as the Type attribute, but cannot be safely accessed, as builds are types within the command as packersdk.Build, which only exposes Name() for this purpose. In order to circumvent this problem, and as a way to present a prototype of solution that will likely have to be discussed before we merge it, this commit hotfixes the issue.	3 years ago
Lucas Bajolet	f3e4c2c3bb	tmp: move HCP logic to unwrap builds/post	3 years ago
Lucas Bajolet	61c810e720	command: rework HCP-related error messages The HCP error messages were sometimes a bit terse in terms of the information it bubbles up to the user. This made them useful for people who already knew what to look for, but newcomers would almost certainly be lost because of the lack of information in those. To improve on this situation, we reword most of those error messages in this commit.	4 years ago
Lucas Bajolet	c0e7e7bd3c	hcl2: report error on build without sources When a template describes a build block without a source reference, the build should be considered invalid as we won't have a CoreBuild produced as a result of the need to have both. In current versions of Packer, this will produce an error message hinting that nothing will happen because of the lack of either build or source block. This commit takes the defined block, and points out to it as missing a source block as being the reason why nothing is happening, making it clearer what is required for an HCL2 build to be processed.	4 years ago
Lucas Bajolet	61c56e161c	hcl2: add example for undefined variable set warn When a variable is set in a pkrvars file, but isn't defined, an error message is output, but does not deliver an example of what is expected by Packer in order to complete a build. To remedy that, we improve the error message by giving an example of variable block to include in the build template.	4 years ago
Eng Zer Jun	f5ec0f1c57	test: use `T.Setenv` to set env vars in tests This commit replaces `os.Setenv` with `t.Setenv` in tests. The environment variable is automatically restored to its original value when the test and all its subtests complete. Reference: https://pkg.go.dev/testing#T.Setenv Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>	4 years ago
Lucas Bajolet	a298617ecb	command/build: warn and exit when nothing to build When a `packer build' is run on a config that does not define anything to be built, the process would run normally, and only state that no artifacts were produced after builds were run successfully. This is misleading as we reach the end of the command without having a proper warning that nothing was even attempted to be built. This commit adds a warning when that happens, and leaves immediately after.	4 years ago
Lucas Bajolet	7cd095062a	parser: extract HCP logic out of the HCL parser Before this commit, some HCP-related logic was embedded in the parser, making all of it common to all commands. This complexifies the logic as some parts rely on the template being in a HCP-valid environment, and the datasources need to be evaluated in order for some operations to be executed. To simplify this and avoid those pitfalls, we move this logic to its own set of functions that will be performed after the parsing is done, on any valid HCL2 or JSON template.	4 years ago
Sylvia Moss	5c3d5c1295	Send channel id to HCP Packer (#11861 )	4 years ago
Lucas Bajolet	6ab16561b5	hcl2: fix crash on malformed overrides (#11881 ) When an override block is either not an object, or if one of its contents are not an object, Packer would crash trying to forcefully cast the cty.Value to a cty.Collection. To avoid this behaviour, we add extra checks that return hcl.Diagnostics when they fail.	4 years ago
Lucas Bajolet	6794c6053a	hcl2template: only iterate on known HCP datasource (#11883 ) When we try to validate a build that contains references to hcp_packer_image, the data is not fetched, and the value is therefore unknown. However, during the decoding phase for the build blocks, we attempt to fetch the ancestry information for the current build, from the information previously fetched from HCP. Since we're validating, there's no way this is set, and attempting to cast without checking causes Packer to crash on the conversion. To avoid this, we only attempt this conversion if the value is known. Closes #11870	4 years ago
Wilken Rivera	846e4e6443	Associate Packer build to HCP Packer parent iteration (#11832 ) * Add support for data source lookup * Update failing test * Fix typo for HCP_PACKER_BUILD_FINGERPRINT env in test * Add test Fingerprint where needed; failing tests were not showing because they are properly set for acctest * Update to use BuildContext	4 years ago
Adrien Delorme	e03ad29ca0	Sanitize var code (#11566 ) * evaluateLocalVariables: modify code for readability and some (not benchmarked) perfs * Make default input variable type the DynamicPseudoType This should be the default, and avoids a panic. This type can represent situations where a type is not yet known. Its meaning is undefined in cty. * do not take Empty types from default value * Update types.variables.go Co-authored-by: Wilken Rivera <wilken@hashicorp.com>	4 years ago
Wilken Rivera	2ad15617cc	core/hcl2: Fix issue preventing builds from pausing between provisioners when the `--debug` argument has been passed (#11537 ) * Update HCL2 PackerConfig to call SetDebug mode for CoreBuild when --debug flag is passed ``` ~> packer build --debug /tmp/source.pkr.hcl Debug mode enabled. Builds will not be parallelized. null.example: output will be in this color. ==> null.example: Pausing after run of step 'StepConnect'. Press enter to continue. ==> null.example: Pausing before the next provisioner . Press enter to continue. ==> null.example: Running local shell script: /var/folders/vz/rv7bk6v15211jxg8q801f7kw0000gq/T/packer-shell4149319610 null.example: hi ==> null.example: Pausing before the next provisioner . Press enter to continue. ==> null.example: Running local shell script: /var/folders/vz/rv7bk6v15211jxg8q801f7kw0000gq/T/packer-shell3210691290 null.example: hi 2 ==> null.example: Pausing after run of step 'StepProvision'. Press enter to continue. ==> null.example: Pausing before cleanup of step 'StepProvision'. Press enter to continue. ==> null.example: Pausing before cleanup of step 'StepConnect'. Press enter to continue. Build 'null.example' finished after 8 seconds 284 milliseconds. ==> Wait completed after 8 seconds 284 milliseconds ==> Builds finished. The artifacts of successful builds are: --> null.example: Did not export anything. This is the null builder ``` * Add call to SetOnError, and SetForce	4 years ago
Adrien Delorme	5d17d7fe65	prevent duplicate local block creation (#11534 ) * prevent duplicate local block creation * remove duplicate locals block bug * local variables: first get block, then decode it + simplify retry loop * Update types.packer_config.go * revert go get of hcl lib	4 years ago
teddylear	1d01ad3651	feat: print all locals errors when there is a circular error (#11527 )	4 years ago
Wilken Rivera	760169b34e	Move to using ectx to preload build names (#11432 ) * Add basic post-processor test Closes #11411	4 years ago
Wilken Rivera	4a0fbd911a	Add support for variable interpolation to hcp_packer_registry and build blocks. (#11421 ) * Add variable interpolation support to hcp_packer_registry block * Add tests for variable interpolation in hcp_packer_registry block * Add variable interpolation support to the build block (#11425) * Add simple test to build command	4 years ago
Wilken Rivera	06b35c39de	hcp packer registry build labels (#11401 ) * Add basic support for build_labels argument * Update support for build_labels configuration argument * Update complete test-fixture with a build_labels configuration * Add test for deprecated labels argument * Add deprecation for hcp_packer_registry.labels When using the now deprecated labels argument of the new bucket_labels a Warning will be presented to the user. ``` ~> HCP_PACKER_BUILD_FINGERPRINT=356786543567865456789656789 packer build source.pkr.hcl Warning: the argument hcp_packer_registry.labels has been deprecated and will be removed in a future release; please use hcp_packer_registry.bucket_labels ``` When trying to use both bucket_labels and labels together an error is presented to the user. ``` ~> HCP_PACKER_BUILD_FINGERPRINT=ss6786543567865456789656789 packer build source.pkr.hcl Error: hcp_packer_registry.labels and hcp_packer_registry.bucket_labels are mutely exclusive; please use the recommended argument hcp_packer_registry.bucket_labels on source.pkr.hcl line 17: (source code not available) ``` * Update documentation for build_labels * Apply suggestions from code review Co-authored-by: Adrien Delorme <azr@users.noreply.github.com> * Update hcl2template/types.build.hcp_packer_registry.go Co-authored-by: Adrien Delorme <azr@users.noreply.github.com>	4 years ago
Adrien Delorme	710d109c55	HCL2: fix a crash when an HCL2 component is nil after a misconfiguration (#11382 ) * add test for crash * fix * show more precise error message when parse of a provisioner fails * Revert "fix" This reverts commit `fc88e89fbb`. * add better severity	4 years ago
Megan Marsh	e3719dd8a4	Merge pull request #11319 from hashicorp/add_iteration_id Add iteration	5 years ago
Megan Marsh	b763b79d9f	implement contextual variable packer.iteration_id	5 years ago
Megan Marsh	e7ba4d390d	Merge pull request #11318 from hashicorp/fix_source_accessor fix source accessor to allow use in source block	5 years ago
Megan Marsh	76c1059680	add tests, and simplify code	5 years ago
Adrien Delorme	254e020b08	Allow to use build variables in a post processor (#11323 ) * hcl: add test to verify that build.name can be used in a pp * allow to use build vars in post processors	5 years ago
Adrien Delorme	3135714465	rename all things 'inexist[e\|a]nt' to 'nonexistent' & fix tests (#11325 ) + a typo	5 years ago
Kerim Satirli	bc6f1e36e6	Fixes typos in docs (#11322 ) * fixes typos in base docs * fixes typos in example code * fixes typos in website docs * fixes link to Terraform provider * fixes typo in function name	5 years ago

1 2 3 4 5 ...

311 Commits (98176f54297b3d6fde4dee8aec08c94de71a67fd)