66 Commits (karthik/powershell_encoding)

Author	SHA1	Message	Date
Karthik P	8e36b92ce3	Preserve User-Defined Variable File Order (#13350) ... * modifying the for loop to preserver user inputted variable files preference * adding test cases * pr comments + refactoring the for loop for better readability. * pr comments + refactoring the for loop for better readability. * pr comments | handling default case * adding additional test case * fixing test cases * fixing test cases. creating a subdirectory to add a test case for both hcl and json auto var files * adding test case for json auto var file	10 months ago
Martin Grogan	5e6e1a2217	hcl: add get GetHCPPackerRegistry method to PackerConfig ... This method will be used to validate and fetch the correct HCPPackerRefistry block to be used in HCP build hcl: check for multiple HCP Packer registry block hcl: support for top level HCP Packer Registry hcl: deprecation warning for build block based hcp config	12 months ago
Martin Grogan	86c8902b81	hcl: add packer_registry at root schema	12 months ago
Gustavo Cruz	7f64ca11f6	fix: `packer validate` unsupported type error ... `packer validate` would output the same error message four times per unsupported root block type found in a template (e.g., 'src' instead of 'source'). This behavior was due to a function being called four times for each file on each stage of the parsing.	1 year ago
Lucas Bajolet	8d6b8da996	hcl2template: locals evaluation returns a variable ... Evaluating local variables used to be directly written to the PackerConfig while each variable was created. This was somewhat of an issue with testing, as we have a bunch of tests that relied on `PackerConfig.Variables` being set only when we actually write something. This is not really a concern for normal use, just for testing, but to limit the number of changes to the tests in hcl2template, I opted to change how variables' values are retained, so that evaluating a single variable returns a Variable in addition to hcl.Diagnostics, so we can reify the approach and only create the map of variables if there's something evaluated.	1 year ago
Lucas Bajolet	d28982c29a	hcl2template: fix datasource dependency detection ... When preparing the datasources to add into the DAG for evaluating the build prerequisites, we ended-up in a weird situation in which the datasources for each vertex pointed to the same one. This is because of the loop semantics of Go, where the same object is reused over and over again during each loop, so in the end every datasource vertex pointed to the same instance of a datasource block. To avoid this, we instead grab them through their reference, making the reference to the datasource purely local, and pointing to the actual datasource block, not the one scoped to the function.	1 year ago
Lucas Bajolet	0dcb8c02c2	hcl2template: don't use Walk for walking on DAG ... Walk uses a reverse topological order to walk on the graph, doing that visit concurrently if possible. This is nice as we can speed-up execution of datasources and locals, however since the `Variables` map stored in the config, and the production of the context for it, are not meant to be used concurrently, this means that we end-up in cases where Packer crashes because of concurrent accesses to that map. So until we can change this behaviour, we will fallback to using the sequential visit algorithm for those vertexes, therefore limiting the risk of those conflicts.	1 year ago
Lucas Bajolet	4a4b837386	hcl2template: rename `Name`->`LocalName` for local ... Local variables had an attribute called Name with the name of the local variable. However, when producing an error while walking the DAG of local/datasources, if an error is encountered during validation, the raw structure of the vertex was printed out, making the error message produced hard to understand. Therefore in order to clean it up, we rename the `Name` attribute for Local variables as `LocalName`, and introduce a `Name()` function for that block so that the complete name of the variable is clearly reported.	1 year ago
Lucas Bajolet	09774aaeb8	internal/dag: adapt Validate to not check for Root ... The implementation of the DAG as extracted from Terraform relied on a Root vertex being injected into the graph as the last node to visit. This is used as a sanity check for Terraform, but doesn't apply to our use-case for now, as we are always executing everything and have no need for this root node. Instead, we change how Validate operates so it does not error in case there is no valid root node for the graph, but enables us calling it to check for self-referencing edges, and circular dependencies.	1 year ago
Lucas Bajolet	586762564b	hcl2template: intro and add UseSequential init opt ... Following up on the DAG work, this commit adds a new option for initialisation that disables DAG on request. By default we are going to use the DAG approach, with an option to fallback to using the older algorithm for evaluation in case users end-up in an edge-case that prevents them from building a template.	1 year ago
Lucas Bajolet	56e08c2eff	hcl2template: add DAG-based eval for local/data ... As we have finished setting-up the codebase for it, this commit adds the logic that uses the internal DAG package, and is able to orchestrate evaluation of datasources and locals in a non-phased way. Instead, this code acts by first detecting the dependencies for those components, builds a graph from them, with edges representing the dependency links between them, and finally walking on the graph breadth-first to evaluate those components. This can act as a drop-in replacement for the current phased logic, but both should be supported until we are confident that the approach works, and that there are little to no bugs left to squash.	1 year ago
Lucas Bajolet	36e43e30ee	hcl2template: detect duplicate locals during parse ... Previously duplicate detection for local variables happened during `Initialise`, through a call to `checkForDuplicateLocalDefinition`. This works in a majority of cases, but for commands like `console`, this was not detected as the return diagnostics for `Initialise` are ignored. That check can be done as early as during parsing however, as the names of blocks are not dynamic in the slightest (no interpolation possible), so we move that detection logic into `Parse`, so that the behaviour is coherent between all commands.	2 years ago
Lucas Bajolet	93bec31ec9	version: embed version/VERSION as version string ... Since we have both version/version.go and version/VERSION to specify version strings, both are a bit redundant. As version/VERSION is supposed to be the source of truth now, we are using it to derive the version informaiton we used to rely on in Packer and its subcommands. Note: doing this prevents us from changing the version/prerelease through ldflags though as we derive Version/VersionPrerelease from the rawVersion variable.	2 years ago
Lucas Bajolet	318529f373	hcl2template: remove value validation for locals ... Local variables can't have a validation block in their definition, so this step in not useful and should be removed. Besides, since the validation was done on the local variables before evaluation, it did nothing at all, as the PackerConfig.LocalVariables collection gets populated during evaluation, so this is essentially a no-op, and can be safely removed.	2 years ago
Lucas Bajolet	eb9e1a4795	packer: remove implicit required plugins ... Since this feature is no longer something we plan to activate later, as it contradicts with our efforts to remove bundled plugins, and encouraging users to move to either manually installing plugins, or managing them through `packer init', we clean-up the code for this feature.	3 years ago
hashicorp-copywrite[bot]	19055df3ec	[COMPLIANCE] License changes (#12568) ... * Updating the license from MPL to Business Source License Going forward, this project will be licensed under the Business Source License v1.1. Please see our blog post for more details at https://hashi.co/bsl-blog, FAQ at https://hashi.co/license-faq, and details of the license at www.hashicorp.com/bsl. * Update copyright file headers to BUSL-1.1 --------- Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com>	3 years ago
Lucas Bajolet	a2930bda4f	packer: warn on bundled plugins usage ... Since bundled plugins will be removed in an upcoming version of Packer, this commit adds a new warning message whenever a template uses one such plugin. This warning has been implemented on build, validate, console and the inspect subcommands. In addition to warning about the upcoming change and potential issue this will cause, this warning message proposes solutions to the user so they know what they'll have to do in order not to rely on those bundled plugins later.	3 years ago
hashicorp-copywrite[bot]	b7df3ca36f	[COMPLIANCE] Add Copyright and License Headers (#12254) ... Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com>	3 years ago
Wilken Rivera	57cbe4e203	core: Update validation options for undeclared variables (#12104) ... * Update validation options for undeclared variables In an effort to help users move from JSON to HCL2 templates the support for variable definitions files are being updated to ignore undeclared variable warnings on build execution. For legacy JSON templates builds no warnings are displayed when var-files contain undeclared variables. Since preferred mode HCL2 templates is to be explicit with variable declarations - they must be declared to be used - validation for undeclared variables still warns when running `packer validate`. A new flag has been added to the validate command that can be used to disable undeclared variable warnings. * Update validation test for unused variables Example Run ``` ~> go run . validate -no-warn-undeclared-var -var-file command/test-fixtures/validate/var-file-tests/undeclared.pkrvars.hcl command/test-fixtures/validate/var-file-tests/basic.pkr.hcl The configuration is valid. ~> go run . validate -var-file command/test-fixtures/validate/var-file-tests/undeclared.pkrvars.hcl command/test-fixtures/validate/var-file-tests/basic.pkr.hcl Warning: Undefined variable The variable "unused" was set but was not declared as an input variable. To declare variable "unused" place this block in one of your .pkr.hcl files, such as variables.pkr.hcl variable "unused" { type = string default = null } The configuration is valid. ~> go run . build -var-file command/test-fixtures/validate/var-file-tests/undeclared.pkrvars.hcl command/test-fixtures/validate/var-file-tests/basic.pkr.hcl file.chocolate: output will be in this color. Build 'file.chocolate' finished after 744 microseconds. ==> Wait completed after 798 microseconds ==> Builds finished. The artifacts of successful builds are: --> file.chocolate: Stored file: chocolate.txt ``` * Rename Strict field to WarnOnUndeclaredVar The field name Strict is a bit vague since it is only used for checking against undeclared variables within a var-file definition. To mitigate against potential overloading of this field it is being renamed to be more explicit on its usage. * command/build: Add warn-on-undeclared-var flag Now that the default behaviour is to not display warnings for undeclared variables an optional flag has been added to toggle the old behaviour. ``` ~> go run . build -warn-on-undeclared-var -var-file command/test-fixtures/validate/var-file-tests/undeclared.pkrvars.hcl command/test-fixtures/validate/var-file-tests/basic.pkr.hcl Warning: Undefined variable The variable "unused" was set but was not declared as an input variable. To declare variable "unused" place this block in one of your .pkr.hcl files, such as variables.pkr.hcl variable "unused" { type = string default = null } file.chocolate: output will be in this color. Build 'file.chocolate' finished after 762 microseconds. ==> Wait completed after 799 microseconds ==> Builds finished. The artifacts of successful builds are: --> file.chocolate: Stored file: chocolate.txt ```	3 years ago
Lucas Bajolet	1cee460d0d	hcp: extract all HCP-related code to hcp package ... As part of the work to displace everything related to HCP from scattered places around the Packer code, we move it all to an hcp package. This in turn reduces the amount of code that the commands have to integrate, and leaves the HCP details to its own enclave.	3 years ago
Lucas Bajolet	7cd095062a	parser: extract HCP logic out of the HCL parser ... Before this commit, some HCP-related logic was embedded in the parser, making all of it common to all commands. This complexifies the logic as some parts rely on the template being in a HCP-valid environment, and the datasources need to be evaluated in order for some operations to be executed. To simplify this and avoid those pitfalls, we move this logic to its own set of functions that will be performed after the parsing is done, on any valid HCL2 or JSON template.	4 years ago
Adrien Delorme	5d17d7fe65	prevent duplicate local block creation (#11534) ... * prevent duplicate local block creation * remove duplicate locals block bug * local variables: first get block, then decode it + simplify retry loop * Update types.packer_config.go * revert go get of hcl lib	4 years ago
Wilken Rivera	a534c743cf	HCP Packer registry enabled Packer client (#11154) ... * Add working registry pkg * Add custom error for handling the loading of PAR environment variables * Working Publish to Build, with proper error handling for bucket names * Update hcp-sdk-go to use branch instead of mod replace directive * Update Packer build status configuration * Add support for HCP_PACKER_BUILD_FINGERPRINT env * Add support for publishing one or more PARtifacts from a single build * add git shas to this branch * Add ability to set provider name if available * Add working RegistryBuilder type * Add RegistryPostProcessor as wrapper post-processor * When in PAR mode a empty RegistryPostProcessor is added to the end of the post-processor list to publish all final image data. * Add support for updating a build from PAR that is not in a DONE state * Fix a small issue with creation the initial builds for an empty iteration. * Add PAR URL to post-processor display * Implement hcp_packer_registry block (#11168) * Update vendored Amazon plugin to v1.0.1-dev * Fix panic when running a Packer registry build in a clean directory * Remove the publishing of post-processor metadata from the registry post-processor. * Remove metadata add from registry_builder * Update registry builder to skip a build that was found to be DONE Co-authored-by: Megan Marsh <megan@hashicorp.com> Co-authored-by: Sylvia Moss <moss@hashicorp.com>	5 years ago
Adrien Delorme	03d79a2c39	HCL2 variables: split validation from getting value (#10843) ... * HCL2 variables: split validation from getting value, to only This way we do this only once and log this only once. The errors were being ignored anyways. * Update types.variables_test.go	5 years ago
Adrien Delorme	77a29fc2f8	Allow to have `dynamic` blocks in a `build` block + tests (#10825) ... This : * allows to have a `build.dynamic` block * add tests * makes sure to show a correct message when a source was not found * display only name of source (instead of a weird map printout) * use a "Did you mean %q" feature where possible Because dynamic blocks need all variables to be evaluated and available, I moved parsing of everything that is not a variable to "after" variables are extrapolated. Meaning that dynamic block get expanded in the `init` phase and then only we start interpreting HCL2 content. After #10819 fix #10657	5 years ago
Adrien Delorme	f32b67c3bb	Simplify error message when config file can't be "stat'd" (#10763) ... * remove confusing message when a stat error happens while trying to list HCL2 files * leave early if our first GetHCL2Files has errors	5 years ago
Adrien Delorme	72e4dc4cb5	update docs to remove 'magic' required_plugin block usages	5 years ago
Sylvia Moss	3ddb17ad86	Don't allow data sources to be used inside data sources (#10559)	5 years ago
Adrien Delorme	ed091163be	HCL2 Parse packer.required_plugins block + packer init (#10304) ... This adds the new `required_plugins` block to be nested under the packer block. Example: ```hcl packer { required_plugins { aws = { version = ">= 2.7.0" source = "azr/aws" } azure = ">= 2.7.0" } } ``` For example on darwin_amd64 Packer will install those under : * "${PACKER_HOME_DIR}/plugin/github.com/azr/amazon/packer-plugin-amazon_2.7.0_x5.0_darwin_amd64" * "${PACKER_HOME_DIR}/plugin/github.com/hashicorp/azure/packer-plugin-azure_2.7.0_x5.0_darwin_amd64_x5" + docs + tests	5 years ago
Megan Marsh	fbbda0f9d9	Sensitive locals (#10509) ... * Allow locals to be delcared as individual blocks, and give them the Sensitive flag * add docs for new local block * linting * add tests * modified parsing to use schema, check for dupes properly * update comment fix wording a liiitle * add tests for duplicate variables definition in two different files * remove unnecessary slice initialisation * fix crash by returning when decode error is hit * parseLocalVariables: only treat a local vars if its not nil also return in case of error return locals in case of error too * fix duplicate_locals test for windows Co-authored-by: Adrien Delorme <azr@users.noreply.github.com>	5 years ago
Sylvia Moss	291121dd55	(2) Implement datasources (#10440)	5 years ago
Megan Marsh	88b7b0e14a	remove sdk	5 years ago
Wilken Rivera	4e58987026	command/fmt: Ensure all variable files ending in `.pkrvars.hcl` get formatted (#10377) ... Before change ``` ⇶ packer fmt -check /tmp/unformatted.pkrvars.hcl Error: Cannot tell whether /tmp/unformatted.pkrvars.hcl contains HCL2 configuration data ⇶ echo $? 1 ``` After fix ``` ⇶ packer fmt -check /tmp/unformatted.pkrvars.hcl /tmp/unformatted.pkrvars.hcl ⇶ echo $? 3 ⇶ packer fmt -check command/test-fixtures/fmt command/test-fixtures/fmt/unformatted.pkr.hcl command/test-fixtures/fmt/unformatted.auto.pkrvars.hcl command/test-fixtures/fmt/unformatted.pkrvars.hcl ```	5 years ago
Megan Marsh	39ab646236	move plugin and rpc code into sdk; other minor tweaks (#10359)	5 years ago
Megan Marsh	a6fdeca099	move logsecretfilter into packer sdk	5 years ago
Megan Marsh	bc85854a53	refactor packer version out of hcltemplate code.	5 years ago
Adrien Delorme	4bc16455b4	HCL2: add a packer block with a required_version input setting (#10149) ... * add the possibility to set the packer.required_version field; to make sure the template file works with that version of Packer * add tests * add documentation on packer.required_version Example: packer { required_version = ">= 1.2.0, < 2.0.0" }	5 years ago
Adrien Delorme	4cb218749b	remove unecessary check	5 years ago
Adrien Delorme	ebb1cb406c	ignore walk error	5 years ago
Adrien Delorme	bb22cfcf34	HCL2: pass sensitive variables to packer.LogSecretFilter ... Co-Authored-By: Megan Marsh <1008838+SwampDragons@users.noreply.github.com>	5 years ago
Sylvia Moss	9181a435aa	Make max_retries a string to allow variable interpolation (#9673)	6 years ago
Adrien Delorme	44616d3bff	refactor initialization out from packer configs + tests (#9627) ... The initialization of packer core in JSON also validates that `null` variables were set, except in the case of `packer validate --syntax-only` , but after the refactor to allow to have all commands work with HCL2 and JSON this subtlety was lost. This refactors the initialisation of the core in order to allow to have `packer validate --syntax-only` not error in case a variable is not set. Since these calls are refactored this works for HCL2 too. fix #9478	6 years ago
Adrien Delorme	125178d943	core: Decode: when in HCL2 decoding mode; reset the whole struct before preparing it. (#9622) ... * core: Decode when in HCL2 decoding mode; reset the whole struct before preparing it. * HCL2: add path variables + docs & tests Co-authored-by: Megan Marsh <megan@hashicorp.com>	6 years ago
Adrien Delorme	bac9c74447	packer console for HCL2 (#9359)	6 years ago
Adrien Delorme	059a782b0e	remove debug log	6 years ago
Adrien Delorme	125ffe36c3	Allow to override some source fields from the build section ... Using `hcl.MergeBodies`	6 years ago
Adrien Delorme	526d040534	document Parser.Parse better	6 years ago
Adrien Delorme	5c2b8da63b	get builds from PackerConfig instead of parser ... to allow just reading the config and to not start anything. This will allow to later on run `validate --syntax-only`. Note that none of the builder/provisioner/post-processor config will be read but simply ignored. HCL2 still needs the body to be properly formatted and it should detect most syntax errors.	6 years ago
Adrien Delorme	2df21496b3	hcl2template: let PackerConfig actually start a builder/provisioner/post-processor	6 years ago
Adrien Delorme	2af40c762b	HCL2: pass on builder type and name (#8956) ... * sets `packer_build_name` and `packer_builder_type` variables for builder provisioners and post-processors in HCL2 * allows to use the new `${source.type}` and `${source.name}` variables in HCL2 * fixes #8932 Note that the common.PackerConfig is used everywhere and was not set for HCL2, this had some implications: For #8923 you can see the issue here: dde74232f2/builder/lxd/config.go (L61-L63) More random examples of where this could cause an issue : 0785c2f6fc/provisioner/ansible-local/provisioner.go (L380-L381) b4efd13a4d/builder/amazon/ebs/builder.go (L232-L236) * [All references to PackerConfig.PackerBuildName](https://sourcegraph.com/github.com/hashicorp/packer@ff6a039d5bb45e34ff761d9c52e8b98972288447/-/blob/common/packer_config.go#L7:2&tab=references) * [All references to PackerConfig.PackerBuilderType](https://sourcegraph.com/github.com/hashicorp/packer@ff6a039d5bb45e34ff761d9c52e8b98972288447/-/blob/common/packer_config.go#L8:2&tab=references)	6 years ago