packer

Commit Graph

Author	SHA1	Message	Date
Lucas Bajolet	accbe97e1e	hcl2template: add text(encode\|decode)base64 funcs (#12997 ) Compared to Terraform, Packer was lacking a capability to encode/decode strings to/from base64-encoded text encoded with another encoding. This could be problematic in some cases, mainly when working with Windows, as most of the OS uses UTF-16LE as its standard encoding for many operations. Therefore, we take a page from Terraform here, and add those functions to what Packer supports in an HCL2 context.	2 years ago
Lucas Bajolet	ec682afc4a	addrs: require at least 3 parts to source URIs	2 years ago
Lucas Bajolet	c7f2508373	addrs: return error on ParsePluginSource The ParsePluginSource function can be invoked from either a HCL2 context (when parsing a required_plugins block), or from the command-line itself. While in the first context a hcl.Diagnostics is coherent, in case the source to parse is a command-line argument, for example when installing or removing a plugin, the error message cannot have an HCL context, leading to errors that are incorrectly prefixed by a <nil> string dure to the lack of a reference to attach the diagnostic to. Therefore, in order to fix this behaviour, the logic that parses plugin sources now returns an error, and attaching the error to an HCL subject is done independently, if needed.	2 years ago
Lucas Bajolet	9b38e0eb5c	addrs: limit source component length to 16 When specifying/installing plugins, a source URI is required for Packer to be able to locate or install a plugin to the local plugin hierarchy. The plugin hierarchy is based on the plugin source, where each component in this hierarchy will become a directory. In order to avoid sources with too many levels of nesting, causing a lot of mkdirs, we limit the number of sources to 16 in this commit, this should be long enough for most of our users.	2 years ago
Lucas Bajolet	d2ade9a783	packer: register plugin components only once When running a packer command on an HCL2 template, depending on whether or not there are required_plugin blocks defined, Packer may need to discover and register a plugin's components multiple times. This is the behaviour ever since those blocks were introduced to Packer, but given we are doing the operation multiple times, this is suboptimal. This commit changes the way things works by first doing the restricted discovery of plugins (as dictated by required_plugins), then proceeding to the global discovery, with the change that subsequent component discoveries will not have precedence over those pre-discovered anymore. This allows us to invert the call order of both discovery phases safely, and maintains the constraints described in the templates.	2 years ago
Lucas Bajolet	6fc1d154bd	packer: relax constraints on sources The source parsing logic was heavily directed towards Github compatible source URIs, however if we want to support more cases, we need to make sure we are able to specify those URIs, and to load plugins installed from those sources. Right now, since the getters available are only github.com, we will not support remotely instlling plugins from sources other than github.com, with the same set of constraints as before. However, we do support now installing from a local plugin binary to any kind of source, and we support loading them, including if a template wants this plugin installed locally with version constraints.	2 years ago
Lucas Bajolet	1bf3e86ee1	hcl2template: fix invalid_inexplicit_source_2 test The invalid_inexplicit_source_2 subtest for parse used to fail in previous versions of Packer, but with the latest changes to plugin source management, it won't, at least at parsing time, instead it fails later when the Github getter (the only available for now) cannot get the plugin binary from the source, as the source is not a valid Github URI.	2 years ago
Lucas Bajolet	93bec31ec9	version: embed version/VERSION as version string Since we have both version/version.go and version/VERSION to specify version strings, both are a bit redundant. As version/VERSION is supposed to be the source of truth now, we are using it to derive the version informaiton we used to rely on in Packer and its subcommands. Note: doing this prevents us from changing the version/prerelease through ldflags though as we derive Version/VersionPrerelease from the rawVersion variable.	2 years ago
Devashish	595b45e67c	Packer tracks Version and Plugins Metadata (#12860 )	2 years ago
Lucas Bajolet	ea1e798c47	fmt: print parse errors on error The fmt command reformats HCL2 templates, provided it can parse the file and reformat its contents according to the standards set by the HCL library's formatters. However, if the file is malformed for some reason, the command will fail with a parse error, but while the parse error message is shown, the actual errors in the template(s) are not forwarded, making it hard for users to understand what went wrong with the contents of the file they're trying to format. In order to be more helpful with those errors, we now forward those parsing errors to the UI.	2 years ago
Lucas Bajolet	9f7e6ca52b	plugins: add a release only flag to Discover Since we now support loading pre-releases, we also want Packer to be able to ignore them by user demand, so we put in place the infrastructure to modulate this.	2 years ago
Lucas Bajolet	e099c5c661	main: move Discover to DetectPluginBinaries When Packer is loaded, we used to perform plugin discovery. This was done for every call to Packer, including when it is executed as a plugin, arguably against what the comments document. Doing this as early in the loading process makes it harder to change this behaviour, as we'd need to introduce flags aside from the rest, and handle them manually, which is not optimal. Therefore, we change this: now when Packer starts executing, it will not attempt to discover installed plugins anymore, and instead will only try to load them when a configuration has been parsed, and is being used to perform actions (typically build/validate).	2 years ago
Lucas Bajolet	b51c350bd6	packer: fix plugin version sorting and pickup When Packer orders the plugins by their version, we'd assumed it was ordered from highest to lowest. However, this was the case because our implementation of `Less' was actually returning `>=', which is not what it was supposed to do. This commit therefore fixes the implementation of `Less' to do what it is documented to do, and ensures the behaviour is correct through additional testing. Changing this requires some changes to the loading process as well because of the aforementioned assumption with regards to ordering.	2 years ago
Lucas Bajolet	fd5f668ee9	plugin: consolidate loading logic Right now we had two paths for discovering installed plugins, i.e. through plugin-getter's `ListInstallations' function, or through the `Discover' call, which relied on a glob to list the installations. This was required since we allowed plugins to be installed in multiple locations, and with different constraints. Now that we force a certain convention, we can consolidate the logic into ListInstallations, and rely on that logic in `Discover' to load a plugin into the PluginConfig for the current Packer run.	2 years ago
Lucas Bajolet	4b00a81bf3	packer: remove directory slices from structs Since we'll only look in the plugin directory, and not from multiple sources now, for installing/listing plugins, we can simplify the structures which used to accept multiple directories so they only accept one.	2 years ago
Lucas Bajolet	94aaff44a1	packer: only support one directory for plugins Since we're removing the alternative plugin installation directories in favour of only supporting installing them in the PACKER_PLUGIN_PATH directory, we only return one directory when getting the known plugin directories.	2 years ago
Lucas Bajolet	dff49df129	hcl2template: check bucket name at parse-time Not validating the bucket's name during parse leads to configurations being marked as valid, even if the bucket name is not, which will fail during a real build afterwards. To avoid this problem and fail with an error as quickly as possible, we add a check during parsing, so that it gets reported for validate as well.	2 years ago
Devashish	682eca2877	Remove repeated code	2 years ago
Devashish	4b591d6937	Add PR suggestions	2 years ago
Devashish	b77d581cda	Add the deprecation warning and update docs	2 years ago
Devashish	cbc16cfed5	Add versionFingerprint to EvalContext	2 years ago
Wilken Rivera	263821ac25	hcl2template/functions: Add Non-null refinements for various functions cty's new "refinements" concept allows us to reduce the range of unknown values from our functions. This initial changeset focuses only on declaring which functions are guaranteed to return a non-null result, which is a helpful baseline refinement because it allows "== null" and "!= null" tests to produce known results even when the given value is otherwise unknown. This commit also includes some updates to test results that are now refined based on cty's own built-in refinement behaviors, just as a result of us having updated cty in the previous commit.	2 years ago
Lucas Bajolet	8c7b97331b	hcl2: don't show an empty source on var errors If a variable is defined and overriden in the packer command-line, but there's a problem during the evaluation of this override (type error typically), we show an error message which details the problem. This message points to a temporary in-memory HCL2 "file" that we use for parsing and evaluating the expression for the variable, but since it's virtual, there's no point in using this as the source for the error, as it will always yield "line 0" and no contents. So, in order to limit confusion here, we remove the source for this error message.	2 years ago
Lucas Bajolet	efe182b221	packer: link to docs if a component is missing When a user invokes packer for a build or validation, the template being processed needs components to be present for Packer to process it without error. If the component cannot be found from the plugins loaded (or from the components bundled with Packer), Packer errors, and the command fails. This is expected, but the error message does not suggest anything to fix the error, potantially leaving users confused at the problem. This commit suggests either a replacement (in case of a typo), or points to the web documentation for Packer, specifically the integrations, so they can look for the plugin they're missing, and install it, so subsequent invocations of Packer work.	2 years ago
Lucas Bajolet	434a1637f8	hcl2template: fix func to get vars from a config The previous implementation of the GetVarsByType function worked only on top-level attributes, ignoring the nested blocks in the structure. This implies that if a datasource depends on another through an expression within a nested block, we may not execute it first, and then executing this datasource before its dependent is possible, resulting in an error in the end. This commit is an attempt at making this more reliable for HCL configs, but only works on configs lifted from HCL files for now. We need to make this more reliable for later iterations.	2 years ago
Lucas Bajolet	51ec78622d	hcl2template: simplify startDatasource function	2 years ago
Lucas Bajolet	d9259d6d12	hcl2template: extract attr filter code from ds Datasources use their attribute's expressions to determine whether or not they depend on another datasource, in order to get the list of dependencies and execute them before executing a datasource. This code may be useful later on for figuring out the dependencies for any block, so we move this code to the utils.go file, and use this for datasources.	2 years ago
Lucas Bajolet	14b6ad10fe	hcl2template: report localtion for cycle detection When a datasource fails to be evaluated because a cycle has been detected, we point out one of the links of the chain now so that users have a better idea of what to look at.	2 years ago
Lucas Bajolet	2a6ccaccab	hcl2template: simplify datasource evaluation Since datasources are recursively evaluated depending on their dependencies, we don't need to pre-execute those that depend on nothing, as the recursive traversal of the datasources will take care of that for us.	2 years ago
Lucas Bajolet	8438d81d4d	hcl2template: remove unused shouldContinue bool Not sure why this was defined and returned, but the value was set, but never used, as such this is not useful to keep in the code, so let's simplify this now.	2 years ago
guoguangwu	7eb6a45cd4	chore: remove refs to deprecated io/ioutil Signed-off-by: guoguangwu <guoguangwu@magic-shield.com>	2 years ago
guoguangwu	324e628b86	chore: unnecessary use of fmt.Sprintf Signed-off-by: guoguangwu <guoguangwu@magic-shield.com>	2 years ago
Lucas Bajolet	318529f373	hcl2template: remove value validation for locals Local variables can't have a validation block in their definition, so this step in not useful and should be removed. Besides, since the validation was done on the local variables before evaluation, it did nothing at all, as the PackerConfig.LocalVariables collection gets populated during evaluation, so this is essentially a no-op, and can be safely removed.	2 years ago
Lucas Bajolet	eb9e1a4795	packer: remove implicit required plugins Since this feature is no longer something we plan to activate later, as it contradicts with our efforts to remove bundled plugins, and encouraging users to move to either manually installing plugins, or managing them through `packer init', we clean-up the code for this feature.	3 years ago
hashicorp-copywrite[bot]	19055df3ec	[COMPLIANCE] License changes (#12568 ) * Updating the license from MPL to Business Source License Going forward, this project will be licensed under the Business Source License v1.1. Please see our blog post for more details at https://hashi.co/bsl-blog, FAQ at https://hashi.co/license-faq, and details of the license at www.hashicorp.com/bsl. * Update copyright file headers to BUSL-1.1 --------- Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com>	3 years ago
Wilken Rivera	9ccec0d4d5	Removed repetitive statement from HCP debug logs (#12552 ) * Remove repetitive log statement The log being removed is duplicated each time evalContext is called, which seems like an unneeded debug log.	3 years ago
Lucas Bajolet	a2930bda4f	packer: warn on bundled plugins usage Since bundled plugins will be removed in an upcoming version of Packer, this commit adds a new warning message whenever a template uses one such plugin. This warning has been implemented on build, validate, console and the inspect subcommands. In addition to warning about the upcoming change and potential issue this will cause, this warning message proposes solutions to the user so they know what they'll have to do in order not to rely on those bundled plugins later.	3 years ago
Lucas Bajolet	456bc2397b	hcl2template: only warn once for missing plugins In HCL2, if a template contains a `required_plugins' block, but the required plugins are not installed, we print a warning. Prior to this commit, this was reported as an error once per missing plugin, which was redundant. This commit changes that by gathering all the missing plugins once, and printing them as a bullet list, so the message isn't redundant anymore.	3 years ago
Wilken Rivera	eca75a6847	Run make fmt to fix checks	3 years ago
Lucas Bajolet	f1a41b488f	packer: add HCL2 support for telemetry Packer's telemetry package captures the field names from a configuration on legacy JSON templates, but did not contain any code for handling HCL2 template configurations. This commit adds a routine to extract the field names from a HCL2 configuration once flattened, and adds some glue code to propagate the configurations to the telemetry structures.	3 years ago
hashicorp-copywrite[bot]	b7df3ca36f	[COMPLIANCE] Add Copyright and License Headers (#12254 ) Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com>	3 years ago
Lucas Bajolet	4d559e23b7	hcl2: don't warn on excluded post-processor (#12187 ) * hcl2template: use && for chaining conditions For deciding if we need to run an error provisioner block, we checked using a nested if that we didn't need to avoid running it. This is unnecessary, and adds a superfluous level of inbrication to the code, so we chain both conditions with &&. * hcl2: don't warn on excluded post-processor When a post-processor is excluded through the -except command-line argument for packer build, we used to print a warning on HCL2 templates if this except statement did not match a build. However, since we can use the except option to avoid running post-processors, we should account for this as well when printing out a warning that there were no matches. This commit adds this counter to the except checks for the post-processors as well.	3 years ago
Lucas Bajolet	ca197afe9b	hcp: remove superfluous return value on GetBuilds This commit irons out one of the pain points of the HCP rework by introducing a HCPPublisher interface, implemented both by the JSON Core, and the HCL2 PackerConfig, which keeps a map of the build names used by Packer to the build names pushed on HCP. This in turn lets us go back to the old implementation of the GetBuilds function, which returns a list of (filtered) builds, and eventually an error if something went wrong while processing.	3 years ago
Wilken Rivera	9002ac263f	Update BuildBlock with valid HCL2Ref (#12167 ) While looking into #11932 I found that Packer will throw an error when an unknown source is referenced from within the sources attribute for a build block. The hcl.Diagnostics error includes a Subject, which should highlight where in the HCL2 configuration file the unknown source resides. But when creating the BuildBlock no HCL2Ref data is copied over, thus the Subject is displayed with with zero value for an hcl.Range type. This change updates the build creation logic to copy the HCL2Ref data from the build block at decode time so that it can be properly referenced downstream. Closes #11932 Failure due to change in output ``` --- FAIL: TestValidateCommand_ShowLineNumForMissing (0.00s) --- FAIL: TestValidateCommand_ShowLineNumForMissing/test-fixtures/validate-invalid/missing_build_block.pkr.hcl (0.00s) validate_test.go:377: Unexpected output: ( """ Error: Unknown source file.cho - on line 0: + on test-fixtures/validate-invalid/missing_build_block.pkr.hcl line 6: (source code not available) ... // 4 identical lines """ ) validate_test.go:379: FAIL FAIL github.com/hashicorp/packer/command 1.002s ```	3 years ago
Wilken Rivera	57cbe4e203	core: Update validation options for undeclared variables (#12104 ) * Update validation options for undeclared variables In an effort to help users move from JSON to HCL2 templates the support for variable definitions files are being updated to ignore undeclared variable warnings on build execution. For legacy JSON templates builds no warnings are displayed when var-files contain undeclared variables. Since preferred mode HCL2 templates is to be explicit with variable declarations - they must be declared to be used - validation for undeclared variables still warns when running `packer validate`. A new flag has been added to the validate command that can be used to disable undeclared variable warnings. * Update validation test for unused variables Example Run ``` ~> go run . validate -no-warn-undeclared-var -var-file command/test-fixtures/validate/var-file-tests/undeclared.pkrvars.hcl command/test-fixtures/validate/var-file-tests/basic.pkr.hcl The configuration is valid. ~> go run . validate -var-file command/test-fixtures/validate/var-file-tests/undeclared.pkrvars.hcl command/test-fixtures/validate/var-file-tests/basic.pkr.hcl Warning: Undefined variable The variable "unused" was set but was not declared as an input variable. To declare variable "unused" place this block in one of your .pkr.hcl files, such as variables.pkr.hcl variable "unused" { type = string default = null } The configuration is valid. ~> go run . build -var-file command/test-fixtures/validate/var-file-tests/undeclared.pkrvars.hcl command/test-fixtures/validate/var-file-tests/basic.pkr.hcl file.chocolate: output will be in this color. Build 'file.chocolate' finished after 744 microseconds. ==> Wait completed after 798 microseconds ==> Builds finished. The artifacts of successful builds are: --> file.chocolate: Stored file: chocolate.txt ``` * Rename Strict field to WarnOnUndeclaredVar The field name Strict is a bit vague since it is only used for checking against undeclared variables within a var-file definition. To mitigate against potential overloading of this field it is being renamed to be more explicit on its usage. * command/build: Add warn-on-undeclared-var flag Now that the default behaviour is to not display warnings for undeclared variables an optional flag has been added to toggle the old behaviour. ``` ~> go run . build -warn-on-undeclared-var -var-file command/test-fixtures/validate/var-file-tests/undeclared.pkrvars.hcl command/test-fixtures/validate/var-file-tests/basic.pkr.hcl Warning: Undefined variable The variable "unused" was set but was not declared as an input variable. To declare variable "unused" place this block in one of your .pkr.hcl files, such as variables.pkr.hcl variable "unused" { type = string default = null } file.chocolate: output will be in this color. Build 'file.chocolate' finished after 762 microseconds. ==> Wait completed after 799 microseconds ==> Builds finished. The artifacts of successful builds are: --> file.chocolate: Stored file: chocolate.txt ```	3 years ago
Wilken Rivera	606e6c48f1	internal: reorganise registry/HCP code This commit reorganises the code for both the registry/API and the Orchestrator/Registry. The main difference with the previous version is how stuff is exposed. Now we only expose a Registry interface to the outside (previously named Orchestrator), which has several implementations: null is the default, and is returned if HCP is not enabled. The other implementations being HCL/JSON, both private to the hcp sub-package. The api (previously `registry') is the set of functionality that abstracts and calls the HCP API. It was meant to be merged with the `hcp' package, but because of a dependency loop with the datasources, both are separated for now.	3 years ago
Lucas Bajolet	1cee460d0d	hcp: extract all HCP-related code to hcp package As part of the work to displace everything related to HCP from scattered places around the Packer code, we move it all to an hcp package. This in turn reduces the amount of code that the commands have to integrate, and leaves the HCP details to its own enclave.	3 years ago
Lucas Bajolet	dad07c6097	hcp: keep map of build names for hcp tracking To be able to track HCP builds during a Packer build, we need to propagate the name used for registering the build to HCP. This works out-of-the-box for JSON templates, as the build's name is always the builder's, so there's no confusion on that. For HCP templates however, there's a possibility for a template to define a name for the build, which is returned by a CoreBuild's Name() function, in addition to the source's Type/Name. The builds are registered to HCP with the source.String() function however, which does not contain the build's name. This prevents any build containing a name to work, as their name/source.String does not match. The name we registered the build with is stored in a CoreBuild as the Type attribute, but cannot be safely accessed, as builds are types within the command as packersdk.Build, which only exposes Name() for this purpose. In order to circumvent this problem, and as a way to present a prototype of solution that will likely have to be discussed before we merge it, this commit hotfixes the issue.	3 years ago
Lucas Bajolet	f3e4c2c3bb	tmp: move HCP logic to unwrap builds/post	3 years ago
Lucas Bajolet	61c810e720	command: rework HCP-related error messages The HCP error messages were sometimes a bit terse in terms of the information it bubbles up to the user. This made them useful for people who already knew what to look for, but newcomers would almost certainly be lost because of the lack of information in those. To improve on this situation, we reword most of those error messages in this commit.	3 years ago

1 2 3 4 5 ...

335 Commits (8d4a9d66ab4eec8bb3efb2e2c363d303ac4db482)