1101 Commits (add_filebase64_function)

Author	SHA1	Message	Date
Lucas Bajolet	98198b5af8	packer: make GetBuilds return *[]CoreBuild ... The GetBuilds function, available on both HCL2 and legacy JSON configuration objects, used to return the Build interface. This typing by interface is not useful in this instance, since all the uses of `GetBuilds' are self-contained within Packer, and we're never using any other implementation for it than `*CoreBuild`. We've been relying on the dynamic type for all the builds being *CoreBuild in some places of the code, so to avoid potential surprises in the future, we'll change the signature now so it returns only concrete types.	1 year ago
Jenna Goldstrich	347c57306c	hcp: use enum for HCP SBOM upload ... Since the protos for uploading an SBOM for a build have been changed to use an enumeration instead of a plain string with the latest revisions to the HCP Packer SBOM support feature, we update how we reference those values for the SBOM format to use that enum instead.	1 year ago
Jenna Goldstrich	362c547211	hcp: integrate SBOM upload to HCP code ... Since packer now supports keeping track of SBOMs produced during a build, we add the code to integrate those changes into the internal/hcp package, so we do upload them on build completion.	1 year ago
Devashish	a353260f5d	packer: add hcp-sbom provisioner ... The hcp-sbom provisioner is a provisioner that acts essentially like a download-only file provisioner, which also verifies the file downloaded is a SPDX/CycloneDX JSON-encoded SBOM file, and sets up its upload to HCP Packer later on.	1 year ago
Lucas Bajolet	9f6f0ba6a2	packer: pick protobuf/gob for serialisation (#13025) ... As we're trying to move away from gob for serialising data over the wire, this commit adds the capability for Packer to pick dynamically between gob or protobuf for the serialisation format to communicate with plugins. As it stands, if all the plugins discovered are compatible with protobuf, and we have not forced gob usage, protobuf will be the serialisation format picked. If any plugin is not compatible with protobuf, gob will be used for communicating with all the plugins that will be used over the course of a command.	1 year ago
huochexizhan	6e417bb883	fix: fix slice init length	2 years ago
Lucas Bajolet	586762564b	hcl2template: intro and add UseSequential init opt ... Following up on the DAG work, this commit adds a new option for initialisation that disables DAG on request. By default we are going to use the DAG approach, with an option to fallback to using the older algorithm for evaluation in case users end-up in an edge-case that prevents them from building a template.	2 years ago
Lucas Bajolet	3e3b136f3c	packer: address unused lint errors	2 years ago
Lucas Bajolet	962ccdfc80	packer: address gosimple lint errors	2 years ago
Lucas Bajolet	e8d3a55b5f	packer: address errcheck lint errors ... In a couple places in the codebase we didn't check the errors from functions we execute. In some cases this is harmless (or at least ignorable), but others may need to log what went wrong, so for all the reported occurrences we either ignore explicitly or handle the error with a log.	2 years ago
Lucas Bajolet	2009079246	packer: remove ineffasign pointed assigns/decls ... Some of the variables we create are flagged by our linters as ineffective assignments, which makes sense as those are generally fed by code below, so we don't need to use the declaration/assignation syntax (:=) but instead can fall back to using var with a type to get the zero value of the declared entity.	2 years ago
Lucas Bajolet	4a7f5f38a6	Makefile: replace enumer upstream with dmarkham's (#13107) ... Since the enumer implementation we used hadn't been updated for 5+ years, this didn't work with recent linux/go versions, and enumer crashed while attempting to parse/analyse the source files. There's another alternative on Github, forked from the one we used, which seems more maintained now, and does produce the expected files in Packer.	2 years ago
IAMDAVID0920	d625694a88	refactor: remove deprecated InitializePluginVersion, replace with NewPluginVersion	2 years ago
Lucas Bajolet	97db9d82c4	packer: fix splitting error message on pathsep	2 years ago
Lucas Bajolet	ec92d231bc	packer: use filepath for getting plugin basename ... When Discovering plugins installed through the `Discover` function, we use the base name of the plugin binary we discovered preliminarly, then we match its name against a regex to extract the prefix for the plugin's components. Extracting the base path used to be done with `path.Base`, which while working perfectly on UNIX systems, does not on Windows as it uses `\\` as their path separator. To circumvent this problem, we use the `filepath` package to extract the base name of the plugin instead, making the discovery logic work again on Windows.	2 years ago
Lucas Bajolet	a852434410	packer: fix listing on windows with .exe ext ... Listing installed plugins on Windows requires the extension to be set in the ListOptions, otherwise they are not discovered. While working on the discovery code, and consolidating it in a single location, we've forgotten to pass the argument to ListInstallations, so that makes it impossible to automatically discover installed components on Windows. This commit fixes this issue for the plugins required, and the general discovery process during build/validate.	2 years ago
Lucas Bajolet	682968d0df	packer: don't load plugins with metadata in name ... If a plugin is installed in the PACKER_PLUGIN_PATH, and its version contains metadata, we reject it. This is because metadata is free-form data, which could then make it possible to have multiple conflicting versions of a plugin installed, so we don't support it and explicitely reject plugins like those. A valid plugin with metadata in its version information should be installed without its metadata part, so there can only be one variant of the plugin installed at a specific version.	2 years ago
Lucas Bajolet	c7f2508373	addrs: return error on ParsePluginSource ... The ParsePluginSource function can be invoked from either a HCL2 context (when parsing a required_plugins block), or from the command-line itself. While in the first context a hcl.Diagnostics is coherent, in case the source to parse is a command-line argument, for example when installing or removing a plugin, the error message cannot have an HCL context, leading to errors that are incorrectly prefixed by a <nil> string dure to the lack of a reference to attach the diagnostic to. Therefore, in order to fix this behaviour, the logic that parses plugin sources now returns an error, and attaching the error to an HCL subject is done independently, if needed.	2 years ago
Lucas Bajolet	9b38e0eb5c	addrs: limit source component length to 16 ... When specifying/installing plugins, a source URI is required for Packer to be able to locate or install a plugin to the local plugin hierarchy. The plugin hierarchy is based on the plugin source, where each component in this hierarchy will become a directory. In order to avoid sources with too many levels of nesting, causing a lot of mkdirs, we limit the number of sources to 16 in this commit, this should be long enough for most of our users.	2 years ago
Lucas Bajolet	d2ade9a783	packer: register plugin components only once ... When running a packer command on an HCL2 template, depending on whether or not there are required_plugin blocks defined, Packer may need to discover and register a plugin's components multiple times. This is the behaviour ever since those blocks were introduced to Packer, but given we are doing the operation multiple times, this is suboptimal. This commit changes the way things works by first doing the restricted discovery of plugins (as dictated by required_plugins), then proceeding to the global discovery, with the change that subsequent component discoveries will not have precedence over those pre-discovered anymore. This allows us to invert the call order of both discovery phases safely, and maintains the constraints described in the templates.	2 years ago
Lucas Bajolet	f80ba2821b	packer: error multiple paths in PACKER_PLUGIN_PATH ... When a user defines PACKER_PLUGIN_PATH in their environment, we need to error if their path defines multiple directories separated by `:`. This used to be supported, but this is removed with 1.11 as we're simplifying the loading process for plugins, so we opted to fall-back to only one plugin directory supported.	2 years ago
Wilken Rivera	20345f9118	Add check to prevent the installation of version constraints matching a prerelease ... * Refactor InstallError string messages	2 years ago
Wilken Rivera	11dc684334	Add ConintuableInstallError for continuing installation on version mismatches	2 years ago
Wilken Rivera	2e8bdb5c4a	Use single buffer for storing binary file before writes ... * Only create plugin directories if there is potential plugin install	2 years ago
Wilken Rivera	2ca5b2707c	Add PrereleaseError for handling failed pre-release version installs ... ``` ~> packer init mondoo_req.pkr.hcl Failed getting the "github.com/mondoohq/cnspec" plugin: error: Remote installation of the plugin version 10.8.1-dev is unsupported. This is likely an upstream issue with the 10.8.1 release, which should be reported. If you require this specific version of the plugin, download the binary and install it manually. packer plugins install --path '<plugin_binary>' github.com/mondoohq/cnspec ```	2 years ago
Wilken Rivera	eafdda8a94	Update plugin-getter version matching check ... This change is an attempt to remove the need for additional temporary files, along with calls to stat the temp files, to reduce the number of files being created, opened, and closed. In addition to this change, the logic for falling back to a previous version if the highest matched version is a pre-release has been removed. Instead we will assume that any prior versions will exhibit the same issue and return immediately. A user can install the version manually if they will or they can modify their version constraint to a properly released version.	2 years ago
Lucas Bajolet	35bd020a3e	plugin-getter: rm vocal log on checksum entries ... When a checksum file for a release is downloaded and iterated upon to find the compatible binary for a release, we used to log each non-compatible entry in the logs. This is noisy as we know there's going to be multiple entries that don't match the host's os/arch, and there's no good reason to show those, so we silence them.	2 years ago
Lucas Bajolet	7823a40a25	packer: ensure versions match for remote installs ... Since we're hardening what Packer is able to load locally when it comes to plugins, we need also to harden the installation process a bit. While testing we noticed some remotes had published their plugins with version mismatches between the tag and the binary. This was not a problem in the past, as Packer did not care for this, only the binary name was important, and the plugin could be installed without problem. Nowadays however, since Packer enforces the plugin version reported in the name to be the same as the plugin self-reported version, this makes it impossible for the installed plugin to load anymore in such an instance. Therefore in order to limit confusion, and so users are able to understand the problem and report it to the plugins with that mismatch, we instead install the plugin as a dev version, and report it to the user so they can still use it, but are able to report it to the plugin developers.	2 years ago
Lucas Bajolet	77d537c389	packer: remove duplicate version variable ... When listing installed plugins, we check that the plugin's reported version through describe matches what's in the name of the file. Doing do, we were parsing the same version string twice without modifying it, which was not necessary, so this commit changes that.	2 years ago
Lucas Bajolet	32e64c98d1	packer: fix filename in verbose logs	2 years ago
Lucas Bajolet	04a2542a6f	packer/getter: don't report other os/arch as errs ... Whenever a Github release exposes an entry for another OS/arch combination, this gets registered as an error, which in the event no binary is compatible with the host's OS/arch, gets reported at the end of the getter process. While this is sound in theory, in practice we get the list of all the combinations that don't match the host's, which is not something a Packer user can act on, and might therefore be more confusing than helping to solve the issue. Therefore we opt in this commit to stop registering those cases as real errors, and only log them as an INFO statement.	2 years ago
Lucas Bajolet	64ae3e8ea4	packer: alias version as goversion for getter ... Since we named the version from the getter `version', this means we have a naming conflict inside the loop that attempts to install a versioned candidate for a plugin, making it impossible to invoke something from the go-version package. Since we'll introduce a change that needs the latter capability, we must either rename the local variable to something else than `version', or we need to alias the package locally. This commit implements the latter, opting to call the package goversion.	2 years ago
Lucas Bajolet	d40df8e899	packer: check if errs is nil before getting length ... When installing a remote plugin, and after we've either successfully installed a binary, or exhausted all the possible sources, we print a final error message if nothing was reported. However, given that errs is a pointer to a structure, and if no errors were produced, the the error list could be nil, leading to the call to `Len()' to crash Packer. This is exceedingly rare as in general the code attempts to read multiple sources from Github, and therefore we almost always get some error reported, but while changing the function's code, I managed to make it crash while removing/changing some error statements. Therefore to avoid future surprises, we first check that `errs' is not nil before invoking `Len()' on it, as no errors and no plugins installed mean that something went wrong all the same.	2 years ago
Lucas Bajolet	56fab30fa1	plugin: factorise calling describe on binaries ... Given that calling the describe command on plugins and deserialising the output as a plugin description is something done in multiple places in the code, we factorise this operation so we don't need to copy/paste the code around.	2 years ago
Lucas Bajolet	747cb00cdc	packer: remove temp zipfile after installation ... The zipfile containing the binaries we attempt to install from a remote source is placed in the temporary directory of the host. In general it is wiped automatically by the OS, but in some cases (windows typically), it isn't. To avoid cluttering the temporary directory, we clean-up after ourselves, and remove the temporary zip file that we create when attempting to install a plugin, regardless of it succeeding or not.	2 years ago
Devashish	403e35d2ff	Add thread-safe add and get methods for plugin storage ... * Add general code improvements	2 years ago
Lucas Bajolet	6fc1d154bd	packer: relax constraints on sources ... The source parsing logic was heavily directed towards Github compatible source URIs, however if we want to support more cases, we need to make sure we are able to specify those URIs, and to load plugins installed from those sources. Right now, since the getters available are only github.com, we will not support remotely instlling plugins from sources other than github.com, with the same set of constraints as before. However, we do support now installing from a local plugin binary to any kind of source, and we support loading them, including if a template wants this plugin installed locally with version constraints.	2 years ago
Lucas Bajolet	33d9550903	packer: make Ui implementations match new SDK's ... The SDK's Ui interface has had some additions recently for supporting format-strings and their arguments as paramereters to Say, Ask and Error. These changes to the interface means that all the implementations of that interface need to match the definition, so in this case the implementations we had in packer/ui.go need to be updated to support those functions.	2 years ago
Devashish	595b45e67c	Packer tracks Version and Plugins Metadata (#12860)	2 years ago
Lucas Bajolet	132e3d29b8	packer: fix version string variable name typo ... When introduced back in January, the version variable extracted from the binary name had a typo in its name, and was named `protocolVerionStr' instead of `protocolVersionStr'. This commit was already merged into main, so it's too late to fix at introduction site, but we can fix it today as a separate commit ontop the stack.	2 years ago
Lucas Bajolet	0e6ee70f6f	plugins: reject non-canonical version numbers ... If a plugin is installed manually, its version number could be valid but non-canonical (ex: 1.2.3 vs 01.002.0003). Since these two versions refer to the same version, but the looks are different, this may become ambiguous which version should be loaded. To avoid such a situation, we reject explicitely non-canonical version numbers in plugins, but only in path, we're aware that because of metadata, the version from `describe' may already differ from the file name.	2 years ago
Lucas Bajolet	208afbebfa	packer: change mock set instantiation methods ... The former way mock plugin sets were created meant that no API version was set, and since it's private in the SDK, it cannot be set outside of the package itself. Fortunately, there is a NewSet function we can call, which initialises a set properly so we can fill-in the information later. However, because all the set maps were created in a `var` section, we cannot create the set with `NewSet`, and then fill the information in (outside of if there was a fluent interface, but this isn't the case here). We therefore opted to keep the variables defined and accessible globally, but gone through a `init` function to initialise their values for tests.	2 years ago
Lucas Bajolet	66912bf254	packer: use API version for ordering installations ... As with versions, API versions are useful possibly for ordering plugin installations in order for Packer to choose which plugin to load. This could be unnecessary as API versions are stable, and only in dev plugins this could be a problem normally (there shouldn't be two same releases of a plugin), but this cements API version in ordering plugins so we avoid surprises later down the line.	2 years ago
Lucas Bajolet	0502e09399	packer: check API version matches between name/bin ... As with the version of the plugin, the API version should also match between the path and the self-reported API version from the describe command. This was not checked before, so users could masquerade a plugin's use of an API version that may be incompatible with Packer. To avoid this problem, we make sure both versions are the same, so that they work as expected.	2 years ago
Lucas Bajolet	385ba4cfac	plugins: ensure valid checksum before exec ... When Packer discovers binary a bunch of checks are performed, which ultimately end with a checksum match check. This however should be the very first thing we do, even before attempting to run `describe' on the plugin binary we're discovering. So this commit moves this checksum match to the top of the discovery process for binaries.	2 years ago
Lucas Bajolet	0d66848578	plugins: rely more on version lib for checks ... When discovering the installed plugins locally, we perform a couple of checks on the version, namely that it is valid, if it is a prerelease, it needs to be a dev, and that the self-reported version matches the one hinted at through the name of the binary. This was done through regexes, but those were a wee bit simple when dealing with versions that have metadata. Those binaries would be completely ignored by Packer, and never loaded, although they are a valid use case. The version library we already used supports those however, and comparisons are more reliable with them. So, in order to simplify our code, and make it more reliable, we're exclusively using this library to perform parsing and comparisons of versions during the discovery phase.	2 years ago
Lucas Bajolet	525b0a7c4c	plugin: use API version for loading plugins ... While migrating to a unified approach for loading plugins, the API major and minor versions were not added to the constraints for discovering plugins from the environment, leading to Packer potentially considering plugins that are not compatible with itself. This should not be possible, but was due to this omission, which we fix with this commit.	2 years ago
Lucas Bajolet	7113dea5ef	plugin-getter: fix typo in API mismatch error	2 years ago
Lucas Bajolet	9f7e6ca52b	plugins: add a release only flag to Discover ... Since we now support loading pre-releases, we also want Packer to be able to ignore them by user demand, so we put in place the infrastructure to modulate this.	2 years ago
Lucas Bajolet	e099c5c661	main: move Discover to DetectPluginBinaries ... When Packer is loaded, we used to perform plugin discovery. This was done for every call to Packer, including when it is executed as a plugin, arguably against what the comments document. Doing this as early in the loading process makes it harder to change this behaviour, as we'd need to introduce flags aside from the rest, and handle them manually, which is not optimal. Therefore, we change this: now when Packer starts executing, it will not attempt to discover installed plugins anymore, and instead will only try to load them when a configuration has been parsed, and is being used to perform actions (typically build/validate).	2 years ago