75 Commits (anshul/plugin_binary_from_official_site)

Author	SHA1	Message	Date
Lucas Bajolet	4cd7ad4721	hcl2template: err on malformed local/data dep (#13340) ... * hcl2template: err on malformed local/data dep When introducing the DAG for locals and datasources, we forgot to handle one limit case: if a dependency for a local or data is malformed, we didn't check that a vertex was associated to it, leading to the final DAG being malformed, and the DAG library would crash in this case. This commit fixes this problem by checking that the dependency does exist before attempting to add it to the graph as an edge for a vertex, so that it is reported accurately, and do that Packer doesn't crash. * error message change --------- Co-authored-by: anshul sharma <anshul.sharma@hashicorp.com>	8 months ago
anshulSharma	25343c4839	changes for fixing the SHA mismatch test case (#13385)	9 months ago
Jenna Goldstrich	f16ca7fa03	Bump /x/net and /x/crypto in plugin tester and main go.mod	10 months ago
Anurag Sharma	d4ebc48b89	packer_test: build acc test for shell provisioner in linux only ... atm darwin and windows have issues with the shell provisioner tests.	11 months ago
Anurag Sharma	7a5fc2525a	packer_test: added shell provisioner test with empty inline list	11 months ago
Anurag Sharma	689024a57f	provisioner: added invalid shebang test in packer_test	11 months ago
Anurag Sharma	cb110b263b	packer_test: added inline shebang test cases for shell provisioner	11 months ago
Lucas Bajolet	a8c9467463	packer_test: add integration tests for hcp-sbom	1 year ago
Lucas Bajolet	56400f27cb	packer_test: add gob/pb test suite ... With the draft to support both gob and protobuf as serialisation formats for Packer, along with the SDK changes that propel them, we add a series of tests that make sure the logic that picks which protocol is solid and functional. These tests rely on building several versions of the tester plugin, with and without protobuf support, to then install them in the tests as needed to test the logic of Packer using packer build with them, and templates that require multiple plugins.	1 year ago
dependabot[bot]	3826a94758	build(deps): bump golang.org/x/net in /packer_test/common/plugin_tester ... Bumps [golang.org/x/net](https://github.com/golang/net) from 0.23.0 to 0.33.0. - [Commits](https://github.com/golang/net/compare/v0.23.0...v0.33.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com>	1 year ago
dependabot[bot]	38f081c9ed	build(deps): bump golang.org/x/crypto ... Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.21.0 to 0.31.0. - [Commits](https://github.com/golang/crypto/compare/v0.21.0...v0.31.0) --- updated-dependencies: - dependency-name: golang.org/x/crypto dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com>	1 year ago
Lucas Bajolet	5ff0f146c6	packer_test: introduce global compilation queue ... Compiling plugins was originally intended to be an idempotent operation. This however starts to change as we introduce build customisations, which have the unfortunate side-effect of changing the state of the plugin directory, leading to conflicts between concurrent compilation jobs. Therefore to mitigate this problem, this commit changes how compilation jobs are processed, by introducing a global compilation queue, and processing plugins' compilation one-by-one from this queue. This however makes such requests asynchronous, so test suites that require plugins to be compiled will now have to wait on their completion before they can start their tests. To this effect, we introduce one more convenience function that processes those errors, and automatically fails the test should one compilation job fail for any reason.	1 year ago
Lucas Bajolet	b6b0a081ad	packer_test: add build customisation capabilities ... When building a plugin, we may want some customisation capabilities beyond changing the version/pre-release/metadata, and instead run commands or change files on the filesystem. To do so, we introduce functions under the BuildCustomisation type, which have two responsabilities: changing the current state of the plugin's directory, and cleaning up afterwards. These customisations are passed as parameters to the BuildSimplePlugin function, and are called one-by-one, deferring their cleanup after the build process is finished. A first implementation of such a customisation is added with this commit, in order to change the version of a module that the plugin depends on, which we'll use to change the version of the plugin SDK in order to test how Packer behaves with different versions of the SDK for a single plugin.	1 year ago
Lucas Bajolet	11c238b9eb	packer_test: add FileGlob checker ... When trying to validate that a particular file exists after a run of Packer in a test suite, we can use the FileExists checker that we provide as part of the gadgets we added for the acceptance test suites. This approach works well, but only if we can extract a file name reliably from the output of Packer core, or if we know what to look for exactly beforehand. For other cases with a generated name however, the FileExists checker is not enough, and therefore to accomodate for those cases, we are introducing a new checker for this purpose: FileGlob. FileGlob, as its name suggests, runs a glob expression on the filesystem, and returns an error if no match was found regarding this glob expression.	1 year ago
Lucas Bajolet	44a94911da	packer_test: add FileExists checker ... Some tests will create files and directories as part of the execution path for Packer, and we need a way to check this, so this commit adds a new file gadget to do those checks after a command executes.	1 year ago
Lucas Bajolet	418ebca7ef	packer_test: add base tests for DAG eval ... Since we introduce the DAG with this series of commits, only on locals and data sources, we need to make sure that the behaviour is what we expect. Therefore, this commit adds a basic test with Packer build, and packer validate, to evaluate a template with both locals and data sources depending on one another. This is rejected with the sequential evaluation methods, as we process the different types one-by-one, whereas the DAG allows us to mix the order between the two, while still rejecting circular dependencies (and doing that before they even get evaluated), and self-references.	1 year ago
Lucas Bajolet	3c449430e1	packer_test: make PluginTestDir a structure ... In order for the creation of a temporary directory to install plugins into to be simpler to understand and use, we change how the directory is created, cleaned-up, and installs plugins into. Now, instead of a tuple of a string (path) and a cleanup function, we return a structure that comprises the test suite, and the temporary directory, along with methods to handle those steps independently.	1 year ago
Lucas Bajolet	31b6109430	packer_test: hide run and introduce Output ... When using a PackerCommand, the Run function was made public as a way to access the contents of an execution. This was clumsy as it had too many responsabilities, and was not needed strictly as Assert was performing the executions, as many times as required. This could introduce cases in which one run as spent by the caller, then the remainder were executed through Assert. Therefore, we change this convention. Now, run is private to the type, and only through Assert can a command be executed. If a test needs access to a command's output, stderr, or error, it can do so through the Output function, which requires Assert to be called first.	1 year ago
Lucas Bajolet	71169416db	packer_test: add func to change assert behaviour ... When a command asserts its output with checkers, by default it will register errors through a t.Errorf. While this works, in some cases we would want to stop execution immediately if a function's Assert fails, as the rest of the test may depend on the assertion being valid. In the current state, this means either getting the result of the run to check if an error was returned (not fully reliable as if the command was run multiple times, and the last run succeeded, we won't get an error), or relying on t.IsFailed() (completely reliable). Instead, we introduce a new function on packerCommand, that lets users change how Assert behaves, so that if an error was reported, instead of logging the error and flagging the test as failed, we can use t.Fatalf, so that the test immedately fails and stops execution.	1 year ago
Lucas Bajolet	35b2317ef3	packer_test: rename/split lib into common/check ... The lib name for the common components for writing packer_test suites was not clear, and did not follow the convention established in Packer core and plugins. Therefore this commit does two things: first the lib is renamed into common as to follow this convention, and clearly document which components are common to all tests. Also checkers are placed in a subpackage of common, common/check, so that it is clearer what is meant to be used as checks for a command's execution status after it's been run, as part of Assert.	1 year ago
Lucas Bajolet	7f7ae8454a	packer_test: rename core test run function ... The function's name was the same as the plugins test suite runner/init function, making it impossible to differentiate between the two when looking at the logs, or when attempting to filter which tests to run using the suite function's name.	1 year ago
Lucas Bajolet	6fc2932b09	packer_test: reorganise plugins test suite init	1 year ago
Lucas Bajolet	2952a6621d	packer_test: remove empty defer func in suite init	1 year ago
Lucas Bajolet	3c8a51bdbf	packer_test: split BuildPackerPlugin in build/get ... The interface for building a plugin through the test suite was confusing, as it would build the plugin and return its path, cache the path for the version built, and return the path regardless if something was built or not. While in the current state this is harmless as builds are idempotent, since the state of the plugin package/module does not change, this will in the future as we introduce customisation techniques on the plugin's directory and files, making this double-use potentially dangerous. Furthermore, the current behaviour is unclear, as the function hides that caching mechanism, which could come as a surprise for users attempting to build a plugin for the duration of a test, while the built plugin is linked to the test suite being run, and not the unit test being evaluated. Therefore this commit changes the sequence in which plugins are built and used. Now the `CompilePlugin` function builds a plugin, and does not return its path anymore, instead terminating the tests immediately if they fail. In normal test usage, a new `GetPluginPath` function is introduced, which looks-up the path in the suite's cache, failing immediately if invoked before the plugin is built. With this change, it is heavily advised to build plugins when initialising the suite, then in the tests, the GetPluginPath function should be used to get a plugin's path for interacting with packer commands.	1 year ago
Lucas Bajolet	393f2e925b	packer_test: replace compiledPlugins by a sync.Map ... The compiledPlugins map with a mutex was essentially a glorified sync.Map, and therefore did not need to be defined as such. Instead, this commit replaces its uses by a normal sync.Map, and keeps it in the base test suite.	1 year ago
Lucas Bajolet	aba64b457c	packer_test: rename init function for base suite	1 year ago
Lucas Bajolet	dbd441f6f3	packer_test: add convenience func for invert grep ... As inverted grep is something we do rather often, we add a convenience function to create an inverted grep gadget.	1 year ago
Lucas Bajolet	ded0500109	packer_test: add plugins used gadget ... When testing a packer build or validate, one common use case is to check which plugins are loaded and used by Packer to run a command. This is generally done through Grep, but repeating the same pattern can be redundant, and if the output changes, all those need to be updated. Therefore, this commit introduces a PluginsUsed gadget, which can be orchestrated to ensure a plugin is used, or not used, allowing to check for multiple plugins at once.	2 years ago
Lucas Bajolet	6b158c5c24	packer_test: move some plugin functions to fs.go ... As a small reorganisation attempt, this commit moves some file-system oriented functions from plugin.go into its own file so they are more logically grouped.	2 years ago
Lucas Bajolet	96b4ae4767	packer_test: move BuildSimplePlugin to plugin.go ... In terms of organisation, we keep functions that interact with plugins into its own file, therefore the function that build plugin versions should really be in plugin.go, not in suite.go.	2 years ago
Lucas Bajolet	2fe11fb967	packer_test: move compiledPlugins to attribute ... The compiledPlugins map used to be a global variable, which can be problematic as we move to independent test suites, since those test suites run in the same process space, the global variable could point to now deleted plugin versions/paths in separate suites, which would make tests fail with random errors. To avoid this, the map is now scoped to the test suite, and a new copy is created lazily if used by the test suite.	2 years ago
Lucas Bajolet	9e4452329f	packer_test: make packer test suite modular ... Having only one test suite for the whole of Packer makes it harder to segregate between test types, and makes for a longer runtime as no tests run in parallel by default. This commit splits the packer_test suite into several components in order to make extension easier. First we have `lib`: this package embeds the core for running Packer test suites. This ships facilities to build your own test suite for Packer core, and exposes convenience methods and structures for building plugins, packer core, and use it to run a test suite in a temporary directory. Then we have two separate test suites: one for plugins, and one for core itself, the latter of which does not depend on plugins being compiled at all. This sets the stage for more specialised test suites in the future, each of which can run in parallel on different parts of the code.	2 years ago
dependabot[bot]	7425ef3a45	build(deps): bump github.com/hashicorp/go-retryablehttp ... Bumps [github.com/hashicorp/go-retryablehttp](https://github.com/hashicorp/go-retryablehttp) from 0.7.0 to 0.7.7. - [Changelog](https://github.com/hashicorp/go-retryablehttp/blob/main/CHANGELOG.md) - [Commits](https://github.com/hashicorp/go-retryablehttp/compare/v0.7.0...v0.7.7) --- updated-dependencies: - dependency-name: github.com/hashicorp/go-retryablehttp dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com>	2 years ago
dependabot[bot]	0f223cc9ac	build(deps): bump google.golang.org/protobuf ... Bumps google.golang.org/protobuf from 1.28.1 to 1.33.0. --- updated-dependencies: - dependency-name: google.golang.org/protobuf dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com>	2 years ago
dependabot[bot]	5843c86c7e	build(deps): bump google.golang.org/grpc in /packer_test/plugin_tester ... Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.50.1 to 1.56.3. - [Release notes](https://github.com/grpc/grpc-go/releases) - [Commits](https://github.com/grpc/grpc-go/compare/v1.50.1...v1.56.3) --- updated-dependencies: - dependency-name: google.golang.org/grpc dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com>	2 years ago
Lucas Bajolet	13c52124de	packer_test: check for a panic during execution ... When a command is run, it is the expectation that no test should make Packer panic. If it did, something is wrong and Packer should be fixed so it doesn't panic anymore in that situation. The way we did the check before was adding a PanicCheck after the command ran, so we could make sure of that during `Assert`. However, since we introduced the possibility to have multiple runs, having this addition as part of the run loop meant that the PanicCheck would be run as many times as there were runs. While this worked, this implied that we'd do the same check multiple times on a single command output, which is not optimal. Instead, this commit moves the check to within the `Run` function, this way for each run of the command we do the check once, and then we can assert the results of the command on what output it produced.	2 years ago
Lucas Bajolet	36e43e30ee	hcl2template: detect duplicate locals during parse ... Previously duplicate detection for local variables happened during `Initialise`, through a call to `checkForDuplicateLocalDefinition`. This works in a majority of cases, but for commands like `console`, this was not detected as the return diagnostics for `Initialise` are ignored. That check can be done as early as during parsing however, as the names of blocks are not dynamic in the slightest (no interpolation possible), so we move that detection logic into `Parse`, so that the behaviour is coherent between all commands.	2 years ago
Lucas Bajolet	14cf4b40d4	hcl2template: recursively evaluate local variables ... The logic for evaluating local variables used to rely on their definition order, with some edge cases. Typically `locals` blocks define multiple local variables, which don't necessarily appear in the same order at evaluation as within the template, leading to inconsistent behaviour, as the order in which those are added to the list of local variables is non-deterministic. To avoid this problem, we change how local variables are evaluated, and we're adopting a workflow similar to datasources, where the local variables first build a list of direct dependencies. Then when evaluation happens, we evaluate all the dependencies recursively for each local variable, which takes care of this issue. As with Datasources, we add a cap to the recursion: 10. I.e. if the evaluation of a single variable provokes an infinite recursion, we stop at that point and return an error to the user, telling them to fix their template.	2 years ago
Lucas Bajolet	9d1dc7d41d	packer_test: disable checkpoint for test cmds	2 years ago
Lucas Bajolet	dbaaab512a	packer_test: add SkipNoAcc function ... The SkipNoAcc function on PackerTestSuite allows to mark a test run as not to be run every time we run `make test`, but only when PACKER_ACC=1 is set in the environment. This allows us to skip executing tests that are either long-running, or that depend on external dependencies (typically Github), which have a higher potential to fail on a normal run of Packer tests.	2 years ago
Lucas Bajolet	d65074c05c	packer_test: dump command outs in case of failure ... When a test fails to exert its assertions on the command-line output, a test fails, but we don't necessarily can troubleshoot what happened, especially when this happens in a CI environment. Therefore, for convenience, we add the faculty for packerCommand.Assert to automatically dump a command's output (both stdout and stderr) if a test fails.	2 years ago
Lucas Bajolet	4a05d19a89	packer_test: add capability to provide stdin ... Some commands need to have an input in order to work. For those, we add the capability for the packerCommand struct to have their stdin defined from a string, which is then fed to the command being executed.	2 years ago
Lucas Bajolet	39a483f762	packer_test: add func to test a cmd multiple times ... When a Packer command is created for testing the tool, we generally run it once, then the command is essentially nooping. This change allows us to run Packer multiple times with the same parameters, and make sure all runs conform to a specific list of checks. This allows us to more reliably test non-deterministic behaviours.	2 years ago
dependabot[bot]	e6acdd5f52	build(deps): bump golang.org/x/net in /packer_test/plugin_tester ... Bumps [golang.org/x/net](https://github.com/golang/net) from 0.17.0 to 0.23.0. - [Commits](https://github.com/golang/net/compare/v0.17.0...v0.23.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com>	2 years ago
dependabot[bot]	5e7607184d	build(deps): bump github.com/go-jose/go-jose/v3 ... Bumps [github.com/go-jose/go-jose/v3](https://github.com/go-jose/go-jose) from 3.0.0 to 3.0.3. - [Release notes](https://github.com/go-jose/go-jose/releases) - [Changelog](https://github.com/go-jose/go-jose/blob/v3.0.3/CHANGELOG.md) - [Commits](https://github.com/go-jose/go-jose/compare/v3.0.0...v3.0.3) --- updated-dependencies: - dependency-name: github.com/go-jose/go-jose/v3 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com>	2 years ago
Lucas Bajolet	6e45bf3c31	packer_test: add remote install with pre-rel test ... Remotely installing plugins with a pre-release as part of the constraint is unsupported by Packer, and should error if that happens. This test makes sure that this gets treated as an error if that's the case, even before attempting to connect to the source.	2 years ago
Lucas Bajolet	8d7a8f126d	packer_test: remove %d from workdir path ... '%d' gets output as-is in the temporary workdir we create. This is unnecessary and could even be problematic in some cases, so we scrub it from the MkdirTemp call.	2 years ago
Lucas Bajolet	8402e00222	packer_test: add test with config.json/components ... Since legacy config files may declare single plugin components, we need to warn that they're not supported anymore. This is in process of being PR'd into main, but to ensure the config works as intended and we do get the error, we add some tests for that.	2 years ago
Wilken Rivera	1d65ad676f	Add test cases for init command ... These changes include a series of test cases for validating packer init using the force and upgrade flag. Include in this test is a test case for validating the plugin installation error when init encounters a plugin whose reported version does not match the version within the plugin name. ``` --- PASS: Test_PackerCoreSuite (18.66s) --- PASS: Test_PackerCoreSuite/TestPackerInitForce (4.91s) --- PASS: Test_PackerCoreSuite/TestPackerInitForce/installs_any_missing_plugins (2.76s) --- PASS: Test_PackerCoreSuite/TestPackerInitForce/reinstalls_plugins_matching_version_constraints (2.14s) --- PASS: Test_PackerCoreSuite/TestPackerInitUpgrade (3.70s) --- PASS: Test_PackerCoreSuite/TestPackerInitUpgrade/upgrades_a_plugin_to_the_latest_matching_version_constraints (2.02s) --- PASS: Test_PackerCoreSuite/TestPackerInitWithMixedVersions (1.96s) --- PASS: Test_PackerCoreSuite/TestPackerInitWithMixedVersions/skips_the_plugin_installation_with_mixed_versions_before_exiting_with_an_error (1.96s) --- PASS: Test_PackerCoreSuite/TestPackerInitWithNonGithubSource (1.22s) --- PASS: Test_PackerCoreSuite/TestPackerInitWithNonGithubSource/try_installing_from_a_non-github_source,_should_fail (0.07s) --- PASS: Test_PackerCoreSuite/TestPackerInitWithNonGithubSource/manually_install_plugin_to_the_expected_source (0.59s) --- PASS: Test_PackerCoreSuite/TestPackerInitWithNonGithubSource/re-run_packer_init_on_same_template,_should_succeed_silently (0.55s) ```	2 years ago
Lucas Bajolet	19594be808	packer_test: add test with both pre/meta in plugin ... To make sure we do scrub the metadata in the plugin name when installing it from a local binary, we add a test that does that installation with both alternatives: 1.0.0-dev and 1.0.0-dev+metadata, which should result in only one alternative being installed (the last one that succeeded).	2 years ago