10 Commits (9d3cdc5dd2079f1d0eb73cba6737bdc3dccc3d4d)

Author	SHA1	Message	Date
Lucas Bajolet	56400f27cb	packer_test: add gob/pb test suite ... With the draft to support both gob and protobuf as serialisation formats for Packer, along with the SDK changes that propel them, we add a series of tests that make sure the logic that picks which protocol is solid and functional. These tests rely on building several versions of the tester plugin, with and without protobuf support, to then install them in the tests as needed to test the logic of Packer using packer build with them, and templates that require multiple plugins.	1 year ago
Lucas Bajolet	3c449430e1	packer_test: make PluginTestDir a structure ... In order for the creation of a temporary directory to install plugins into to be simpler to understand and use, we change how the directory is created, cleaned-up, and installs plugins into. Now, instead of a tuple of a string (path) and a cleanup function, we return a structure that comprises the test suite, and the temporary directory, along with methods to handle those steps independently.	2 years ago
Lucas Bajolet	31b6109430	packer_test: hide run and introduce Output ... When using a PackerCommand, the Run function was made public as a way to access the contents of an execution. This was clumsy as it had too many responsabilities, and was not needed strictly as Assert was performing the executions, as many times as required. This could introduce cases in which one run as spent by the caller, then the remainder were executed through Assert. Therefore, we change this convention. Now, run is private to the type, and only through Assert can a command be executed. If a test needs access to a command's output, stderr, or error, it can do so through the Output function, which requires Assert to be called first.	2 years ago
Lucas Bajolet	71169416db	packer_test: add func to change assert behaviour ... When a command asserts its output with checkers, by default it will register errors through a t.Errorf. While this works, in some cases we would want to stop execution immediately if a function's Assert fails, as the rest of the test may depend on the assertion being valid. In the current state, this means either getting the result of the run to check if an error was returned (not fully reliable as if the command was run multiple times, and the last run succeeded, we won't get an error), or relying on t.IsFailed() (completely reliable). Instead, we introduce a new function on packerCommand, that lets users change how Assert behaves, so that if an error was reported, instead of logging the error and flagging the test as failed, we can use t.Fatalf, so that the test immedately fails and stops execution.	2 years ago
Lucas Bajolet	35b2317ef3	packer_test: rename/split lib into common/check ... The lib name for the common components for writing packer_test suites was not clear, and did not follow the convention established in Packer core and plugins. Therefore this commit does two things: first the lib is renamed into common as to follow this convention, and clearly document which components are common to all tests. Also checkers are placed in a subpackage of common, common/check, so that it is clearer what is meant to be used as checks for a command's execution status after it's been run, as part of Assert.	2 years ago
Lucas Bajolet	6fc2932b09	packer_test: reorganise plugins test suite init	2 years ago
Lucas Bajolet	3c8a51bdbf	packer_test: split BuildPackerPlugin in build/get ... The interface for building a plugin through the test suite was confusing, as it would build the plugin and return its path, cache the path for the version built, and return the path regardless if something was built or not. While in the current state this is harmless as builds are idempotent, since the state of the plugin package/module does not change, this will in the future as we introduce customisation techniques on the plugin's directory and files, making this double-use potentially dangerous. Furthermore, the current behaviour is unclear, as the function hides that caching mechanism, which could come as a surprise for users attempting to build a plugin for the duration of a test, while the built plugin is linked to the test suite being run, and not the unit test being evaluated. Therefore this commit changes the sequence in which plugins are built and used. Now the `CompilePlugin` function builds a plugin, and does not return its path anymore, instead terminating the tests immediately if they fail. In normal test usage, a new `GetPluginPath` function is introduced, which looks-up the path in the suite's cache, failing immediately if invoked before the plugin is built. With this change, it is heavily advised to build plugins when initialising the suite, then in the tests, the GetPluginPath function should be used to get a plugin's path for interacting with packer commands.	2 years ago
Lucas Bajolet	aba64b457c	packer_test: rename init function for base suite	2 years ago
Lucas Bajolet	dbd441f6f3	packer_test: add convenience func for invert grep ... As inverted grep is something we do rather often, we add a convenience function to create an inverted grep gadget.	2 years ago
Lucas Bajolet	9e4452329f	packer_test: make packer test suite modular ... Having only one test suite for the whole of Packer makes it harder to segregate between test types, and makes for a longer runtime as no tests run in parallel by default. This commit splits the packer_test suite into several components in order to make extension easier. First we have `lib`: this package embeds the core for running Packer test suites. This ships facilities to build your own test suite for Packer core, and exposes convenience methods and structures for building plugins, packer core, and use it to run a test suite in a temporary directory. Then we have two separate test suites: one for plugins, and one for core itself, the latter of which does not depend on plugins being compiled at all. This sets the stage for more specialised test suites in the future, each of which can run in parallel on different parts of the code.	2 years ago