82 Commits (f2bc9164873bdec092456aba719d7e17d10fa342)

Author	SHA1	Message	Date
Liam Cervante	aaf617d6d7	Add test case to ensure no regression of already fixed crash (#36583)	1 year ago
Samsondeen	3f4ed7e019	Ensure that all states are cleaned up (#36616)	1 year ago
Samsondeen	1bd312f155	Produce detailed diagnostic objects when test run assertions fail (#36522)	1 year ago
Liam Cervante	0acd732911	Fix flaky test in test_test.go (#36566)	1 year ago
Samsondeen	85db2264db	skip flaky test (#36540)	1 year ago
Samsondeen	018110c012	No state to cleaup when it is empty (#36526)	1 year ago
Samsondeen	2e91113224	Terraform test: Consolidate test execution procedure (#36459)	1 year ago
Samsondeen	4161e88391	Continue test execution after an expected failure (#36447)	1 year ago
Samsondeen	7f29df96a0	Allow terraform init when only test files are present in directory (#36429)	1 year ago
Samsondeen	2f0f537e69	Terraform test: Execute eligible test runs in parallel (#36300)	1 year ago
Samsondeen	16a34fe47f	Accept CLI option for the number of parallel ops in a test run's plan/apply (#36323)	1 year ago
Sarah French	39224976a8	Make test fail if JUnit feature is experimental (#36348)	1 year ago
Kévin Andrieux	258f91ab0c	feat: add state_key for test run blocks (#36185) ... * feat: add state_alias for test run blocks * fix: move `state_alias` to run block + rename to `state_key` * docs: add state_key changelog entry * docs: add state_key in run block documentation * Revert "docs: add state_key in run block documentation" This reverts commit ccccf62a76. * Revert "docs: add state_key changelog entry" This reverts commit 86e2ad8dc3.	1 year ago
Liam Cervante	408f323efa	test: also allow mock_data and mock_resource blocks to generate data during planning (#36317)	1 year ago
Sarah French	ab6e4f2299	Refactor JUnit XML output of `terraform test` into a new `junit` package (#36304) ... * Refactor JUnit XML output to use new concept of an Artifact * Move JUnit-related code into new `artifact` package * Refactor Artifact's Save method to return diagnostics, update comments Previously TestJUnitXMLFile implemented the View interface, which cannot return errors. Now it's not a View any more we can simplify things. * Make junitXMLTestReport output deterministic by iterating over a slice instead of a map, add test * Provide sources to junitXMLTestReport, allowing complete error messages in the XML We need to ensure that artifact.NewTestJUnitXMLFile is called once the config Loader is available as a non-nil pointer * Whitespace * Add some test coverage for JUnit XML output for `terraform test` * Refactor how file is saved, add tests * Move XML structs definitions outside of `junitXMLTestReport` * Fix nil pointer bug * Add missing file headers * Refactor comparison of byte slices * Rename package to `junit`, rename structs to match * Add a test showing JUnit output when a test is skipped by the user	1 year ago
Samsondeen	aec7c3cc89	terraform test: allow computed/mocked values override during planning (#36227)	1 year ago
Daniel Schmidt	b701e152e5	show value is ephemeral in terraform test	2 years ago
Daniel Schmidt	3e7fdd0de9	improve wording of error message	2 years ago
Daniel Schmidt	ae103fc491	test: emit a nice error message if the user uses ephemeral resource in terrafom test	2 years ago
Daniel Schmidt	97ba67edea	test: ensure ephemeral values are not shown in diagnostics and warning is printed	2 years ago
Daniel Schmidt	725add6a1a	test: pass in variables during apply ... ephemeral variables need to be re-supplied	2 years ago
MicahKimel	3258744166	Remove invalid warning during cleanup phase (#35172) ... * stop invalid warning during cleanup phase * stop invalid warning during cleanup phase * invalid warning during cleanup phase * remove unwanted comment * update GetVariables parameter for including warnings * invalid warning during cleanup phase test case * update invalid warnings in cleanup test case to throw a warning for the validation variable before cleanup phase * remove unwanted warnings	2 years ago
Liam Cervante	448775102c	terraform test: Push evaluation of variables to as late as possible (#35014) ... * terraform test: Push evaluation of variables to as late as possible * Update internal/moduletest/hcl/variable_cache.go Co-authored-by: kmoe <5575356+kmoe@users.noreply.github.com> * address comments --------- Co-authored-by: kmoe <5575356+kmoe@users.noreply.github.com>	2 years ago
Liam Cervante	6963c54bec	terraform test: don't remove sensitive marks from inputs (#35021)	2 years ago
UKEME BASSEY	0ea25e3b40	fixing tests internal/command/init, view and argumento	2 years ago
UKEME BASSEY	370a471ecb	fix json -help display and failing tests	2 years ago
Martin Atkins	14b08c3745	terraform: Move variable validation into its separate node type ... The parent commit added a new node to the graph for each existing input variable node, but that node didn't do anything yet. Now those new nodes are actually responsible for checking the variable's validation rules, instead of that being included in the same graph node that decides the variable's final value. This also refines the validation node graph transformer to only insert a validation node if there's at least one rule to check, which should avoid bloating the graph for anyone who isn't actually using input variable validation. Therefore the plan graph builder tests revert back to their old expected result because that input configuration does not include any validation rules. For the moment this just moves around functionality that already existed and doesn't add anything new, but having the validation handled in a separate graph node means that we can evaluate the validation rules in the EvalContext of the module where the variable was declared instead of where it was defined, and so a future commit will allow -- initially just as an experiment -- validation rules that can refer to other objects in the module where the variable is declared, including the values of other input variables. As a nice bonus, this fixed a small bug in "terraform test" where problems with variable values defined in the test run were reported as problems with the variable declaration, rather than with the expression assigning the value to it. Since it's now NodeRootVariable that decides the definition source range, rather than the validation-rule-checking helper, it has access to the source range provided in a terraform.InputValue object, which was already populated correctly for per-test-run definitions.	2 years ago
Liam Cervante	c6ff573180	Include provider functions in scope used to evaluate test assertions (#34825)	2 years ago
Daniel Schmidt	7ae7a203d8	test: allow using global variables in suite-level variable definitions ... Closes #34534	2 years ago
Liam Cervante	9658f9df6b	mocking overrides: default to concrete empty object when values are missing (#34563)	2 years ago
Liam Cervante	430970c02b	terraform test: Fix crash when file level variables reference variables. (#34531) ... * terraform test: Fix crash when referencing variables within the file level variable block * also allow functions within variables blocks * add tests	2 years ago
Liam Cervante	7e18b1183d	Add test verifying issue #34476 has been fixed (#34478)	2 years ago
Jamie Finnigan	05b0bdac99	adopt hashicorp/cli fork of mitchellh/cli (#34429)	2 years ago
Liam Cervante	5cbf7c5f96	Add docs for automatic var files in test directories (#34349)	2 years ago
Kubo	c98e355ca1	terraform test: Collect variables from default var file within testing directory (#34341) ... * Add logic to collect variables for terrafrom test * Add tests for test variable collection * Update the test variable collection implementation * Update internal/backend/local/test.go Co-authored-by: Liam Cervante <liam.cervante@hashicorp.com> * Update internal/backend/local/test.go Co-authored-by: Liam Cervante <liam.cervante@hashicorp.com> * Move test variables into var file * resolve diff from cross-branch switch * go fmt --------- Co-authored-by: Liam Cervante <liam.cervante@hashicorp.com>	2 years ago
Martin Atkins	0f845adfdb	moduletest: EvalContext lives outside of the "terraform" package ... The initial implementation of the module testing language included some elements deeply embedded inside "package terraform", which is primarily concerned with the traditional Terraform language, which we might now call the "module language" to distinguish it from the test language and the stacks language. That was a reasonable design decision due to the modules runtime needing to evaluate some expressions in a similar way to how the modules language does it, but unfortunately it caused the test language runtime to depend a little too closely on some current implementation details of how the modules runtime deals with expression evaluation, and those details are about to change as we teach Terraform to be able to deal with unknown values in new places, as part of resolving this issue: https://github.com/hashicorp/terraform/issues/30937 This commit therefore refactors the test runtime a little so that it doesn't intrude so deeply into the module runtime's business. Instead of directly extending the module runtime's expression evaluator, we'll instead call into it in a similar way to how the "terraform console" command does, by wrapping the evaluation data produced by the module runtime to extend it with the only two special things that the test runtime needs: references to the output values of previous runs, and references to test-only variables that aren't declared as part of the module under test. This effectively means that the test runtime can now evaluate a superset of the expressions that would be valid in the global scope of the module under test using only the public API of the modules runtime, with no testing-specific evaluation logic embedded inside the modules runtime. This new strategy is also somewhat similar to how the stacks runtime calls into the modules runtime to evaluate components, but the stacks runtime only actually cares about output values and so doesn't need a full evaluation scope. This means the result here is a little more symmetrical with the approach we took in stacks, which will hopefully allow experience working on one to translate better to the other.	2 years ago
Liam Cervante	9ea1aa0a86	terraform test: skip destroy step for empty run blocks (#34331)	2 years ago
Liam Cervante	392e2a6e21	Fix the compile error in command/test_test.go (#34324)	2 years ago
Liam Cervante	6670ab43f7	terraform test: rearrange the order of destroy operations (#34293)	2 years ago
Liam Cervante	08eafe0f2e	Fix flaky test in command/test_test.go (#34320)	2 years ago
Liam Cervante	dedb296773	terraform test: connect mock behaviour to test framework (#34205) ... * connect mock behaviour to test framework * fix calling unconfigured providers	2 years ago
Liam Cervante	57edb9c248	testing framework: add warnings for override blocks with invalid targets (#34181) ... * testing framework: add warnings for override blocks with invalid targets * pre-add the override sources for upcoming features * address comments	2 years ago
Liam Cervante	b9f6b23ba3	testing framework: add support for functions in variables and providers (#34204)	2 years ago
Liam Cervante	ad77877136	terraform test: remove marks before passing variables as inputs to a plan (#34190) ... * remove marks before passing variables as inputs to a plan * fix doc comments	2 years ago
Liam Cervante	f90d71f723	testing framework: allow providers to reference run blocks (#34118) ... * testing framework: allow providers to reference run block * missing copywrite header	2 years ago
Liam Cervante	5bbdc4c3b6	terraform test: allow providers in test files to reference variables (#34069) ... * Process only relevant variables in the testing framework * add support for variables within test provider configurations * remove unused function * fix consistency checks	3 years ago
Liam Cervante	5294c2bb47	Process only relevant variables in the testing framework (#34072)	3 years ago
Liam Cervante	01c0480e77	Test framework will produce regular updates in -json mode (#33953) ... * Test framework will produce regular updates in -json mode * add tests * docs * docs tweaking	3 years ago
Liam Cervante	fba5dad097	Test files should be marked as skipped if all their run blocks were skipped (#33950)	3 years ago
Liam Cervante	f8d4664bcd	Add additional validation around unknown and null values in test variables (#33861) ... * Add additional validation around unknown values in test variables * address comments	3 years ago