53 Commits (27770ee8053da48fbb684fd23d415aae376d2bed)

Author	SHA1	Message	Date
Sebastien Dionne	1047b5355c	Fix typos and linguistic errors in documentation (#37770) ... Signed-off-by: Sebastien Dionne <survivant00@gmail.com>	4 months ago
Liam Cervante	551ba2e525	Implement controlling destroy functionality within Terraform Test (#37359) ... * Add ability to parse backend blocks present in a test file's run blocks, validate configuration (#36541) * Add ability to parse backend blocks from a run block * Add validation to avoid multiple backend blocks across run blocks that use the same internal state file. Update tests. * Add validation to avoid multiple backend blocks within a single run block. Update tests. * Remove use of quotes in diagnostic messages * Add validation to avoid backend blocks being used in plan run blocks. Update tests. * Correct local backend blocks in new test fixtures * Add test to show that different test files can use same backend block for same state key. * Add validation to enforce state-storage backend types are used * Remove TODO comment We only need to consider one file at a time when checking if a state_key already has a backend associated with it; parallelism in `terraform test` is scoped down to individual files. * Add validation to assert that the backend block must be in the first apply command for an internal state * Consolidate backend block validation inside a single if statement * Add initial version of validation that ensures a backend isn't re-used within a file * Explicitly set the state_key at the point of parsing the config TODO: What should be done with method (moduletest.Run).GetStateKey? * Update test fixture now that reusing backend configs has been made invalid * Add automated test showing validation of reused configuration blocks * Skip test due to flakiness, minor change to test config naming * Update test so it tolerates non-deterministic order run blocks are evaluated in * Remove unnecessary value assignment to r.StateKey * Replace use of GetStateKey() with accessing the state key that's now set during test config parsing * Fix bug so that run blocks using child modules get the correct state key set at parsing time * Update acceptance test to also cover scenario where root and child module state keys are in use * Update test name * Add newline to regex * Ensure consistent place where repeat backend error is raised from * Write leftover test state(s) to file (#36614) * Add additional validation that the backend used in a run is a supported type (#36648) * Prevent test run when leftover state data is present (#36685) * `test`: Set the initial state for a state files from a backend, allow the run that defines a backend to write state to the backend (#36646) * Allow use of backend block to set initial state for a state key * Note about alternative place to keep 'backend factories' * Allow the run block defining the backend to write state to it * Fix rebase * Change to accessing backend init functions via ContextOpts * Add tests demonstrating how runs containing backend blocks use and update persisted state * Fix test fixture * Address test failure due to trouble opening the state file This problem doesn't happen on MacOS, so I assume is due to the Linux environment of GitHub runners. * Fix issue with paths properly I hope * Fix defect in test assertion * Pivot back to approach introduced in 4afc3d7 * Let failing tests write to persistent state, add test case covering that. I split the acceptance tests into happy/unhappy paths for this, which required some of the helper functions' declarations to be raised up to package-level. * Change how we update internal state files, so that information about the associated backend is never lost * Fix UpdateStateFile * Ensure that the states map set by TestStateTransformer associates a backend with the correct run. * Misc spelling fixes in comments and a log * Replace state get/set functions with existing helpers (#36747) * Replace state get/set functions with existing helpers * Compare to string representation of state * Compare to string representation of state * Terraform Test: Allow skipping cleanup of entire test file or individual run blocks (#36729) * Add validation to enforce skip_cleanup=false cannot be used with backend blocks (#36857) * Integrate use of backend blocks in tests with skip_cleanup feature (#36848) * Fix nil pointer error, update test to not be table-driven * Make using a backend block implicitly set skip_cleanup to true * Stop state artefacts being created when a backend is in use and no cleanup errors have occurred * Return diagnostics so calling code knows if cleanup experienced issues or not * Update tests to show that when cleanup fails a state artefact is created * Add comment about why diag not returned * Bug fix - actually pull in the state from the state manager! * Split and simplify (?) tests to show the backend block can create and/or reuse prior state * Update test to use new fixtures, assert about state artefact. Fix nil pointer * Update test fixture in use, add guardrail for flakiness of forced error during cleanup * Refactor so resource ID set in only one place * Add documentation for using a `backend` block during `test` (#36832) * Add backend as a documented block in a run block * Add documentation about backend blocks in run blocks. * Make the relationship between backends and state keys more clear, other improvements * More test documentation (#36838) * Terraform Test: cleanup command (#36847) * Allow cleanup of states that depend on prior runs outputs (#36902) * terraform test: refactor graph edge calculation * create fake run block nodes during cleanup operation * tidy up TODOs * fix tests * remove old changes * Update internal/moduletest/graph/node_state_cleanup.go Co-authored-by: Samsondeen <40821565+dsa0x@users.noreply.github.com> * Improve diagnostics around skip_cleanup conflicts (#37385) * Improve diagnostics around skip_cleanup conflicts * remove unused dynamic node * terraform test: refactor manifest file for simplicity (#37412) * test: refactor apply and plan functions so no run block is needed * terraform test: write and load state manifest files * Terraform Test: Allow skipping cleanup of entire test file or individual run blocks (#36729) * terraform test: add support for skip_cleanup attr * terraform test: add cleanup command * terraform test: add backend blocks * pause * fix tests * remove commented code * terraform test: make controlling destroy functionality experimental (#37419) * address comments * Update internal/moduletest/graph/node_state_cleanup.go Co-authored-by: Samsondeen <40821565+dsa0x@users.noreply.github.com> --------- Co-authored-by: Samsondeen <40821565+dsa0x@users.noreply.github.com> * add experimental changelog entries --------- Co-authored-by: Sarah French <15078782+SarahFrench@users.noreply.github.com> Co-authored-by: Samsondeen <40821565+dsa0x@users.noreply.github.com> Co-authored-by: Samsondeen Dare <samsondeen.dare@hashicorp.com>	6 months ago
Liam Cervante	dec0edfd5a	terraform test: enable deferrals in Terraform Test with the specified flag (#37370)	7 months ago
Samsondeen	a1332299a5	Test run Parallelism of 1 should not result in deadlock (#37292)	8 months ago
Mattias Fjellström	3252fa8f42	Add missing JUnit XML flag to test command help string (#37161) ... The test command supports outputting a JUnit XML file with the -junit-xml flag. This flag was missing from the test command help string.	9 months ago
Samsondeen	16a34fe47f	Accept CLI option for the number of parallel ops in a test run's plan/apply (#36323)	1 year ago
Sarah French	21cafd70ea	Add more details to JUnit `terraform test` output to describe why a test was skipped (#36315) ... * Add ability for TestJUnitXMLFile to access data about whether the test runner was Stopped * Add details to XML describing why a Run was skipped * Fix wording * Code consistency changes * Move all JUnit-related code down to where it's used Away from the Views section of the code where it was relevant before * Move JUnit-related error and warning diags to above where cancellable contexts are created * Fix wording of user feedback * Fix test to match updated skipped message text * Fix test	1 year ago
Sarah French	8e1d36681c	Promote JUnit output 'terraform test' feature from experimental status, make incompatibility with remote test execution explicit via flag validation (#36324) ... * Promote JUnit reports for `terraform test` out of experimental status * Make JUnit output explicitly for local execution only * Refactor how local test runner is passed JUnit data * Add change file * Add test for incompatible flags	1 year ago
Sarah French	ab6e4f2299	Refactor JUnit XML output of `terraform test` into a new `junit` package (#36304) ... * Refactor JUnit XML output to use new concept of an Artifact * Move JUnit-related code into new `artifact` package * Refactor Artifact's Save method to return diagnostics, update comments Previously TestJUnitXMLFile implemented the View interface, which cannot return errors. Now it's not a View any more we can simplify things. * Make junitXMLTestReport output deterministic by iterating over a slice instead of a map, add test * Provide sources to junitXMLTestReport, allowing complete error messages in the XML We need to ensure that artifact.NewTestJUnitXMLFile is called once the config Loader is available as a non-nil pointer * Whitespace * Add some test coverage for JUnit XML output for `terraform test` * Refactor how file is saved, add tests * Move XML structs definitions outside of `junitXMLTestReport` * Fix nil pointer bug * Add missing file headers * Refactor comparison of byte slices * Rename package to `junit`, rename structs to match * Add a test showing JUnit output when a test is skipped by the user	1 year ago
James Bardin	3a1a9408d9	fix Descendants spelling ... this has proliferated everywhere	1 year ago
Nate Douglas	df653772e1	Fix typo in test.go	1 year ago
Sebastian Rivera	015f795ff0	Rebrand Terraform Cloud to HCP Terraform	2 years ago
UKEME BASSEY	c7bbc09631	move all command args from command/init to args/init	2 years ago
Martin Atkins	b43ddd8ff6	terraform test: Experimental JUnit XML reporting ... This adds an experimental new option -junit-xml=FILENAME for the "terraform test" command. When specified, it writes a JUnit XML report to the specified filename once the test run is complete, while continuing to report test progress in the UI in the usual way. This is only experimental for now because it remains to be seen if this particular mapping to the JUnit XML schema is actually useful in real software -- this format is woefully underdocumented and implemented slightly differently by each consumer -- and so we might change this significantly before stabilizing it, or remove it altogether if it turns out that there's no useful mapping to JUnit XML here. Hopefully those who are interested in JUnit XML reports will try this experiment against their favorite JUnit XML-consuming software and report back whether the report is presented in a helpful way. It's a de-facto convention for JUnit XML to be reported separately to a file, rather than replacing the normal test run output, since tools that consume this format tend to present its results in a separate and less prominent place than the output of the command itself. This option is designed to follow that convention for consistency with various other software that produces this format. The implementation here is intentionally pretty minimal and simplistic just as a starting point for gathering feedback. The main priority is that it be easy to evolve this based on feedback and to remove it altogether if we decide not to stabilize this at all. If this does become stabilized, it might deserve being factored out into a separate package so that we can minimize the amount of logic embedded directly inside the views package, and it will certainly need some unit tests to represent what we've committed to supporting in future versions.	2 years ago
Kubo	c98e355ca1	terraform test: Collect variables from default var file within testing directory (#34341) ... * Add logic to collect variables for terrafrom test * Add tests for test variable collection * Update the test variable collection implementation * Update internal/backend/local/test.go Co-authored-by: Liam Cervante <liam.cervante@hashicorp.com> * Update internal/backend/local/test.go Co-authored-by: Liam Cervante <liam.cervante@hashicorp.com> * Move test variables into var file * resolve diff from cross-branch switch * go fmt --------- Co-authored-by: Liam Cervante <liam.cervante@hashicorp.com>	2 years ago
Liam Cervante	8ecda5f708	Increase timeout length after fatal interrupts for test cloud runs (#33956)	2 years ago
Liam Cervante	ffbcaf8bef	Implement TFC integration for private module registry test runs (#33710) ... * implement module testing via TFC * ready for review * fix static checks * licence headers	2 years ago
Liam Cervante	4c83e2d524	Terraform test directory must be local to the config directory (#33760)	3 years ago
Liam Cervante	edcc35c96c	testing framework: move local implementation into local directory (#33734)	3 years ago
Jared Baker	c566bcd6f7	Merge pull request #33730 from hashicorp/s3/modernization ... backend/s3: Update S3 client to AWS SDK Go V2	3 years ago
Liam Cervante	5e63aa01c7	Allow referencing higher level variables from run blocks (#33722)	3 years ago
Jared Baker	de02f3461b	internal/command: fix slices.SortFunc signature	3 years ago
CJ Horton	98b81d3c50	add missing copyright headers	3 years ago
Liam Cervante	9742f22c4e	Introduce 'run' keyword for referencing outputs from earlier run blocks (#33683) ... * introduce 'run' keyword for referencing outputs from earlier run blocks * fix code consistency	3 years ago
Liam Cervante	836c64f172	simplify test_context evaluation (#33677)	3 years ago
Liam Cervante	126f34e4e8	Implement updates to testing framework based on recent feedback	3 years ago
Liam Cervante	3bea1171af	test framework: expand variables available to test assertions (#33611)	3 years ago
Liam Cervante	4560a83721	test framework: include transformed config before validating config (#33608)	3 years ago
Liam Cervante	e1019b3641	testing framework: validate the configuration before terraform test (#33559) ... * testing framework: call validate on the configuration before running terraform test * address comments * make tests pass after merge * fix tests	3 years ago
Liam Cervante	080ddead6e	testing framework: perform a plan before starting any tests (#33561)	3 years ago
Liam Cervante	083c9afd5b	testing framework: add trace and debug statements during test executions (#33554)	3 years ago
Liam Cervante	c91f91cc9e	testing framework: change test file extension to .tftest.hcl (#33553)	3 years ago
Liam Cervante	6c7db16566	testing framework: refactor interrupt logic for immediate exits (#33532) ... * testing framework: refactor interrupt logic * fix formatting	3 years ago
Liam Cervante	6882dd9530	testing framework: introduce test command optional flags (#33504) ... * testing framework: introduce test command optional flags * address consistency checks	3 years ago
Liam Cervante	2cc81cfec6	testing framework: implement panic handling (#33525)	3 years ago
Liam Cervante	837716a703	testing framework: finalise expect_failures functionality (#33506)	3 years ago
Liam Cervante	4862812c94	testing framework: introduce interrupts for stopping tests (#33477) ... * [testing framework] prepare for beta phase of development * [Testing Framework] Add module block to test run blocks * [testing framework] allow tests to define and override providers * testing framework: introduce interrupts for stopping tests * remove panic handling, will do it properly later	3 years ago
Liam Cervante	4b34902fab	[testing framework] allow tests to define and override providers (#33466) ... * [testing framework] prepare for beta phase of development * [Testing Framework] Add module block to test run blocks * [testing framework] allow tests to define and override providers	3 years ago
Liam Cervante	5acc95dda7	[Testing Framework] Add module block to test run blocks (#33456) ... * [testing framework] prepare for beta phase of development * [Testing Framework] Add module block to test run blocks	3 years ago
Liam Cervante	ea162f6ab5	[testing framework] prepare for beta phase of development (#33445)	3 years ago
Liam Cervante	2eb99a0e44	Fix the Terraform build after recent merges (#33436)	3 years ago
Liam Cervante	dfc26c2ac4	[Testing Framework] Add test command to Terraform CLI (#33327) ... * Add test structure to views package for rendering test output * Add test file HCL configuration and parser functionality * Adds a TestContext structure for evaluating assertions against the state and plan * Add test command to Terraform CLI	3 years ago
Liam Cervante	48c818927c	Remove old implementation of terraform test command	3 years ago
hashicorp-copywrite[bot]	325d18262e	[COMPLIANCE] Add Copyright and License Headers	3 years ago
Alisdair McDiarmid	8df065a2fe	initwd: Switch from earlyconfig to configs ... This is a mostly mechanical refactor with a handful of changes which are necessary due to the semantic difference between earlyconfig and configs. When parsing root and descendant modules in the module installer, we now check the core version requirements inline. If the Terraform version is incompatible, we drop any other module loader diagnostics. This ensures that future language additions don't clutter the output and confuse the user. We also add two new checks during the module load process: * Don't try to load a module with a `nil` source address. This is a necessary change due to the move away from earlyconfig. * Don't try to load a module with a blank name (i.e. `module ""`). Because our module loading manifest uses the stringified module path as its map key, this causes a collision with the root module, and a later panic. This is the bug which triggered this refactor in the first place.	3 years ago
Radek Simko	7feef1c4aa	Use hashicorp/terraform-registry-address as a decoupled library (#28338) ... * refactor: Use tfaddr for provider address parsing * refactor: Use tfaddr for module address parsing * deps: introduce hashicorp/terraform-registry-address	4 years ago
kmoe	40ec62c139	command: make module installation interruptible ... Earlier work to make "terraform init" interruptible made the getproviders package context-aware in order to allow provider installation to be cancelled. Here we make a similar change for module installation, which is now also cancellable with SIGINT. This involves plumbing context through initwd and getmodules. Functions which can make network requests now include a context parameter whose cancellation cancels those requests. Since the module installation code is shared, "terraform get" is now also interruptible during module installation.	4 years ago
Martin Atkins	89b05050ec	core: Functional-style API for terraform.Context ... Previously terraform.Context was built in an unfortunate way where all of the data was provided up front in terraform.NewContext and then mutated directly by subsequent operations. That made the data flow hard to follow, commonly leading to bugs, and also meant that we were forced to take various actions too early in terraform.NewContext, rather than waiting until a more appropriate time during an operation. This (enormous) commit changes terraform.Context so that its fields are broadly just unchanging data about the execution context (current workspace name, available plugins, etc) whereas the main data Terraform works with arrives via individual method arguments and is returned in return values. Specifically, this means that terraform.Context no longer "has-a" config, state, and "planned changes", instead holding on to those only temporarily during an operation. The caller is responsible for propagating the outcome of one step into the next step so that the data flow between operations is actually visible. However, since that's a change to the main entry points in the "terraform" package, this commit also touches every file in the codebase which interacted with those APIs. Most of the noise here is in updating tests to take the same actions using the new API style, but this also affects the main-code callers in the backends and in the command package. My goal here was to refactor without changing observable behavior, but in practice there are a couple externally-visible behavior variations here that seemed okay in service of the broader goal: - The "terraform graph" command is no longer hooked directly into the core graph builders, because that's no longer part of the public API. However, I did include a couple new Context functions whose contract is to produce a UI-oriented graph, and _for now_ those continue to return the physical graph we use for those operations. There's no exported API for generating the "validate" and "eval" graphs, because neither is particularly interesting in its own right, and so "terraform graph" no longer supports those graph types. - terraform.NewContext no longer has the responsibility for collecting all of the provider schemas up front. Instead, we wait until we need them. However, that means that some of our error messages now have a slightly different shape due to unwinding through a differently-shaped call stack. As of this commit we also end up reloading the schemas multiple times in some cases, which is functionally acceptable but likely represents a performance regression. I intend to rework this to use caching, but I'm saving that for a later commit because this one is big enough already. The proximal reason for this change is to resolve the chicken/egg problem whereby there was previously no single point where we could apply "moved" statements to the previous run state before creating a plan. With this change in place, we can now do that as part of Context.Plan, prior to forking the input state into the three separate state artifacts we use during planning. However, this is at least the third project in a row where the previous API design led to piling more functionality into terraform.NewContext and then working around the incorrect order of operations that produces, so I intend that by paying the cost/risk of this large diff now we can in turn reduce the cost/risk of future projects that relate to our main workflow actions.	5 years ago
Martin Atkins	36d0a50427	Move terraform/ to internal/terraform/ ... This is part of a general effort to move all of Terraform's non-library package surface under internal in order to reinforce that these are for internal use within Terraform only. If you were previously importing packages under this prefix into an external codebase, you could pin to an earlier release tag as an interim solution until you've make a plan to achieve the same functionality some other way.	5 years ago
Martin Atkins	f40800b3a4	Move states/ to internal/states/ ... This is part of a general effort to move all of Terraform's non-library package surface under internal in order to reinforce that these are for internal use within Terraform only. If you were previously importing packages under this prefix into an external codebase, you could pin to an earlier release tag as an interim solution until you've make a plan to achieve the same functionality some other way.	5 years ago