terraform

Commit Graph

Author	SHA1	Message	Date
Samsondeen	ccf00c29ba	terraform test: return error when provider config is invalid (#38140 )	4 months ago
github-actions[bot]	530f2b30ef	Backport of Colorize diag details into v1.14 (#38148 )	4 months ago
github-actions[bot]	b0ebd07f11	providers lock: include test files when loading configuration (#37904 ) Co-authored-by: Liam Cervante <liam.cervante@hashicorp.com>	7 months ago
Liam Cervante	e2327fdc31	Backport of ephemeral fixes into v1.14 (#37850 ) * ephemerals: allow root ephemeral outputs in stacks and test * remember to set new opt for apply stage * Stacks: temporarily disable ephemeral outputs in Stack components * don't write ephemeral outputs to state	7 months ago
github-actions[bot]	b7b7fdc825	Backport of Allow inconsistent filesystem function results for provider configuration into v1.14 (#37870 ) * allow inconsistent function results for providers Due to the inherently ephemeral nature of provider configuration, inconsistent function results were tolerated while evaluating provider config. This loophole was found to be used by a number of configurations, which took advantage of it to create the equivalent of ephemeral values before they formally existed in the language. In order to work around this, we create a special evaluation scope just for providers, allowing us to override the results check for filesystem functions. I've opted to not further clutter the EvalContext interface since this is intended to be a temporary workaround, and does not contribute to the testing of that interface (the interface is solely for internal unit tests anyway). * provider eval scope test * CHANGELOG --------- Co-authored-by: James Bardin <j.bardin@gmail.com> Co-authored-by: Daniel Schmidt <danielmschmidt92@gmail.com>	7 months ago
github-actions[bot]	129d7bd4d4	query: add -query flag to validate command (#37671 ) (#37707 ) Co-authored-by: Samsondeen <40821565+dsa0x@users.noreply.github.com>	8 months ago
github-actions[bot]	94af117ea4	Backport of evaluate: return diagnostics instead of unknown for uninitialised locals and resources into v1.14 (#37667 ) * no-op commit due to failed cherry-picking * evaluate: return diagnostics instead of unknown for uninitialised locals and resources * changelog * also input variables --------- Co-authored-by: temp <temp@hashicorp.com> Co-authored-by: Liam Cervante <liam.cervante@hashicorp.com>	9 months ago
Daniel Schmidt	d83cd73ecf	fix action reference	9 months ago
Samsondeen	8097374c3d	Fix panic when generating json config for tf query (#37596 )	9 months ago
Liam Cervante	551ba2e525	Implement controlling destroy functionality within Terraform Test (#37359 ) * Add ability to parse backend blocks present in a test file's run blocks, validate configuration (#36541) * Add ability to parse backend blocks from a run block * Add validation to avoid multiple backend blocks across run blocks that use the same internal state file. Update tests. * Add validation to avoid multiple backend blocks within a single run block. Update tests. * Remove use of quotes in diagnostic messages * Add validation to avoid backend blocks being used in plan run blocks. Update tests. * Correct local backend blocks in new test fixtures * Add test to show that different test files can use same backend block for same state key. * Add validation to enforce state-storage backend types are used * Remove TODO comment We only need to consider one file at a time when checking if a state_key already has a backend associated with it; parallelism in `terraform test` is scoped down to individual files. * Add validation to assert that the backend block must be in the first apply command for an internal state * Consolidate backend block validation inside a single if statement * Add initial version of validation that ensures a backend isn't re-used within a file * Explicitly set the state_key at the point of parsing the config TODO: What should be done with method (moduletest.Run).GetStateKey? * Update test fixture now that reusing backend configs has been made invalid * Add automated test showing validation of reused configuration blocks * Skip test due to flakiness, minor change to test config naming * Update test so it tolerates non-deterministic order run blocks are evaluated in * Remove unnecessary value assignment to r.StateKey * Replace use of GetStateKey() with accessing the state key that's now set during test config parsing * Fix bug so that run blocks using child modules get the correct state key set at parsing time * Update acceptance test to also cover scenario where root and child module state keys are in use * Update test name * Add newline to regex * Ensure consistent place where repeat backend error is raised from * Write leftover test state(s) to file (#36614) * Add additional validation that the backend used in a run is a supported type (#36648) * Prevent test run when leftover state data is present (#36685) * `test`: Set the initial state for a state files from a backend, allow the run that defines a backend to write state to the backend (#36646) * Allow use of backend block to set initial state for a state key * Note about alternative place to keep 'backend factories' * Allow the run block defining the backend to write state to it * Fix rebase * Change to accessing backend init functions via ContextOpts * Add tests demonstrating how runs containing backend blocks use and update persisted state * Fix test fixture * Address test failure due to trouble opening the state file This problem doesn't happen on MacOS, so I assume is due to the Linux environment of GitHub runners. * Fix issue with paths properly I hope * Fix defect in test assertion * Pivot back to approach introduced in `4afc3d7` * Let failing tests write to persistent state, add test case covering that. I split the acceptance tests into happy/unhappy paths for this, which required some of the helper functions' declarations to be raised up to package-level. * Change how we update internal state files, so that information about the associated backend is never lost * Fix UpdateStateFile * Ensure that the states map set by TestStateTransformer associates a backend with the correct run. * Misc spelling fixes in comments and a log * Replace state get/set functions with existing helpers (#36747) * Replace state get/set functions with existing helpers * Compare to string representation of state * Compare to string representation of state * Terraform Test: Allow skipping cleanup of entire test file or individual run blocks (#36729) * Add validation to enforce skip_cleanup=false cannot be used with backend blocks (#36857) * Integrate use of backend blocks in tests with skip_cleanup feature (#36848) * Fix nil pointer error, update test to not be table-driven * Make using a backend block implicitly set skip_cleanup to true * Stop state artefacts being created when a backend is in use and no cleanup errors have occurred * Return diagnostics so calling code knows if cleanup experienced issues or not * Update tests to show that when cleanup fails a state artefact is created * Add comment about why diag not returned * Bug fix - actually pull in the state from the state manager! * Split and simplify (?) tests to show the backend block can create and/or reuse prior state * Update test to use new fixtures, assert about state artefact. Fix nil pointer * Update test fixture in use, add guardrail for flakiness of forced error during cleanup * Refactor so resource ID set in only one place * Add documentation for using a `backend` block during `test` (#36832) * Add backend as a documented block in a run block * Add documentation about backend blocks in run blocks. * Make the relationship between backends and state keys more clear, other improvements * More test documentation (#36838) * Terraform Test: cleanup command (#36847) * Allow cleanup of states that depend on prior runs outputs (#36902) * terraform test: refactor graph edge calculation * create fake run block nodes during cleanup operation * tidy up TODOs * fix tests * remove old changes * Update internal/moduletest/graph/node_state_cleanup.go Co-authored-by: Samsondeen <40821565+dsa0x@users.noreply.github.com> * Improve diagnostics around skip_cleanup conflicts (#37385) * Improve diagnostics around skip_cleanup conflicts * remove unused dynamic node * terraform test: refactor manifest file for simplicity (#37412) * test: refactor apply and plan functions so no run block is needed * terraform test: write and load state manifest files * Terraform Test: Allow skipping cleanup of entire test file or individual run blocks (#36729) * terraform test: add support for skip_cleanup attr * terraform test: add cleanup command * terraform test: add backend blocks * pause * fix tests * remove commented code * terraform test: make controlling destroy functionality experimental (#37419) * address comments * Update internal/moduletest/graph/node_state_cleanup.go Co-authored-by: Samsondeen <40821565+dsa0x@users.noreply.github.com> --------- Co-authored-by: Samsondeen <40821565+dsa0x@users.noreply.github.com> * add experimental changelog entries --------- Co-authored-by: Sarah French <15078782+SarahFrench@users.noreply.github.com> Co-authored-by: Samsondeen <40821565+dsa0x@users.noreply.github.com> Co-authored-by: Samsondeen Dare <samsondeen.dare@hashicorp.com>	9 months ago
Liam Cervante	00d680d819	apply: include sensitive metadata when comparing changed input values (#37582 ) * apply: include sensitive metadata when comparing changed input values * correct changelog entry	9 months ago
Samsondeen	af8cc7e315	test: create all edges to leaf cleanup nodes (#37564 )	9 months ago
Sarah French	61a7914dd9	Add test helper to create mock provider that contains a state store (#37556 ) * Add a helper for mocking a provider with a state store * Update tests to use the new helper * Update other test fixtures for consistency The related tests pass before and after this change * Update testing helper to return the underlying type, not the interface	9 months ago
Liam Cervante	f931bc4e0e	test: don't panic when resolving references that haven't been evaluated (#37484 )	10 months ago
Ricard Bejarano	4ac49bad2b	terraform test: fix regression when running with no tests (#37475 ) (#37477 ) * terraform test: fix regression when running with no tests (#37475) * added changelog entry * move changelog entry from v1.14 to v1.13 * Return moduletest.Error instead of suite.Status if test operation is cancelled * Add "terraform test" test with no tests Yes, that's right. It's OK if you need to read it again.	10 months ago
Sarah French	da76dba3dc	PSS: Add alternative, experimental version of `init` command that downloads providers in two stages (#37350 ) * Add forked version of `run` logic that's only used if experiments are enabled * Reorder actions in experimental init - load in full config before configuring the backend. * Add getProvidersFromConfig method, initially as an exact copy of getProviders * Make getProvidersFromConfig not use state to get providers * Add `appendLockedDependencies` method to `Meta` to allow multi-phase saving to the dep locks file * Update experimental init to use new getProvidersFromConfig method * Add new getProvidersFromState method that only accepts state information as input for getting providers. Use in experimental init and append values to existing deps lock file * Update messages sent to view about provider download phases * Change init to save updates to the deps lock file only once * Make Terraform output report that a lock file _will_ be made after providers are determined from config * Remove use of `ProviderDownloadOutcome`s * Move repeated code into separate method * Change provider download approach: determine if locks changed at point of attempting to update the lockfile, keep record of incomplete providers inside init command struct * Refactor `mergeLockedDependencies` and update test * Add comments to provider download methods * Fix issue where incorrect message ouput to view when downloading providers * Update `mergeLockedDependencies` method to be more generic * Update `getProvidersFromState` method to receive in-progress config locks and merge those with any locks on file. This allows re-use of providers downloaded by `getProvidersFromConfig` in the same init command * Fix config for `TestInit_stateStoreBlockIsExperimental` * Improve testing of mergeLockedDependencies; state locks are always missing version constraints * Add tests for 2 phase provider download * Add test case to cover use of the `-upgrade` flag * Change the message shown when a provider is reused during the second provider download step. When downloading providers described only in the state then the provider may already be downloaded from a previous init (i.e. is recorded in the deps lock file) or downloaded during step 1 of provider download. The message here needs to cover both potential scenarios. * Update mergeLockedDependencies comment * fix: completely remove use of upgrade flag in getProvidersFromState * Fix: avoid nil pointer errors by returning an empty collection of locks when there is no state * Fix: use state store data only in diagnostic * Change how we make PSS experimental - avoid relying on a package level variable that causes tests to interact. * Remove full-stop in view message, update tests * Update span names to be unique * Re-add lost early returns * Remove unused view messages * Add comments to new view messages	10 months ago
Daniel Schmidt	ecfbdaf18d	add number of action invocations to change summary	10 months ago
Daniel Schmidt	102a561e3d	add actions to the planfile config	10 months ago
Samsondeen	3f7d043566	Append warning in CLI output when query result is empty (#37379 )	10 months ago
Liam Cervante	dec0edfd5a	terraform test: enable deferrals in Terraform Test with the specified flag (#37370 )	11 months ago
Liam Cervante	84c2e2f9e5	terraform test: override prevent_destroy meta attribute (#37364 )	11 months ago
Samsondeen	36836fd456	Tests for query command outputs (#37343 )	11 months ago
Daniel Banck	3607089016	Support tfquery files in `terraform fmt` (#37347 )	11 months ago
Liam Cervante	0a7701f19a	terraform test: refactor graph edge calculation (#37357 ) * terraform test: refactor graph edge calculation * remove unused attribute	11 months ago
Sarah French	d1e412fcf0	PSS: Add initial (incomplete) version of code changes to the `init` command for using pluggable state storage (#37321 ) * Store the FQN of the provider used in PSS in representations of the parsed config. This can only be done once modules have been parsed and the required providers data is available. There are multiple places where config is parsed, into either Config or Module structs, so this needs to be implemented in multiple places. * Update affected tests, improve error diagnostic * Begin enabling method to return a backend.Backend made using state_store config. State store config can now be received via BackendOpts and there is rough validation of whether the config makes sense (does the provider offer a store with the given name?). * Update code's logic to include possibility of a state store being in use At this point there are no cases that actually handle the state store scenarios though! * Add empty cases for handle all broad init scenarios involving PSS * Update default case's error to report state store variables * Improve how we resolve the builtin terraform provider's address * Add test that hits the code path for adding a state store to a new (or implied local) project * Add test for use of `-reconfigure` flag; show that it hits the code path for adding a state store for the first time * Add test that hits the code path for removing use of a state store, migrating to (implied) local backend * Add test that hits the code path for changing a state store's configuration * Update existing test names to be backend-specific * Add tests that hits the code path for migrating between PSS and backends * Consolidate PSS-related tests at end of the file * Fix log text * Add test showing that using variables is disallowed with state_store and nested provider blocks * Update test name * Fix test cases * Add TODOs so we remember to remove experiments from tests * Update state store-related tests to use t.Cleanup * Remove use of `testChdir`	11 months ago
Samsondeen	a1332299a5	Test run Parallelism of 1 should not result in deadlock (#37292 )	11 months ago
Sarah French	b3d7dae793	Add config parsing and validation for PSS (#37178 ) * Enable parsing of blocks in individual files Co-authored-by: Radek Simko <radeksimko@users.noreply.github.com> * Enable handling of state stores when parsing a module from its constituent files. This includes: validations of duplicates and clashes, supporting override files. Co-authored-by: Radek Simko <radeksimko@users.noreply.github.com> * Add tests for validation that detects conflicts between state_store blocks and others * Add tests for state_store override behavior * Add tests for validation that detects when conflicting state-related blocks are used, either in the same file or across separate files * Update error message summaries to explicitly say blocks conflict * Add small changes to assertions in state_store override tests * Update state_store block parsing to expect scoped provider block * Update tests following syntax change * Make config parsing experimental * Remove testModuleFromDirWithExperiment, as testModuleFromDirWithExperiments exists! * Update code comment --------- Co-authored-by: Radek Simko <radeksimko@users.noreply.github.com> Co-authored-by: Radek Simko <radek.simko@gmail.com>	1 year ago
Liam Cervante	777a916435	terraform test: move providers into the graph (#37216 ) * terraform test: move providers into the graph * consistency checks * remove flaky cycle test * address comments * Update internal/moduletest/graph/node_test_run.go Co-authored-by: Samsondeen <40821565+dsa0x@users.noreply.github.com> * go fmt --------- Co-authored-by: Samsondeen <40821565+dsa0x@users.noreply.github.com>	1 year ago
Liam Cervante	53afa2cd2f	terraform test: move variable evaluation into the terraform test graph (#37205 ) * terraform test: move variable evaluation into the terraform test graph * make copyrightfix	1 year ago
Liam Cervante	063b7cf62b	terraform test: check specific dependencies before skipping run blocks (#37204 ) * terraform test: check specific dependencies before skipping run blocks * fix data race * complete comment	1 year ago
Liam Cervante	757efaed9a	terraform test: automatically create edges based on references (#37200 ) * terraform test: automatically create edges based on references * address comments * remove unused function	1 year ago
Liam Cervante	f80762d3d5	terraform test: add variable definitions to test files (#37195 ) * terraform test: add variable definitions to test files Currently, `terraform test` attempts to work out the type of any external variables by delaying evaluation until each run block executes so it can use the definitions within the run blocks's module. This means that the values of variables can technically change between run blocks which isn't ideal. This commit is the first in a chain which will move the evaluation of variables into the terraform test graph. We need to give the users the option of specifying the type for external variables within the file as these variables are now going to be assessed outside of the context of a run block. To do this, we introduce the optional variable blocks that are added by this commit. * update comment	1 year ago
Sarah French	884fc64618	Remove unused test fixture "push-backend-new" (#36842 )	1 year ago
Liam Cervante	7fadbe34de	write-only attributes: internal providers should set write-only attributes to null (#36824 ) * write-only attributes: internal providers should set write-only attributes to null * add changelog * fix copywrite headers	1 year ago
Samsondeen	595cd80666	CLI command for listing state identities (#36705 )	1 year ago
Liam Cervante	aaf617d6d7	Add test case to ensure no regression of already fixed crash (#36583 )	1 year ago
Samsondeen	3f4ed7e019	Ensure that all states are cleaned up (#36616 )	1 year ago
Samsondeen	1bd312f155	Produce detailed diagnostic objects when test run assertions fail (#36522 )	1 year ago
James Bardin	332bce3531	diagnostics should not contain URLs Especially if they are not permanent. Remove the URL and update the wording/quoting to better match existing style.	1 year ago
Samsondeen	018110c012	No state to cleaup when it is empty (#36526 )	1 year ago
Samsondeen	2e91113224	Terraform test: Consolidate test execution procedure (#36459 )	1 year ago
Samsondeen	4161e88391	Continue test execution after an expected failure (#36447 )	1 year ago
Samsondeen	7f29df96a0	Allow terraform init when only test files are present in directory (#36429 )	1 year ago
Radek Simko	70a1fd5e98	Fix diagnostic & avoid variable override via environment (#36435 ) * add changie entry * add test * Fix diagnostic & avoid variable override via environment	1 year ago
Samsondeen	2f0f537e69	Terraform test: Execute eligible test runs in parallel (#36300 )	1 year ago
Sarah French	6b81f7184c	Change JUnit `terraform test` output to include test failure details inside `<failure>` elements, use the error message as the `message` attribute (#36316 ) * Add details to `<failure>` element describing which assertion failed * Remove unused diagnostic string * Set `message` attribute of `<failure>` element to failed assertion's error message * Make `<failure>` contain diagnostic's message, refactor how `message` attribute is set * Ensure that system-err is only added when needed * Update test fixtures * Make diags usage clearer, ensure all test failure diags in "failure" element * Refactor how "skipped" element value is set * Fix failing test Test_TestJUnitXMLFile_Save	1 year ago
Kévin Andrieux	258f91ab0c	feat: add state_key for test run blocks (#36185 ) * feat: add state_alias for test run blocks * fix: move `state_alias` to run block + rename to `state_key` * docs: add state_key changelog entry * docs: add state_key in run block documentation * Revert "docs: add state_key in run block documentation" This reverts commit `ccccf62a76`. * Revert "docs: add state_key changelog entry" This reverts commit `86e2ad8dc3`.	1 year ago
Liam Cervante	408f323efa	test: also allow mock_data and mock_resource blocks to generate data during planning (#36317 )	1 year ago
Sarah French	ab6e4f2299	Refactor JUnit XML output of `terraform test` into a new `junit` package (#36304 ) * Refactor JUnit XML output to use new concept of an Artifact * Move JUnit-related code into new `artifact` package * Refactor Artifact's Save method to return diagnostics, update comments Previously TestJUnitXMLFile implemented the View interface, which cannot return errors. Now it's not a View any more we can simplify things. * Make junitXMLTestReport output deterministic by iterating over a slice instead of a map, add test * Provide sources to junitXMLTestReport, allowing complete error messages in the XML We need to ensure that artifact.NewTestJUnitXMLFile is called once the config Loader is available as a non-nil pointer * Whitespace * Add some test coverage for JUnit XML output for `terraform test` * Refactor how file is saved, add tests * Move XML structs definitions outside of `junitXMLTestReport` * Fix nil pointer bug * Add missing file headers * Refactor comparison of byte slices * Rename package to `junit`, rename structs to match * Add a test showing JUnit output when a test is skipped by the user	1 year ago
Samsondeen	aec7c3cc89	terraform test: allow computed/mocked values override during planning (#36227 )	1 year ago

1 2 3 4

181 Commits (87b46e1263f2947085c93e42b02b76d1e620f2ed)