terraform

Commit Graph

Author	SHA1	Message	Date
Daniel Schmidt	fceb418f62	remove loadStaticConfigWithTests	1 month ago
Daniel Schmidt	ef179c102e	remove loadStaticConfig	1 month ago
Daniel Banck	834c2f4c21	Fix modules command To be able to show version constraint from modules, we now store them during configuration loading.	1 month ago
Daniel Banck	4ef9684188	Rework most of the configuration loading We previously used a loader -> BuildConfig flow to load configuration. This commit changes most (but not all yet) flows to use the new graph-based approach. Instead of simply recursively loading the modules, we now need to take a stepped approach: 1. Load the root module 2. Collect the variables and their values 3. Build the configuration with the graph-based approach Because this approach relies on different parts from different packages, it can't easliy be done within the `configload` package. So, now we do most of in the backend or command.	1 month ago
Daniel Banck	11d819048a	Store module source and version expressions Instead of evaluating and parsing a module source and version on configuration loading, we now simply store the expression. Decoding is now done during the graph-based configuration loading in the module install node.	1 month ago
Radek Simko	cd9257cd53	PSS: Enable state store configuration change (#38153 )	1 month ago
Daniel Schmidt	a2e232856b	read deprecation message for resource deprecations	2 months ago
Sarah French	3db7c751a8	PSS: Fix use of reattached providers in init, enable use of reattached providers during plan-apply workflow (#38182 ) * refactor: Check that the state storage provider is present when beginning to initialise a state store for use in a non-init command. Ensure reattached providers can be used. Previously we passed all required providers into backend options to be used within `stateStoreConfig`, which is invoked via (Meta).Backend. The new approach enforces that the provider is present while assembling the backend options passed to (Meta).Backend from (Meta).backend, which is non-init specific. As this code is defending against users running non-init commands before an init, this place feels appropriate and isn't able to impact the init command. * fix: Reattached PSS providers should return early when checking locks, and an empty locks file is only bad if there isn't a reattached PSS provider * test: Assert that running init with reattached PSS provider is ok, via an E2E test that uses the reattach feature. * fix: Allow builtin or reattached providers to be used for state stores when generating a plan file * test: Expand E2E test to show using a reattached provider can be used for a workflow of init, plan with -out, and apply. * chore: Replace 'io/ioutil' and format code in unmanaged e2e tests	2 months ago
Daniel Banck	8ab5ded5b9	Merge pull request #38146 from hashicorp/dbanck/variable-const-attribute Add `const` attribute to variable blocks	2 months ago
Radek Simko	0fe906fa8c	make copyrightfix	2 months ago
Daniel Banck	f32a248752	config: Remove unused enter package methods	2 months ago
Daniel Banck	4da132e639	Add `const` attribute to variable blocks	2 months ago
Sarah French	b8c2cabee8	init: Fix when error diagnostics are acted on in PSS's experimental version of `init`. Avoid trying to initialise a state store with insufficient config. (#38125 )	2 months ago
Abdurahman Abdelgany	587fc8e66b	Merge branch 'main' into stacks_validate_relative_path_fix	2 months ago
Radek Simko	87ba30ef88	plan: Improve diagnostics for uninitialized state_store provider (#38075 ) * plan: Improve diagnostics for uninitialized state_store provider * turn suggestion into panic per PR feedback	2 months ago
Abdurahman Abdelgany	9a9938d950	Use an actual config with a relative path in unit test	2 months ago
Daniel Schmidt	7eaf6daf60	add deprecation marks to resources based on schema	2 months ago
Daniel Schmidt	958a1ae1e7	variable and output deprecation Variables can be deprecated through the `deprecated` attribute. If set the variable will emit a diagnostic if a values is passed to it. This entails both root level and module variables. Outputs can be deprecated through the `deprecated` attribute as well. If set wherever the value is used a diagnostic will be emitted. Root level outputs can not be deprecated. The only acceptable usage of a deprecated output is another deprecated output (forwarding the deprecation to the module user). If modules not under your control have deprecation warnings you can add a `suppress_deprecations_warnigns` attribute to the module call in question to silence any deeply nested warnings.	2 months ago
Abdurahman Abdelgany	0f25b994d8	cleanup	2 months ago
Abdurahman Abdelgany	4fe0dd4a99	Create source_bundle_parser_test.go	2 months ago
Samsondeen	0b35407d2c	Allow functions in test mock blocks (#37883 )	3 months ago
Abdurahman Abdelgany	8a63ac9f2a	Only convert absolute path to relative, otherwise use relativepath in source bundle parser	3 months ago
Nick Fagerlund	b2aad914f1	stacks: Make path.module and path.root relative, to match documentation (#37982 ) * stacks: Make path.module and path.root relative, to match documentation Previously, we were just handing off the absolute path produced by the sourcebundle struct. But that's no good, because the value doesn't stay consistent between plan and apply when running under tfc-agent. (It uses working directories named after the unique job ID.) It also doesn't match the documentation, which describes these as relative paths. This was preventing people from uploading module-provided files to create aws lambda functions, for example. This commit addresses that by converting the source-bundle provided module path to a relative path (relative to Terraform's working directory). In tfc-agent for stacks runs, that ends up being the directory directly above the sourcebundle directory, and all the paths below that are consistent between plan and apply. * I think this is how the changelog thing works?	4 months ago
James Bardin	6ed7151b2c	Merge pull request #37923 from hashicorp/jbardin/nesting-group-schema-fixes fix handling of NestingGroup in the configschema package	5 months ago
James Bardin	07a7d50f14	fix handling of NestingGroup ImpliedType did not have a clause to handle NestingGroup object types. CoerceValue would incorrectly return a null value for NestingGroup attributes.	5 months ago
Sarah French	1dd8e60f15	chore: Bump golang.org/x/crypto dependency, ignore SA4003 linting errors globally (#37901 ) * chore: Upgrade golang.org/x/crypto - `go get golang.org/x/crypto@v0.44.0` * chore: Upgrade golang.org/x/crypto in submodules - `make syncdeps` * chore: Run `make generate` * chore: Ignore SA4003 linter error globally, as `golang.org/x/tools/cmd/stringer` generates code that breaks this rule when it's used with custom types that have unsigned integers or bytes as the underlying type.	5 months ago
Sarah French	f4d0ec5136	PSS: Fix hashing without provider version (#37852 ) * Fix panic hashing backend config when a builtin or reattached provider is used for PSS * Add test showing a nil version.Version is tolerated by the Hash method, refactor existing test Protect against provider version data missing when using an un-reattached, un-builtin provider. Add test coverage. * Make reporting errors from reattach.IsProviderReattached more consistent * Run `make syncdeps` * Update tests to show when nil provider data is tolerated	5 months ago
Liam Cervante	411f18e6ac	ephemerals: allow root ephemeral outputs in stacks and test (#37813 ) * ephemerals: allow root ephemeral outputs in stacks and test * remember to set new opt for apply stage	6 months ago
Sarah French	88e4c9e3d3	Update inaccurate comment (#37789 )	6 months ago
Sarah French	efb2a1c567	PSS: Let the `init` command recognise when there are no changes in configuration. (#37777 ) * Pull determining of PSS provider's version from current locks into a separate method * Add code for identifying when config and provider version match existing backend state (i.e. no changes) * Update test - locks are now needed before it hits expected error diag return * Add test showing successful init when no config changes are detected. * Update `getStateStorageProviderVersion` to return nil versions for builtin and re-attached providers. This makes comparison easier when determining if config has changed since last init. * Add test coverage for `getStateStorageProviderVersion` * Move testing fixtures around, preparing for different types of changed state_store config changes being tested * Add test showing that changing the state_store config is detected as a change, but handling this scenario isn't implemented yet * Update hashes in test fixture backend state file to be accurate Previously dummy values were fine, but as tests using hashes to identify changes these values need to be accurate! * Update existing test cases so that Terraform uses the same test provider version as described in the backend state file fixture for the test. * Add test showing that changing the PSS provider's config is detected as a change, but handling this scenario isn't implemented yet * Add test showing that swapping to a different state storage implementation in the same provider is detected as a change, but handling this scenario isn't implemented yet * Add test showing that changing the provider used for PSS is detected as a change, but handling this scenario isn't implemented yet * Add test showing that upgrading a provider is detected as a change, but handling this scenario isn't implemented yet * Update test to use v1.2.3 for consistency with other tests Just to avoid any confusion if copy-pasting happens in future. * More corrections to existing test fixtures - unset config should be null, and replace dummy hash values with correct values. * Fix test for using -reconfigure with state_store; the default workspace would already exist in this scenario * Update TestInit_stateStore_configUnchanged to assert that init was a no-op for backend state * Remove unused fixture * Remove test that's replaced by new tests in command/init_test.go * Replace old references to deleted "state-store-changed" test fixture & update test to not expect a value for region attr in provider config * Make test fixture coupling a little more understandable * Refactor detection of no need to migrate into a function * Add TODO about more involved provider version change tests We will allow downgrades to succeed as long as the schema version number is unchanged * Update (configs.StateStore)Hash method to return a single hash that's impacted by: state store config, provider config, state store type, provider source * Update calling code and test helper code to reflect that the nested provider block no longer has its own hash * Remove test; there is now a single hash that SHOULD be affected by the provider block! * Also use provider name, from config, in hash * Update tests to reflect changes in how hashes are made * Remove unused `stateStoreConfigNeedsMigration` function * Remove duplicate isProviderReattached function. * Fixes to affected tests * Allow provider version to impact the state storage hash, update impacted tests and test fixtures * Update tests that now require locks data to be present in test setup * Update comment for accuracy * Fixes to other test fixtures - remove excess hash field, set hash to 0 to indicate they're not set accurately. * Make upgrade test actually use upgrade code path * Add lock files to test fixture directories that represent a project that's had a successful prior init using PSS	6 months ago
Daniel Schmidt	accaa8d027	also check against action references in depends_on during config parsing	6 months ago
Sarah French	6b73f710f8	PSS: Implement initialisation of new working directory (or use of `-reconfigure` flag) while using `state_store` (#37732 ) * Minor fixes in diagnostics This can only be done once modules have been parsed and the required providers data is available. There are multiple places where config is parsed, into either Config or Module structs, so this needs to be implemented in multiple places. * Rename test to make it specific to use of backend block in config * Update initBackend to accept whole initArgs collection * Only process --backend-config data, when setting up a `backend`, if that data isn't empty * Simplify how mock provider factories are made in tests * Update mock provider's default logic to track and manage existing workspaces * Add `ProviderSchema` method to `Pluggable` structs. This allows calling code to access the provider schema when using provider configuration data. * Add function for converting a providerreqs.Version to a hashicorp/go-version Version. This is needed for using locks when creating the backend state file. * Implement initial version of init new working directories using `stateStore_C_s`. Default to creating the default workspace if no workspaces exist. * Update test fixtures to match the hashicorp/test mock provider used in PSS tests * Allow tests to obtain locks that include `testingOverrides` providers. The `testingOverrides` field will only be set in tests, so this should not impact end users. * Add tests showing TF can initialize a working directory for the first time (and do the same when forced by -reconfigure flag). Remove replaced tests. * Add -create-default-workspace flag, to be used to disable creating the default workspace by default when -input=false (i.e for use in CI). Refactor creation of default workspace logic. Add tests. * Allow reattached providers to be used during init for PSS * Rename variable to `backendHash` so relation to `backend` is clearer * Allow `(m Meta) Backend` to return warning diagnostics Protect against nil testingOverrides in providerFactoriesFromLocks * Add test case seeing what happens if default workspace selected, doesn't exist, but other workspaces do exist. The consequences here are due to using `selectWorkspace` in `stateStore_C_s`, matching what's done in `backend_C_r_s`. * Address code consistency check failure on PR * Refactor use of mock in test that's experiencing EOF error... * Remove test that requires test to supply input for user prompt This test passes when run in isolation but fails when run alongside other tests, even when skipping all other tests using `testStdinPipe`. I don't think the value of this test is great enough to start changing how we test stdin input. * Allow -create-default-workspace to be used regardless of whether input is enabled or disabled * Add TF_SKIP_CREATE_DEFAULT_WORKSPACE environment variable * Responses to feedback, including making testStdinPipe helper log details of errors copying data to stdin. Note: We cannot call t.Fatal from a non-test goroutine. * Use Errorf instead * Allow backend state files to not include version data when a builtin or reattached provider is in use. * Add clarifying comment about re-attached providers when finding the matching entry in required_providers * Report that the default workspace was created to the view * Refactor: use error comparison via `errors.Is` to identify when no workspaces exist. * Move handling of TF_ENABLE_PLUGGABLE_STATE_STORAGE into init's ParseInit func. * Validate that PSS-related flags can only be used when experiments are enabled, enforce coupling of PSS-related flags when in use. * Slight rewording of output message about default workspace * Update test to assert new output about default workspace	6 months ago
Samsondeen	719aefdb89	tf query: ready list blocks for beta (#37619 )	7 months ago
Daniel Schmidt	8f05946b6e	remove ooutdated comment	7 months ago
Daniel Schmidt	c5f9b060c7	remove mention of before_destroy, we postponed implementing this event type	7 months ago
Daniel Schmidt	3be1d8a892	enable actions outside of experiments	7 months ago
Kristin Laemmert	df113486a1	actions: remove references to action types, linked resources (#37616 ) * action renaming * actions: remove references to action types * actions: remove references to linked_resources or action types from the plan proto	7 months ago
Samsondeen	1048510fec	tf query: disable depends_on for list blocks (#37618 )	7 months ago
Liam Cervante	551ba2e525	Implement controlling destroy functionality within Terraform Test (#37359 ) * Add ability to parse backend blocks present in a test file's run blocks, validate configuration (#36541) * Add ability to parse backend blocks from a run block * Add validation to avoid multiple backend blocks across run blocks that use the same internal state file. Update tests. * Add validation to avoid multiple backend blocks within a single run block. Update tests. * Remove use of quotes in diagnostic messages * Add validation to avoid backend blocks being used in plan run blocks. Update tests. * Correct local backend blocks in new test fixtures * Add test to show that different test files can use same backend block for same state key. * Add validation to enforce state-storage backend types are used * Remove TODO comment We only need to consider one file at a time when checking if a state_key already has a backend associated with it; parallelism in `terraform test` is scoped down to individual files. * Add validation to assert that the backend block must be in the first apply command for an internal state * Consolidate backend block validation inside a single if statement * Add initial version of validation that ensures a backend isn't re-used within a file * Explicitly set the state_key at the point of parsing the config TODO: What should be done with method (moduletest.Run).GetStateKey? * Update test fixture now that reusing backend configs has been made invalid * Add automated test showing validation of reused configuration blocks * Skip test due to flakiness, minor change to test config naming * Update test so it tolerates non-deterministic order run blocks are evaluated in * Remove unnecessary value assignment to r.StateKey * Replace use of GetStateKey() with accessing the state key that's now set during test config parsing * Fix bug so that run blocks using child modules get the correct state key set at parsing time * Update acceptance test to also cover scenario where root and child module state keys are in use * Update test name * Add newline to regex * Ensure consistent place where repeat backend error is raised from * Write leftover test state(s) to file (#36614) * Add additional validation that the backend used in a run is a supported type (#36648) * Prevent test run when leftover state data is present (#36685) * `test`: Set the initial state for a state files from a backend, allow the run that defines a backend to write state to the backend (#36646) * Allow use of backend block to set initial state for a state key * Note about alternative place to keep 'backend factories' * Allow the run block defining the backend to write state to it * Fix rebase * Change to accessing backend init functions via ContextOpts * Add tests demonstrating how runs containing backend blocks use and update persisted state * Fix test fixture * Address test failure due to trouble opening the state file This problem doesn't happen on MacOS, so I assume is due to the Linux environment of GitHub runners. * Fix issue with paths properly I hope * Fix defect in test assertion * Pivot back to approach introduced in `4afc3d7` * Let failing tests write to persistent state, add test case covering that. I split the acceptance tests into happy/unhappy paths for this, which required some of the helper functions' declarations to be raised up to package-level. * Change how we update internal state files, so that information about the associated backend is never lost * Fix UpdateStateFile * Ensure that the states map set by TestStateTransformer associates a backend with the correct run. * Misc spelling fixes in comments and a log * Replace state get/set functions with existing helpers (#36747) * Replace state get/set functions with existing helpers * Compare to string representation of state * Compare to string representation of state * Terraform Test: Allow skipping cleanup of entire test file or individual run blocks (#36729) * Add validation to enforce skip_cleanup=false cannot be used with backend blocks (#36857) * Integrate use of backend blocks in tests with skip_cleanup feature (#36848) * Fix nil pointer error, update test to not be table-driven * Make using a backend block implicitly set skip_cleanup to true * Stop state artefacts being created when a backend is in use and no cleanup errors have occurred * Return diagnostics so calling code knows if cleanup experienced issues or not * Update tests to show that when cleanup fails a state artefact is created * Add comment about why diag not returned * Bug fix - actually pull in the state from the state manager! * Split and simplify (?) tests to show the backend block can create and/or reuse prior state * Update test to use new fixtures, assert about state artefact. Fix nil pointer * Update test fixture in use, add guardrail for flakiness of forced error during cleanup * Refactor so resource ID set in only one place * Add documentation for using a `backend` block during `test` (#36832) * Add backend as a documented block in a run block * Add documentation about backend blocks in run blocks. * Make the relationship between backends and state keys more clear, other improvements * More test documentation (#36838) * Terraform Test: cleanup command (#36847) * Allow cleanup of states that depend on prior runs outputs (#36902) * terraform test: refactor graph edge calculation * create fake run block nodes during cleanup operation * tidy up TODOs * fix tests * remove old changes * Update internal/moduletest/graph/node_state_cleanup.go Co-authored-by: Samsondeen <40821565+dsa0x@users.noreply.github.com> * Improve diagnostics around skip_cleanup conflicts (#37385) * Improve diagnostics around skip_cleanup conflicts * remove unused dynamic node * terraform test: refactor manifest file for simplicity (#37412) * test: refactor apply and plan functions so no run block is needed * terraform test: write and load state manifest files * Terraform Test: Allow skipping cleanup of entire test file or individual run blocks (#36729) * terraform test: add support for skip_cleanup attr * terraform test: add cleanup command * terraform test: add backend blocks * pause * fix tests * remove commented code * terraform test: make controlling destroy functionality experimental (#37419) * address comments * Update internal/moduletest/graph/node_state_cleanup.go Co-authored-by: Samsondeen <40821565+dsa0x@users.noreply.github.com> --------- Co-authored-by: Samsondeen <40821565+dsa0x@users.noreply.github.com> * add experimental changelog entries --------- Co-authored-by: Sarah French <15078782+SarahFrench@users.noreply.github.com> Co-authored-by: Samsondeen <40821565+dsa0x@users.noreply.github.com> Co-authored-by: Samsondeen Dare <samsondeen.dare@hashicorp.com>	7 months ago
Kristin Laemmert	e315a07c71	remove destroy_ events from list (#37586 ) I expect these will come back, but since we don't have a firm roadmap just yet it's better not to make future promises.	7 months ago
Samsondeen	9b015c6c28	Allow duplicate list names for different types (#37578 )	7 months ago
Daniel Schmidt	003edcf5a1	disallow using the same event in the same action_trigger multiple times	7 months ago
Sarah French	61a7914dd9	Add test helper to create mock provider that contains a state store (#37556 ) * Add a helper for mocking a provider with a state store * Update tests to use the new helper * Update other test fixtures for consistency The related tests pass before and after this change * Update testing helper to return the underlying type, not the interface	7 months ago
Sarah French	5be24bf1d0	Fix experimental gating of PSS feature (#37526 ) * Stop `state_store` being parsed when experiments aren't enabled * Update tests that show the feature is experimentally gated * Refactor to use unparsed hcl.Block for diagnostics when experiments aren't enabled	8 months ago
Liam Cervante	866363ffff	actions: add invoke nodes to the graph (#37521 ) * actions: add invoke graph nodes * fix small bugs * add additional tests * remove whitespace * fix copyright headers * go generate * address comments	8 months ago
James Bardin	2e8eae52f5	add configschema GetBlockByPath This adds a block version of the GetAttributeByPath method for extract nested schema information to correlate with cty values.	8 months ago
Kristin Laemmert	76a9aa288b	fix: move parsing action reference expressions into terraform (#37458 ) * properly parse actions during configuration loading * parse action expression during ActionPlanTransformer transform * fix: properly parse action expressions (not traversals) Previously, we were storing action and linkedResource expressions as Traversals during config loading. This PR replaces those fields with hcl.Expression, moves some validation around to account for the fact we don't have fully evaluated traversals yet (some checks have been removed entirely and will be replaced in a follow up PR, I wanted to get this right first), and finally moves the actual evaluation into the terraform graph build phase. I added a permissive parsing of the ActionExpression during ActionPlanTransformer.transformSingle (to get the config action) and store the action expression inside the nodeActionTriggerPlanExpand.lifecycleActionTrigger so it can be fully evaluated (with access to the resource's repetition data, which is necessary if the action in the resource's lifecycle block uses count.index or for_each.key) during nodeActionTriggerPlanExpand.DynamicExpand	8 months ago
Sarah French	da76dba3dc	PSS: Add alternative, experimental version of `init` command that downloads providers in two stages (#37350 ) * Add forked version of `run` logic that's only used if experiments are enabled * Reorder actions in experimental init - load in full config before configuring the backend. * Add getProvidersFromConfig method, initially as an exact copy of getProviders * Make getProvidersFromConfig not use state to get providers * Add `appendLockedDependencies` method to `Meta` to allow multi-phase saving to the dep locks file * Update experimental init to use new getProvidersFromConfig method * Add new getProvidersFromState method that only accepts state information as input for getting providers. Use in experimental init and append values to existing deps lock file * Update messages sent to view about provider download phases * Change init to save updates to the deps lock file only once * Make Terraform output report that a lock file _will_ be made after providers are determined from config * Remove use of `ProviderDownloadOutcome`s * Move repeated code into separate method * Change provider download approach: determine if locks changed at point of attempting to update the lockfile, keep record of incomplete providers inside init command struct * Refactor `mergeLockedDependencies` and update test * Add comments to provider download methods * Fix issue where incorrect message ouput to view when downloading providers * Update `mergeLockedDependencies` method to be more generic * Update `getProvidersFromState` method to receive in-progress config locks and merge those with any locks on file. This allows re-use of providers downloaded by `getProvidersFromConfig` in the same init command * Fix config for `TestInit_stateStoreBlockIsExperimental` * Improve testing of mergeLockedDependencies; state locks are always missing version constraints * Add tests for 2 phase provider download * Add test case to cover use of the `-upgrade` flag * Change the message shown when a provider is reused during the second provider download step. When downloading providers described only in the state then the provider may already be downloaded from a previous init (i.e. is recorded in the deps lock file) or downloaded during step 1 of provider download. The message here needs to cover both potential scenarios. * Update mergeLockedDependencies comment * fix: completely remove use of upgrade flag in getProvidersFromState * Fix: avoid nil pointer errors by returning an empty collection of locks when there is no state * Fix: use state store data only in diagnostic * Change how we make PSS experimental - avoid relying on a package level variable that causes tests to interact. * Remove full-stop in view message, update tests * Update span names to be unique * Re-add lost early returns * Remove unused view messages * Add comments to new view messages	8 months ago
Daniel Schmidt	632ccc0c28	run make generate	8 months ago
Daniel Schmidt	064a55e356	fix typo	8 months ago

1 2 3 4 5 ...

259 Commits (add-missing-variable-loading)