terraform

Commit Graph

Author	SHA1	Message	Date
Radek Simko	27770ee805	PSS: Migration from a backend to a state store (#38048 ) * backend/inmem: Make it easier to use the backend in tests * cloud: Make the cloud backend testable * command/views: Introduce migration UI messages * command/init: Add 3 tests for migrations from a backend to PSS - TestInit_backend_to_stateStore_singleWorkspace - TestInit_backend_to_stateStore_multipleWorkspaces - TestInit_cloud_to_stateStore * command/init: Implement migration from a backend to PSS * address PR feedback * remove local state after migration	2 weeks ago
Radek Simko	87ba30ef88	plan: Improve diagnostics for uninitialized state_store provider (#38075 ) * plan: Improve diagnostics for uninitialized state_store provider * turn suggestion into panic per PR feedback	3 weeks ago
Sarah French	0ef56b2853	PSS : Test edge cases when the selected workspace doesn't exist during `init` with a `state_store` (#38119 ) * test: Clarify purpose of existing test related to workspace edge-cases during init. * test: Add test case showing Terraform prompt users to select a new workspace if the current one doesn't exist during init (and others exist).	3 weeks ago
Sarah French	02a4ddce1b	fix: Fail an `apply` command if the plan file was generated for a workspace that isn't the selected workspace (#37955 ) * fix: Fail apply command if the plan file was generated for a workspace that isn't the selected workspace. * Add change file * test: Update test helper to include Workspace name in plan representation * fix: Make error message more generic, so is applicable to backend and cloud blocks. * fix: Make error message specific to backend or cloud block * test: Add separate tests for backend/cloud usage * test: Update remaining tests to include a value for Workspace in mocked plans * Apply suggestions from code review Co-authored-by: Radek Simko <radeksimko@users.noreply.github.com> * fix: Panic when a plan file has missing workspace data * test: Update test to match changes in error text --------- Co-authored-by: Radek Simko <radeksimko@users.noreply.github.com>	3 weeks ago
Daniel Schmidt	75445e1ef8	use a string as deprecation origin to help with indirect references We want to be able to give better information if e.g. the entire module is stored in a local and the deprecated value is only later used. Where the diag is emitted we might only see the local and not the true origin of the deprecation A string identifying the source of the deprecation should help	3 weeks ago
Daniel Schmidt	a96df66a9a	display origin of deprecated value in diagnostic	3 weeks ago
Daniel Schmidt	3a36e7732b	refactor: extract write snippet in diagnostic rendering	3 weeks ago
Daniel Schmidt	ea7f1e3043	add extra origin information for deprecation diagnostics	3 weeks ago
Radek Simko	41102accc6	command: Fix copy-paste error in error message (#38116 )	3 weeks ago
Radek Simko	1fde2537c2	command/init: Avoid ignoring errors when setting PSS configs (#38115 )	3 weeks ago
Sarah French	2465e326a2	refactor: Use existing methods to detect if a provider is built in (#38118 )	3 weeks ago
Daniel Schmidt	d813ad14f7	add deprecation marks	3 weeks ago
Daniel Schmidt	4b34066cf2	add error to Pre and PostDiff hook We need this information for stacks to forward the error to the progress stream as stacks expects everything that reports itself starting to e.g. plan to also finish either in an error or with success.	3 weeks ago
Sarah French	8e59c3296a	PSS: Address edge case in `workspace list` when no state files/workspace artefacts exist (#38094 ) * test: Add test demonstrating how the `workspace list` command behaves when no workspaces are reported from a pluggable state store. This scenario isn't possible when using backends. See the code comment for the test for more information. * feat: Make Terraform warn when no workspaces exist, including guidance to users for how to create the workspace. This change is not a breaking change because all backends report that the "default" workspace always exists. This new warning can only be viewed in specific circumstances by users of pluggable state storage.	4 weeks ago
Sarah French	ac3e32b62b	test: Add test asserting deprecation warnings are present when legacy `env` subcommands are used (#38091 )	4 weeks ago
Samsondeen	0b35407d2c	Allow functions in test mock blocks (#37883 )	1 month ago
Sarah French	fa620ad0e9	test: Add and update some `init` command tests, linting and code comment changes (#38056 )	1 month ago
Sarah French	44e5f86375	PSS: Allow use of pluggable state stores with `-backend=false` during `init` commands (#38066 )	1 month ago
Sarah French	827f693e32	test: Update `TestInit_fromModule_dstInSrc` to reflect current behaviour when -from-module references the location of the current directory (#38059 ) * test: Update `TestInit_fromModule_dstInSrc` to use modern `t.Chdir` approach, add context through comments * test: Add context about how `TestInit_fromModule_dstInSrc` has been refactored since it was first created Originally the test's args were []string{".", "foo"} (https://github.com/hashicorp/terraform/pull/547), where the second arg is the directory TF should use as the root directory. Then the test was changed to use []string{"-from-module=.", "foo"} when the -from-module flag was added in `8a7a0a7459`. Finally, the second arg was removed in https://github.com/hashicorp/terraform/pull/27664, and replaced with the use of os.Chdir in this test. * test: Update `TestInit_fromModule_dstInSrc` to define the new behavior of -from-module in this scenario; the current directory is copied once, but not recursively. If you run this test from commit `84f5b863ef` with a debugger we can see that foo is never copied into the current directory foo (./foo/foo is never made). Now, after refactoring ./foo/foo IS made, but it's not copied recursively like in the original issue. * docs: Add comment about emergent behavior when using -chdir with -from-module in this scenario	1 month ago
Radek Simko	1186262c8c	backend/http: Refresh state earlier (align with other backends) (#38033 ) * correct changelog entry for #38027 * backend/http: Refresh state before returning StateMgr This aligns the http backend with the implementation of StateMgr in other remote backends, which in turn enables easier testing of higher level logic that relies on certain backend behaviours. * backend/http: Refactor TestHTTPBackend to allow custom method implementations * add regression test for PR 38027	1 month ago
Mark Hall	7029dfac86	Fix nil pointer dereference in backend state migration (#38028 ) * fix: handle all StateMgr errors during backend migration Fixes a nil pointer dereference panic that occurs during backend migration when StateMgr returns an error other than ErrDefaultWorkspaceNotSupported. The bug occurred because the code only checked for the specific ErrDefaultWorkspaceNotSupported error. When any other error occurred (such as permission errors like storage.objects.get access denied), destinationState remained nil, but the code continued and attempted to call destinationState.RefreshState(), causing a panic. This fix adds an else clause to catch and return all other errors from StateMgr, preventing the nil pointer dereference and providing users with a clear error message instead of a crash. Fixes #24100 * Add changelog entry for backend migration nil pointer fix 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com>	1 month ago
Sarah French	193867f060	test: Add E2E test coverage for inspecting a plan file using the `terraform show` command (#38020 )	2 months ago
Sarah French	fd7f25120b	PSS: Allow pluggable state store configuration to be read from a plan file (#37957 ) * feat: Allow reading state store configuration from a planfile and using it to prepare a Local backend that uses the state store * test: Assert that we can get and use state store configuration from a plan file * test: Add integration test showing that an apply command can use a plan file to configure and use a state store * test: Add E2E test showing pluggable state storage being used with the full init-plan-apply workflow * feat: A plan file will report the state storage provider among its required providers, if PSS is in use. See the code comment added in this commit. This addition does not impact an apply command as the missing provider will be detected before this code is executed. However I'm making this change so that the method is still accurate is being able to return a complete list of providers needed by the plan. * fix: Include error messages when there is a problem parsing provider or state store config when getting a backend from a planfile * feat: Add trace logs to BackendForLocalPlan indicating when the provider is launched and the state store is configured * chore: Small grammar change in error diagnostic * refactor: Remove suggestions when the plan's state store doesn't match the implementations in the provider * test: Add test coverage of what happens when the contents of a plan file using PSS doesn't match the resources available in the project	2 months ago
Sarah French	f88d09544e	PSS: Add tests showing `provider` commands being used with PSS (#37959 ) * test: Add E2E test for using pluggable state storage with the `providers` command Note: I've excluded the `terraform providers locks` and `terraform providers mirror` commands as they don't interact with backends. * test: Add integration test for using pluggable state storage with the `providers` command * refactor: Change ioutil.ReadDir to os.ReadDir * test: Add integration test for using pluggable state storage with the `providers schema` command * feat: Allow state store schema's to be included when schemas are marshalled into JSON output * test: Assert that state stores are present in provider schemas returned from `providers schema`. * test: Update existing tests to accommodate state stores being in provider schema output * test: Update E2E test for `providers` commands to be better scoped to testing use of a state store to access and use state when generating output. This complements TestProvidersSchema that tests that state stores in a provider are reflected in the JSON representations of the schemas that the command returns. * chore: Replace `io/ioutil` with `io` in `providers schema` tests	2 months ago
Sarah French	578766bdc0	PSS: Add tests showing `output` and `show` commands being used with PSS (#37894 ) * test: Add E2E test demonstrating `output` command used with PSS * test: Add E2E test demonstrating `show` command used with PSS * docs: Fix code comment * test: Add integration test for using pluggable state storage with the `output` command * test: Add integration test for using pluggable state storage with the `show` command	2 months ago
Sarah French	940fbfb7ed	test: Add more test coverage of `validate` command's interaction with `backend` blocks (#37977 ) * test: Show how the validate command doesn't detect when an unknown backend type is in the config. * test: Minor fixes to error messages when test fails * test: Add some TODOs in code comments These tests are here to define existing behaviour, not define desired behaviour. I wanted to make that clear!	2 months ago
Sarah French	33d54b9a7c	test: Update module used for testing `init`'s `-from-module` feature, so that the test can be run on darwin_arm64 platforms. (#38015 )	2 months ago
Radek Simko	213d133d86	command/meta: Enable migration from PSS to a backend (#37949 ) * command/meta: Enable migration from PSS to a backend * Address PR feedback * Update internal/command/meta_backend.go Co-authored-by: Sarah French <15078782+SarahFrench@users.noreply.github.com> * meta_backend: Rename stateStore_c_S to stateStore_to_backend --------- Co-authored-by: Sarah French <15078782+SarahFrench@users.noreply.github.com>	2 months ago
Sarah French	f591872699	PSS: Allow pluggable state store configuration to be stored in a plan file (#37956 ) * refactor: Rename Meta's backendState field to backendConfigState This helps with navigating ambiguity around the word backend. The new name should indicate that the value represents a `backend` block, not a more general interpretation of what a backend is. * fix: Only set backendConfigState to synthetic object if it's nil due to a lack of data. Don't change it if pluggable state storage is in use. * feat: Enable recording a state store's details in an Operation, and using that data when creating a plan file. * fix: Include provider config when writing a plan file using pluggable state storage * fix: Having `backendConfigState` be nil may be valid, but it definitely isn't valid for `stateStoreConfigState` to be nil When backendConfigState is nil it means that an implied local backend is in use, i.e. there is no backend block in the config. * test: Add integration test showing that a plan command creates a plan file with the expected state_store configuration data * refactor: Apply suggestion from @radeksimko Co-authored-by: Radek Simko <radeksimko@users.noreply.github.com> * fix: Allow panics to occur if an unexpected implementation of `backend.Backend` is encountered when managing a state store * docs: Add code comment explaining the current situation with passing backend config state to downstream logic. In future this should be simplified, either via refactoring or changes affecting the implied local backend --------- Co-authored-by: Radek Simko <radeksimko@users.noreply.github.com>	2 months ago
Sarah French	a83b7402c2	PSS: Add tests showing `state` subcommands being used with PSS (#37891 ) * test: Add E2E tests for `state list` and `state show` commands * test: Update `mockPluggableStateStorageProvider` to log a warning during tests where the values in `MockStates` aren't compatible with the `ReadStateBytesFn` default function. Make existing test set an appropriate value in `MockStates`. * test: Update `mockPluggableStateStorageProvider` helper to include a resource schema * test: Add command-level test for `state list` showing integration with pluggable state storage code. * test: Add command-level test for `state show` showing integration with pluggable state storage code. * test: Add command-level test for `state pull` showing integration with pluggable state storage code. * test: Add command-level test for `state identities` showing integration with pluggable state storage code. * test: Add command-level test for `state rm` showing integration with pluggable state storage code. * test: Add command-level test for `state mv` showing integration with pluggable state storage code. * test: Add command-level test for `state push` showing integration with pluggable state storage code. * test: Add command-level test for `state replace-provider` showing integration with pluggable state storage code. * test: Change shared test fixture to not be named after a specific command under test. This test fixure is reused across tests that need the config to define a state store but otherwise rely on the mock provider to set up the test scenario. * test: Update test to use shared test fixture * test: Remove redundant test fixture The internal/command/testdata/state-commands-state-store and internal/command/testdata/state-store-unchanged test fixtures are the same. * fix: Re-add logic for setting chunk size in the context of E2E tests using grpcwrap package This was removed, incorrectly, in https://github.com/hashicorp/terraform/pull/37899 * refactor: Let panic happen if there's incompatibility between mock returned from `mockPluggableStateStorageProvider` and the `MockStates` that's been set in the mock * test: Refactor to contain paths in reused variable, remove unnecessary .gitkeep * test: Remove unneeded test code	2 months ago
Sarah French	3eea12a1d8	test: Add test coverage for what validate command checks in resource blocks versus backend blocks. (#37972 ) * test: Fix typo in test name * test: Show how the validate command validates resource blocks using their schema. * test: Show how the validate command doesn't use backends' schemas when validating backend blocks.	3 months ago
Sarah French	0941763d05	test: Add a test that shows a custom workspace being baked-into a planfile (#37917 )	3 months ago
Sarah French	9fb91c2229	PSS: Refactor Plan struct to reference `Backend` and `StateStore` via pointers, update calling code (#37946 ) * refactor: Update Plan struct's Backend field to be a pointer * fix: Add some refactorings or protections against nil pointers when accessing the Backend field on a Plan struct * refactor: Update Plan struct's StateStore field to be a pointer, update a nil check in calling code.	3 months ago
Sarah French	cf047be4e4	PSS: Use interfaces to configure state stores (#37899 ) * refactor: Move chunk size limit constants to new `chunks` package * refactor: Make `NewPluggable` return a `Pluggable` concrete type, instead of an instance of the `backend.Backend` interface. * refactor: Configure state stores through the backend.Backend interface, instead of directly using methods related to RPCs. This requires changing where we call `SetStateStoreChunkSize`. * docs: Add godocs comment to `StateStoreChunkSizeSetter` interface To summarize, we don't really need the `SetStateStoreChunkSize` method, and instead methods like `(GRPCProvider).ConfigureStateStore` in the `plugin6` package can directly inspect the negotiation process that passes through that code and pull out the chunk size. However that means that that code would also need to implement validation. And that's just `(GRPCProvider).ConfigureStateStore`; what about all the test mocks that are used in different types of test? They would all need to be implemented similarly to GRPCProvider to be good, useful mocks, and then a lot of things that fulfil the provider.Interface interface are coupled to each other. Instead, it's easier to have validation in the `grpcClient` struct's methods in the `remote` package, as that code is common to all scenarios. That code needs a method to 'reach into' the provider.Interface value, so we use the `SetStateStoreChunkSize` method. * chore: Make it clearer that the v6 GRPCProvider implements `SetStateStoreChunkSize` * fix: Remove unnecessary assignment of chunk size I'm surprised that removing this doesn't break E2E tests of PSS that use grpcwrap, but I think there's `plugin6` code that runs in that situation, so maybe chunking is handled elsewhere. * chore: Add panic to try detect unexpected cases when setting chunk size. * feat: Add `providers.StateStoreChunkSizeSetter` implementation to provider-simple-v6 * docs: Update code comments for NewPluggable describing its intended use	3 months ago
Kevin Vu	87b3390189	fix: Make init backend messages JSON-compatible (#37920 ) * fix: Make all init backend messages JSON-compatible Fixes #37911 by converting all backend configuration messages to use the View abstraction, which properly formats output as JSON when the -json flag is used. Previously, certain messages were output directly using the legacy Ui abstraction, bypassing JSON formatting. Changes: - Added 11 new message codes to init.go MessageRegistry with both human and JSON formatted values - Replaced 11 direct m.Ui.Output() calls in meta_backend.go with View abstraction calls - Fixed output in backend_C_r_s(), backend_C_r_S_changed(), backend_c_r_S(), and stateStore_c_S() functions All init tests pass successfully. * docs: Add changelog entry for init JSON backend messages fix * cleanup: Remove unused backend output constants The following constants were moved to init.go as message registry entries and are no longer used in meta_backend.go: - outputBackendMigrateChange - outputBackendMigrateLocal - outputStateStoreMigrateLocal - outputBackendReconfigure - successBackendUnset - successBackendSet	3 months ago
Sarah French	4381860dc6	fix: Make `state show` command exit with code 1 if the stored state cannot be marshalled for rendering (#37933 ) Prior to this change, when running a `state show` command with an address for a resource that existed in state but didn't match the current schema of the resource, users would see the "Failed to marshal" error but would also see output saying "The state file is empty. No resources are represented". This is misleading, as the state file isn't empty. Also, the command would exit with code 0 despite the error.	3 months ago
Sarah French	1a547775ff	refactor: Make the JSON view for `init` commands use defaults from the hclog package (#37840 ) * refactor: Change `(v InitJSON) Output` to use logic from the hclog package to take advantage of default settings docs: Add code comments	3 months ago
Steven Masley	3df28c3101	fix: Make `terraform init` JSON log timestamps include millisecond precision (#37818 ) * fix: terraform init log timestamps include millisecond precision * docs: add changelog entry	3 months ago
Radek Simko	d884031b4b	PSS: Implement unsetting of a state store (#37875 ) * Remove BackendOpts.ProviderFactory abstraction layer * PSS: Implement unsetting of a state store * address PR feedback	3 months ago
Sarah French	4420c4dcc0	PSS: Update the `workspace new` subcommand to work with PSS, add E2E and integration tests for using `workspace` commands with PSS. (#37855 ) * feat: Update the `workspace new` subcommand to work with PSS, add E2E testing * refactor: Replace instances of `ioutil` with `os` while looking at the workspace command * docs: Update code comments in `workspace new` command * test: Update E2E test using PSS with workspace commands to assert state files are created by given commands * test: Include `workspace show` in happy path E2E test using PSS * fix: Allow DeleteState RPC to include the id of the state to delete * test: Include `workspace delete` in happy path E2E test using PSS * fix: Avoid assignment to nil map in mock provider during WriteStateBytes * test: Add integration test for workspace commands when using PSS We still need an E2E test for this, to ensure that the GRPC-related packages pass all the expected data between core and the provider. * test: Update test to reflect changes in the test fixture configuration * docs: Fix code comment * test: Change test to build its own Terraform binary with experiments enabled * refactor: Replace use of `newMeta` with reuse of Meta that we re-set the UI value on	3 months ago
Kevin Vu	dc0eba33af	fix: support ephemeral resources in jsonchecks (#37893 ) Fixes #37834	3 months ago
Liam Cervante	cfbfdc5797	providers lock: include test files when loading configuration (#37851 )	3 months ago
Daniel Schmidt	053738fbf0	add additional warning that providers won't be downloaded	3 months ago
Daniel Schmidt	1b662d78e5	init: skip dependencies declared in development override	3 months ago
Sarah French	f2818db795	PSS : Add `fs` and `inmem` state storage implementations to the builtin `simplev6` provider, update `grpcwrap` package, use PSS implementation in E2E test (#37790 ) * feat: Implement `inmem` state store in provider-simple-v6 * feat: Add filesystem state store `fs` in provider-simple-v6, no locking implemented * refactor: Move PSS chunking-related constants into the `pluggable` package, so they can be reused. * feat: Implement PSS-related methods in grpcwrap package * test: Add E2E test checking an init and apply (no plan) workflow is usable with both PSS implementations * fix: Ensure state stores are configured with a suggested chunk size from Core --------- Co-authored-by: Radek Simko <radeksimko@users.noreply.github.com>	3 months ago
Max Qian	078ac7cb21	fix: `TestTest_Parallel` adds delay to ensure order (#37890 ) Signed-off-by: Max Qian <zq2@illinois.edu>	3 months ago
Sarah French	f4d0ec5136	PSS: Fix hashing without provider version (#37852 ) * Fix panic hashing backend config when a builtin or reattached provider is used for PSS * Add test showing a nil version.Version is tolerated by the Hash method, refactor existing test Protect against provider version data missing when using an un-reattached, un-builtin provider. Add test coverage. * Make reporting errors from reattach.IsProviderReattached more consistent * Run `make syncdeps` * Update tests to show when nil provider data is tolerated	4 months ago
Radek Simko	5001deb4a2	PSS: Ensure experimental backend codepath is isolated + gated correctly (#37868 ) * PSS: Ensure experimental backend codepath is isolated + gated correctly * Update init_run.go Co-authored-by: Sarah French <15078782+SarahFrench@users.noreply.github.com> --------- Co-authored-by: Sarah French <15078782+SarahFrench@users.noreply.github.com>	4 months ago
James Bardin	39eb8c7fb4	Merge pull request #37854 from hashicorp/jbardin/provider-eval-scope Allow inconsistent filesystem function results for provider configuration	4 months ago
Sarah French	f5a28cfa8b	PSS: Update how commands access backends, so both `backend` and `state_store` configuration can be used (#37569 ) * Add a generic method for loading an operations backend in non-init commands * Refactor commands to use new prepareBackend method: group 1 * Refactor commands to use new prepareBackend method: group 2, where config parsing needs to be explicitly added * Refactor commands to use new prepareBackend method: group 3, where we can use already parsed config * Additional, more nested, places where logic for accessing backends needs to be refactored * Remove duplicated comment * Add test coverage of `(m Meta) prepareBackend()` Add TODO related to using plans for backend/state_store config in apply commands * Add `testStateStoreMockWithChunkNegotiation` test helper * Add assertions to tests about the backend (remote-state, local, etc) in use within operations backend * Stop prepareBackend taking locks as argument * Code comment in prepareBackend * Replace c.Meta.prepareBackend with c.prepareBackend * Change `c.Meta.loadSingleModule` to `c.loadSingleModule` * Rename (Meta).prepareBackend to (Meta).backend, update godoc comment to make relationship to (Meta).Backend more obvious. * Revert change from config.Module to config.Root.Module * Update `(m Meta) backend` method to parse config itself, and also to adhere to calling code's viewtype instructions Update all tests and calling code following previous commit * Change how an operations backend is obtained by autocomplete code * Update autocomplete to return nil if no workspace names are returned from the backend * Add test coverage for autocompleting workspace names when using a pluggable state store * Fix output command: pass view type data to new `backend` method * Fix in plan command: pass correct view type to `backend` method * Fix `providers schema` command to use correct viewtype when preparing a backend	4 months ago

1 2 3 4 5 ...

999 Commits (27770ee8053da48fbb684fd23d415aae376d2bed)