93 Commits (f8fb61de12dea9ac94de3060a0d3e97ce9b5480b)

Author	SHA1	Message	Date
Sarah French	a28750d8d1	PSS: Remove handling of state migrations including PSS from the `init` command (#38388) ... * feat: Any state migration situations involving PSS are now resolved through init -reconfigure or the new state migrate command. Also, remove methods and structs that aren't used anymore: stateStore_to_backend, backend_to_stateStore, stateStore_changed, removeLocalState, errStateStoreClearSaved * test: Fix test log contents to reference correct test name * fix: Make sure that errStateStoreInitDiag suggests `init` when the store is uninitialised, and suggest `state migrate` + `init -reconfigure` when a migration is needed * fix: Make errBackendToStateStoreInitDiag suggest the correct commands for a state migration. * fix: Standardise on "run <command>" over "use <command>" in error messages * test: Update tests about handling changes in a state store's configuration. We need to re-implement these tests in the context of testing a new `state migrate` command * test: Remove test about handling provider updates in the context of PSS. We need to re-implement this test in the context of testing a new `state migrate` command with an `-upgrade` flag. We also need to implement behaviour in `init` that blocks upgrading the PSS provider. * test: Update test where `init` detects a `state_store` block is removed. * test: Update test where `init` detects a migration from state_store to backend * test: Update test where `init` detects a migration from backend to state_store * test: Update test where `init` detects a migration from the cloud backend to state_store * test: Update E2E test error message assertions to match recent changes	2 months ago
Sarah French	c05eaaf20d	PSS: Refactor how we handle unmanaged providers, add support for using dev_override (#38269) ... * feat: Enable use of developer override providers when using PSS * feat: Store how a provider was supplied to Terraform in the backend state file. This allows triggering a prompt for state migration if there's a change between using a managed provider versus an unmanaged/override provider. * test: Update all PSS-related backend state files in test fixtures to include new, necessary provider_supply_mode data. * fix: Avoid duplicate dev_override warnings * refactor: Change how we determine how a provider is supplied to Terraform. By using supply mode data we prevent multiple parts of the code checking if a given provider is reattached/builtin/etc... * test: Add separate tests showing a built-in or dev override provider being used for the main workflow using PSS * test: Add E2E tests that characterise current behaviour when how a provider is supplied is changed partway through different workflows * fix: Tolerate missing default workspace when using PSS and init handles minor changes in state store configuration. * feat: `init`-related code will prompt a user to migrate state if Terraform detects a swap between managed and unmanaged providers being used for PSS. Internally, what happens is the different version data (null versus a value being present) impacts the hash of the state store. This then triggers logic where TF checks if a migration is needed or if the new hash value should be silently updated in the backend state file. We always prompt for migration. * refactor: Add `NotManagedByTerraform` method to `ProviderSupplyMode` * Apply suggestions from code review Co-authored-by: Radek Simko <radeksimko@users.noreply.github.com> --------- Co-authored-by: Radek Simko <radeksimko@users.noreply.github.com>	2 months ago
Sarah French	2dbb7d9c05	PSS: Remove automatic creation of the default workspace's state file during `init` (#38281) ... * refactor: Stop Terraform creating the default workspace during init when using PSS This is part of reconciling how in the past backends always reported that the default backend existed, even when it didn't. We want state stores to report reality only, so we need to let Terraform handle the discrepancy. Prior to this commit we handled it by making reality match the old lie that the default workspace always exists. After this commit we're just embracing Terraform working with truthful information.	3 months ago
Sarah French	d7baa140f8	fix: Add missing or correct values to error diagnostics raise when initialising a state store from backend state file data (#38275)	3 months ago
Radek Simko	cd9257cd53	PSS: Enable state store configuration change (#38153)	4 months ago
Sarah French	3db7c751a8	PSS: Fix use of reattached providers in init, enable use of reattached providers during plan-apply workflow (#38182) ... * refactor: Check that the state storage provider is present when beginning to initialise a state store for use in a non-init command. Ensure reattached providers can be used. Previously we passed all required providers into backend options to be used within `stateStoreConfig`, which is invoked via (Meta).Backend. The new approach enforces that the provider is present while assembling the backend options passed to (Meta).Backend from (Meta).backend, which is non-init specific. As this code is defending against users running non-init commands before an init, this place feels appropriate and isn't able to impact the init command. * fix: Reattached PSS providers should return early when checking locks, and an empty locks file is only bad if there isn't a reattached PSS provider * test: Assert that running init with reattached PSS provider is ok, via an E2E test that uses the reattach feature. * fix: Allow builtin or reattached providers to be used for state stores when generating a plan file * test: Expand E2E test to show using a reattached provider can be used for a workflow of init, plan with -out, and apply. * chore: Replace 'io/ioutil' and format code in unmanaged e2e tests	4 months ago
Radek Simko	0fe906fa8c	make copyrightfix	4 months ago
Radek Simko	27770ee805	PSS: Migration from a backend to a state store (#38048) ... * backend/inmem: Make it easier to use the backend in tests * cloud: Make the cloud backend testable * command/views: Introduce migration UI messages * command/init: Add 3 tests for migrations from a backend to PSS - TestInit_backend_to_stateStore_singleWorkspace - TestInit_backend_to_stateStore_multipleWorkspaces - TestInit_cloud_to_stateStore * command/init: Implement migration from a backend to PSS * address PR feedback * remove local state after migration	4 months ago
Radek Simko	87ba30ef88	plan: Improve diagnostics for uninitialized state_store provider (#38075) ... * plan: Improve diagnostics for uninitialized state_store provider * turn suggestion into panic per PR feedback	5 months ago
Sarah French	02a4ddce1b	fix: Fail an `apply` command if the plan file was generated for a workspace that isn't the selected workspace (#37955) ... * fix: Fail apply command if the plan file was generated for a workspace that isn't the selected workspace. * Add change file * test: Update test helper to include Workspace name in plan representation * fix: Make error message more generic, so is applicable to backend and cloud blocks. * fix: Make error message specific to backend or cloud block * test: Add separate tests for backend/cloud usage * test: Update remaining tests to include a value for Workspace in mocked plans * Apply suggestions from code review Co-authored-by: Radek Simko <radeksimko@users.noreply.github.com> * fix: Panic when a plan file has missing workspace data * test: Update test to match changes in error text --------- Co-authored-by: Radek Simko <radeksimko@users.noreply.github.com>	5 months ago
Radek Simko	1fde2537c2	command/init: Avoid ignoring errors when setting PSS configs (#38115)	5 months ago
Sarah French	2465e326a2	refactor: Use existing methods to detect if a provider is built in (#38118)	5 months ago
Sarah French	44e5f86375	PSS: Allow use of pluggable state stores with `-backend=false` during `init` commands (#38066)	5 months ago
Sarah French	fd7f25120b	PSS: Allow pluggable state store configuration to be read from a plan file (#37957) ... * feat: Allow reading state store configuration from a planfile and using it to prepare a Local backend that uses the state store * test: Assert that we can get and use state store configuration from a plan file * test: Add integration test showing that an apply command can use a plan file to configure and use a state store * test: Add E2E test showing pluggable state storage being used with the full init-plan-apply workflow * feat: A plan file will report the state storage provider among its required providers, if PSS is in use. See the code comment added in this commit. This addition does not impact an apply command as the missing provider will be detected before this code is executed. However I'm making this change so that the method is still accurate is being able to return a complete list of providers needed by the plan. * fix: Include error messages when there is a problem parsing provider or state store config when getting a backend from a planfile * feat: Add trace logs to BackendForLocalPlan indicating when the provider is launched and the state store is configured * chore: Small grammar change in error diagnostic * refactor: Remove suggestions when the plan's state store doesn't match the implementations in the provider * test: Add test coverage of what happens when the contents of a plan file using PSS doesn't match the resources available in the project	6 months ago
Radek Simko	213d133d86	command/meta: Enable migration from PSS to a backend (#37949) ... * command/meta: Enable migration from PSS to a backend * Address PR feedback * Update internal/command/meta_backend.go Co-authored-by: Sarah French <15078782+SarahFrench@users.noreply.github.com> * meta_backend: Rename stateStore_c_S to stateStore_to_backend --------- Co-authored-by: Sarah French <15078782+SarahFrench@users.noreply.github.com>	6 months ago
Sarah French	f591872699	PSS: Allow pluggable state store configuration to be stored in a plan file (#37956) ... * refactor: Rename Meta's backendState field to backendConfigState This helps with navigating ambiguity around the word backend. The new name should indicate that the value represents a `backend` block, not a more general interpretation of what a backend is. * fix: Only set backendConfigState to synthetic object if it's nil due to a lack of data. Don't change it if pluggable state storage is in use. * feat: Enable recording a state store's details in an Operation, and using that data when creating a plan file. * fix: Include provider config when writing a plan file using pluggable state storage * fix: Having `backendConfigState` be nil may be valid, but it definitely isn't valid for `stateStoreConfigState` to be nil When backendConfigState is nil it means that an implied local backend is in use, i.e. there is no backend block in the config. * test: Add integration test showing that a plan command creates a plan file with the expected state_store configuration data * refactor: Apply suggestion from @radeksimko Co-authored-by: Radek Simko <radeksimko@users.noreply.github.com> * fix: Allow panics to occur if an unexpected implementation of `backend.Backend` is encountered when managing a state store * docs: Add code comment explaining the current situation with passing backend config state to downstream logic. In future this should be simplified, either via refactoring or changes affecting the implied local backend --------- Co-authored-by: Radek Simko <radeksimko@users.noreply.github.com>	6 months ago
Sarah French	cf047be4e4	PSS: Use interfaces to configure state stores (#37899) ... * refactor: Move chunk size limit constants to new `chunks` package * refactor: Make `NewPluggable` return a `Pluggable` concrete type, instead of an instance of the `backend.Backend` interface. * refactor: Configure state stores through the backend.Backend interface, instead of directly using methods related to RPCs. This requires changing where we call `SetStateStoreChunkSize`. * docs: Add godocs comment to `StateStoreChunkSizeSetter` interface To summarize, we don't really need the `SetStateStoreChunkSize` method, and instead methods like `(*GRPCProvider).ConfigureStateStore` in the `plugin6` package can directly inspect the negotiation process that passes through that code and pull out the chunk size. However that means that that code would also need to implement validation. And that's just `(*GRPCProvider).ConfigureStateStore`; what about all the test mocks that are used in different types of test? They would all need to be implemented similarly to GRPCProvider to be good, useful mocks, and then a lot of things that fulfil the provider.Interface interface are coupled to each other. Instead, it's easier to have validation in the `grpcClient` struct's methods in the `remote` package, as that code is common to all scenarios. That code needs a method to 'reach into' the provider.Interface value, so we use the `SetStateStoreChunkSize` method. * chore: Make it clearer that the v6 GRPCProvider implements `SetStateStoreChunkSize` * fix: Remove unnecessary assignment of chunk size I'm surprised that removing this doesn't break E2E tests of PSS that use grpcwrap, but I think there's `plugin6` code that runs in that situation, so maybe chunking is handled elsewhere. * chore: Add panic to try detect unexpected cases when setting chunk size. * feat: Add `providers.StateStoreChunkSizeSetter` implementation to provider-simple-v6 * docs: Update code comments for NewPluggable describing its intended use	7 months ago
Kevin Vu	87b3390189	fix: Make init backend messages JSON-compatible (#37920) ... * fix: Make all init backend messages JSON-compatible Fixes #37911 by converting all backend configuration messages to use the View abstraction, which properly formats output as JSON when the -json flag is used. Previously, certain messages were output directly using the legacy Ui abstraction, bypassing JSON formatting. Changes: - Added 11 new message codes to init.go MessageRegistry with both human and JSON formatted values - Replaced 11 direct m.Ui.Output() calls in meta_backend.go with View abstraction calls - Fixed output in backend_C_r_s(), backend_C_r_S_changed(), backend_c_r_S(), and stateStore_c_S() functions All init tests pass successfully. * docs: Add changelog entry for init JSON backend messages fix * cleanup: Remove unused backend output constants The following constants were moved to init.go as message registry entries and are no longer used in meta_backend.go: - outputBackendMigrateChange - outputBackendMigrateLocal - outputStateStoreMigrateLocal - outputBackendReconfigure - successBackendUnset - successBackendSet	7 months ago
Radek Simko	d884031b4b	PSS: Implement unsetting of a state store (#37875) ... * Remove BackendOpts.ProviderFactory abstraction layer * PSS: Implement unsetting of a state store * address PR feedback	7 months ago
Sarah French	f2818db795	PSS : Add `fs` and `inmem` state storage implementations to the builtin `simplev6` provider, update `grpcwrap` package, use PSS implementation in E2E test (#37790) ... * feat: Implement `inmem` state store in provider-simple-v6 * feat: Add filesystem state store `fs` in provider-simple-v6, no locking implemented * refactor: Move PSS chunking-related constants into the `pluggable` package, so they can be reused. * feat: Implement PSS-related methods in grpcwrap package * test: Add E2E test checking an init and apply (no plan) workflow is usable with both PSS implementations * fix: Ensure state stores are configured with a suggested chunk size from Core --------- Co-authored-by: Radek Simko <radeksimko@users.noreply.github.com>	7 months ago
Sarah French	f4d0ec5136	PSS: Fix hashing without provider version (#37852) ... * Fix panic hashing backend config when a builtin or reattached provider is used for PSS * Add test showing a nil *version.Version is tolerated by the Hash method, refactor existing test * Protect against provider version data missing when using an un-reattached, un-builtin provider. Add test coverage. * Make reporting errors from reattach.IsProviderReattached more consistent * Run `make syncdeps` * Update tests to show when nil provider data is tolerated	8 months ago
Sarah French	f5a28cfa8b	PSS: Update how commands access backends, so both `backend` and `state_store` configuration can be used (#37569) ... * Add a generic method for loading an operations backend in non-init commands * Refactor commands to use new prepareBackend method: group 1 * Refactor commands to use new prepareBackend method: group 2, where config parsing needs to be explicitly added * Refactor commands to use new prepareBackend method: group 3, where we can use already parsed config * Additional, more nested, places where logic for accessing backends needs to be refactored * Remove duplicated comment * Add test coverage of `(m *Meta) prepareBackend()` * Add TODO related to using plans for backend/state_store config in apply commands * Add `testStateStoreMockWithChunkNegotiation` test helper * Add assertions to tests about the backend (remote-state, local, etc) in use within operations backend * Stop prepareBackend taking locks as argument * Code comment in prepareBackend * Replace c.Meta.prepareBackend with c.prepareBackend * Change `c.Meta.loadSingleModule` to `c.loadSingleModule` * Rename (Meta).prepareBackend to (Meta).backend, update godoc comment to make relationship to (Meta).Backend more obvious. * Revert change from config.Module to config.Root.Module * Update `(m *Meta) backend` method to parse config itself, and also to adhere to calling code's viewtype instructions * Update all tests and calling code following previous commit * Change how an operations backend is obtained by autocomplete code * Update autocomplete to return nil if no workspace names are returned from the backend * Add test coverage for autocompleting workspace names when using a pluggable state store * Fix output command: pass view type data to new `backend` method * Fix in plan command: pass correct view type to `backend` method * Fix `providers schema` command to use correct viewtype when preparing a backend	8 months ago
Sarah French	efb2a1c567	PSS: Let the `init` command recognise when there are no changes in configuration. (#37777) ... * Pull determining of PSS provider's version from current locks into a separate method * Add code for identifying when config and provider version match existing backend state (i.e. no changes) * Update test - locks are now needed before it hits expected error diag return * Add test showing successful init when no config changes are detected. * Update `getStateStorageProviderVersion` to return nil versions for builtin and re-attached providers. This makes comparison easier when determining if config has changed since last init. * Add test coverage for `getStateStorageProviderVersion` * Move testing fixtures around, preparing for different types of changed state_store config changes being tested * Add test showing that changing the state_store config is detected as a change, but handling this scenario isn't implemented yet * Update hashes in test fixture backend state file to be accurate Previously dummy values were fine, but as tests using hashes to identify changes these values need to be accurate! * Update existing test cases so that Terraform uses the same test provider version as described in the backend state file fixture for the test. * Add test showing that changing the PSS provider's config is detected as a change, but handling this scenario isn't implemented yet * Add test showing that swapping to a different state storage implementation in the same provider is detected as a change, but handling this scenario isn't implemented yet * Add test showing that changing the provider used for PSS is detected as a change, but handling this scenario isn't implemented yet * Add test showing that upgrading a provider is detected as a change, but handling this scenario isn't implemented yet * Update test to use v1.2.3 for consistency with other tests Just to avoid any confusion if copy-pasting happens in future. * More corrections to existing test fixtures - unset config should be null, and replace dummy hash values with correct values. * Fix test for using -reconfigure with state_store; the default workspace would already exist in this scenario * Update TestInit_stateStore_configUnchanged to assert that init was a no-op for backend state * Remove unused fixture * Remove test that's replaced by new tests in command/init_test.go * Replace old references to deleted "state-store-changed" test fixture & update test to not expect a value for region attr in provider config * Make test fixture coupling a little more understandable * Refactor detection of no need to migrate into a function * Add TODO about more involved provider version change tests We will allow downgrades to succeed as long as the schema version number is unchanged * Update (configs.StateStore)Hash method to return a single hash that's impacted by: state store config, provider config, state store type, provider source * Update calling code and test helper code to reflect that the nested provider block no longer has its own hash * Remove test; there is now a single hash that SHOULD be affected by the provider block! * Also use provider name, from config, in hash * Update tests to reflect changes in how hashes are made * Remove unused `stateStoreConfigNeedsMigration` function * Remove duplicate isProviderReattached function. * Fixes to affected tests * Allow provider version to impact the state storage hash, update impacted tests and test fixtures * Update tests that now require locks data to be present in test setup * Update comment for accuracy * Fixes to other test fixtures - remove excess hash field, set hash to 0 to indicate they're not set accurately. * Make upgrade test actually use upgrade code path * Add lock files to test fixture directories that represent a project that's had a successful prior init using PSS	8 months ago
Sarah French	6b73f710f8	PSS: Implement initialisation of new working directory (or use of `-reconfigure` flag) while using `state_store` (#37732) ... * Minor fixes in diagnostics This can only be done once modules have been parsed and the required providers data is available. There are multiple places where config is parsed, into either Config or Module structs, so this needs to be implemented in multiple places. * Rename test to make it specific to use of backend block in config * Update initBackend to accept whole initArgs collection * Only process --backend-config data, when setting up a `backend`, if that data isn't empty * Simplify how mock provider factories are made in tests * Update mock provider's default logic to track and manage existing workspaces * Add `ProviderSchema` method to `Pluggable` structs. This allows calling code to access the provider schema when using provider configuration data. * Add function for converting a providerreqs.Version to a hashicorp/go-version Version. This is needed for using locks when creating the backend state file. * Implement initial version of init new working directories using `stateStore_C_s`. Default to creating the default workspace if no workspaces exist. * Update test fixtures to match the hashicorp/test mock provider used in PSS tests * Allow tests to obtain locks that include `testingOverrides` providers. The `testingOverrides` field will only be set in tests, so this should not impact end users. * Add tests showing TF can initialize a working directory for the first time (and do the same when forced by -reconfigure flag). Remove replaced tests. * Add -create-default-workspace flag, to be used to disable creating the default workspace by default when -input=false (i.e for use in CI). Refactor creation of default workspace logic. Add tests. * Allow reattached providers to be used during init for PSS * Rename variable to `backendHash` so relation to `backend` is clearer * Allow `(m *Meta) Backend` to return warning diagnostics * Protect against nil testingOverrides in providerFactoriesFromLocks * Add test case seeing what happens if default workspace selected, doesn't exist, but other workspaces do exist. The consequences here are due to using `selectWorkspace` in `stateStore_C_s`, matching what's done in `backend_C_r_s`. * Address code consistency check failure on PR * Refactor use of mock in test that's experiencing EOF error... * Remove test that requires test to supply input for user prompt This test passes when run in isolation but fails when run alongside other tests, even when skipping all other tests using `testStdinPipe`. I don't think the value of this test is great enough to start changing how we test stdin input. * Allow -create-default-workspace to be used regardless of whether input is enabled or disabled * Add TF_SKIP_CREATE_DEFAULT_WORKSPACE environment variable * Responses to feedback, including making testStdinPipe helper log details of errors copying data to stdin. Note: We cannot call t.Fatal from a non-test goroutine. * Use Errorf instead * Allow backend state files to not include version data when a builtin or reattached provider is in use. * Add clarifying comment about re-attached providers when finding the matching entry in required_providers * Report that the default workspace was created to the view * Refactor: use error comparison via `errors.Is` to identify when no workspaces exist. * Move handling of TF_ENABLE_PLUGGABLE_STATE_STORAGE into init's ParseInit func. * Validate that PSS-related flags can only be used when experiments are enabled, enforce coupling of PSS-related flags when in use. * Slight rewording of output message about default workspace * Update test to assert new output about default workspace	8 months ago
Sarah French	d11bdf6832	Refactor backend error strings as custom errors or pre-made diagnostics (#37762) ... * Replace `errBackendLocalRead` constant with custom error of same name * Replace `errBackendNoExistingWorkspaces` constant with custom error of same name * Replace `errBackendMigrateLocalDelete` constant with custom error of same name * Replace `errBackendNewUnknown` constant with custom error of same name * Replace `errBackendSavedUnknown` constant with custom error of same name * Replace `errBackendClearSaved` constant with custom error of same name * Replace `errBackendInit` constant with custom error of same name * Replace `errBackendInitCloud` constant with custom error of same name * Replace `errBackendWriteSaved` constant with custom error of same name * Move `migrateOrReconfigDiag` next to custom errors * Replace `errBackendInit` custom error with `func errBackendInitDiag` for creating diagnostics directly * Replace `errBackendInitCloud` custom error with `func errBackendInitCloudDiag` for creating diagnostics directly * Replace `errBackendWriteSaved` custom error with `func errBackendWriteSavedDiag` for creating diagnostics directly * Add new usage of errBackendWriteSavedDiag; this is another place where failing to write to backend state files happens * Add godoc comments * Ensure errors aren't repeated in terminal output, and ensure warnings are not lost.	8 months ago
Sarah French	d573104221	PSS: Add `stateStoreInitFromConfig` method to `Meta` (#37723) ... * Add SetStateStoreChunkSize to the mock provider for tests * Implement configurable state chunk size * Add chunk size negotiation to `savedStateStore`, update happy path test to assert it's set * Update `savedStateStore` to return diagnostic if a nil factory is passed in, add unhappy path tests * Add test coverage for stateStoreConfig, including use of config overrides with state stores * Add godoc comment for backendInitFromConfig * Avoid nil error in stateStoreConfig when there's no provider factory * Add stateStoreInitFromConfig method and test coverage * Refactor stateStoreInitFromConfig to accept factory directly, instead of accepting all BackendOpts This is to avoid confusion, e.g. unnecessary duplication of config being passed in. * Implement configurable state chunk size * Remove TODO which has since been addressed * Update happy path test for `stateStoreInitFromConfig` to check that chunk size is negotiated * Update chunk size negotiation code in `stateStoreInitFromConfig` to error if the provider doesn't return a chunk size > 0 * Fix test error message --------- Co-authored-by: Radek Simko <radek.simko@gmail.com>	9 months ago
Sarah French	17445f6c5f	PSS: Add chunk size negotiation to `savedStateStore` method, update mock provider for tests. (#37726) ... * Add SetStateStoreChunkSize to the mock provider for tests * Implement configurable state chunk size * Add chunk size negotiation to `savedStateStore`, update happy path test to assert it's set * Update `savedStateStore` to return diagnostic if a nil factory is passed in, add unhappy path tests * Fix test error message * Apply suggestions from code review Co-authored-by: Radek Simko <radeksimko@users.noreply.github.com> * Fix rename --------- Co-authored-by: Radek Simko <radek.simko@gmail.com> Co-authored-by: Radek Simko <radeksimko@users.noreply.github.com>	9 months ago
Sarah French	312f296c2d	PSS: Add `savedStateStore` method to `Meta` (#37558) ... * Add test coverage for Meta's `savedBackend` method * Add new Meta `savedStateStore` method and test coverage * Streamline test - remove unneeded assertions and update comments * Remove marks from config before configuring the provider * Remove marks from config before configuring the state store * Add test case for savedStateStore to assert marks aren't passed * Fix call to ConfigureStateStore * Show that tests pass despite not trying to remove marks * Allow Config methods to add marks when reading pluggable state store config from the backend state file * This code is now necessary to let the tests pass * Stop adding marks to PSS-related config when it's parsed from the backend state file * Stop removing marks that aren't there * Remove unnecessary test related to marks	9 months ago
Sarah French	b9522561c8	PSS: Add reusable method for obtaining the provider factory needed for accessing a state store. (#37665) ... * Add method to allow accessing factories from locks that are in memory * Create new getStateStoreProviderFactory method for accessing a factory from config * Update getStateStoreProviderFactory to use in memory locks instead of reading them from the deps lock file. Update calling code to accommodate this. * Add tests for getStateStoreProviderFactory, improve errors returned from method * Update test following schema change in simple providers * Move test case into e2etest package, so we protect against environments where building binaries isn't possible * Fix issues with running test in e2etest package * Update code comments Co-authored-by: Radek Simko <radeksimko@users.noreply.github.com> --------- Co-authored-by: Radek Simko <radeksimko@users.noreply.github.com>	9 months ago
Sarah French	6847e763de	Update calls to StateMgr method to use diagnostics more conventionally (#37597) ... * Update call to StateMgr method to use diagnostics more explicitly * Update calling code in remote backend tests * Update calling code in non-test code to handle diags conventionally. * Fix test failure messages from using wrong variable * Additional refactoring	9 months ago
Sarah French	62b3ba590a	Update `backend.Backend` interface to return diagnostics from `Workspaces` and `DeleteWorkspace` methods (#37430) ... * Update Backend interface to use diagnostics for `DeleteWorkspace` and `Workspaces` methods * Update testing helpers in backend package * Update pluggable package to match interface changes * Update builtin terraform provider package to match interface changes * Update code in Meta-related files in command package; update error check logic * Update workspace commands; update error check logic * Update Local backend package to match interface changes * Update Cloud backend to match interface changes * Update Remote backend to match interface changes * Update cos remote-state backend to match interface changes * Update inmem remote-state backend to match interface changes * Update pg remote-state backend to match interface changes * Update http remote-state backend to match interface changes * Update consul remote-state backend to match interface changes * Update oss remote-state backend to match interface changes * Update kubernetes remote-state backend to match interface changes * Update gcs remote-state backend to match interface changes * Update s3 remote-state backend to match interface changes * Update oci remote-state backend to match interface changes * Update azure remote-state backend to match interface changes * Allow warnings from Workspaces and DeleteWorkspace methods to be logged or returned to calling code * Fix defect in inmem backend test * Change how warnings from workspace-related methods are output to view/ui	10 months ago
Sarah French	d1e412fcf0	PSS: Add initial (incomplete) version of code changes to the `init` command for using pluggable state storage (#37321) ... * Store the FQN of the provider used in PSS in representations of the parsed config. This can only be done once modules have been parsed and the required providers data is available. There are multiple places where config is parsed, into either Config or Module structs, so this needs to be implemented in multiple places. * Update affected tests, improve error diagnostic * Begin enabling method to return a backend.Backend made using state_store config. State store config can now be received via BackendOpts and there is rough validation of whether the config makes sense (does the provider offer a store with the given name?). * Update code's logic to include possibility of a state store being in use At this point there are no cases that actually handle the state store scenarios though! * Add empty cases for handle all broad init scenarios involving PSS * Update default case's error to report state store variables * Improve how we resolve the builtin terraform provider's address * Add test that hits the code path for adding a state store to a new (or implied local) project * Add test for use of `-reconfigure` flag; show that it hits the code path for adding a state store for the first time * Add test that hits the code path for removing use of a state store, migrating to (implied) local backend * Add test that hits the code path for changing a state store's configuration * Update existing test names to be backend-specific * Add tests that hits the code path for migrating between PSS and backends * Consolidate PSS-related tests at end of the file * Fix log text * Add test showing that using variables is disallowed with state_store and nested provider blocks * Update test name * Fix test cases * Add TODOs so we remember to remove experiments from tests * Update state store-related tests to use t.Cleanup * Remove use of `testChdir`	11 months ago
Sarah French	145e152a57	Clarify usage of BackendOpts, rename field in struct to enable usage with PSS in future (#37085) ... * Update comments to distinguish between operations and state-storage backends more clearly * Rename `BackendOpts` field `Config` to more specific `BackendConfig`	12 months ago
Sarah French	677e5ea276	Add test coverage for Meta's determineInitReason method, fix error in diagnostic (#37303) ... * Add test coverage for Meta's determineInitReason method, fix error in diagnostic when migrating away from HCP Terraform * Add godoc comment to determineInitReason method	12 months ago
Sarah French	d02a1ed1b7	PSS: Make the state storage provider's config stateful (backend state, planfiles) (#37286)	12 months ago
Sarah French	2349481b23	Update data stored in plan files to enable using PSS with saved plans (#37246) ... * Implement `ForPlan` method on `StateStoreConfigState`, add `Planner` interface * Rename `ForPlan` method to `Plan` * Allow plan files to contain information about state stores * Add code needed for representing a state store in the Plan struct, which is used when handling plan files * Add ability to read/write either a backend or state store's data in a plan file. Add some test coverage. * Update plan's `ProviderAddrs` method to include the provider used for PSS, if present * Split interfaces * Apply feedback from code review * Refactor `SetVersion` to use appropriate constructor * Split `ProviderAddrs` method test into two * Fix method call after rename * Fix test * Remove change to `(p *Plan) ProviderAddrs()` We may re-add this when we implement PSS for use during apply commands with plan files * Remove changes to test, now that the plan doesn't report the provider used for PSS anymore.	12 months ago
Sarah French	d3b02a8469	Refactor code to use backend config state's `SetConfig` method, protect against nil receiver (#37254) ... * Add protection against nil receivers to SetConfig * Refactor code to use SetConfig method instead of setting ConfigRaw directly	1 year ago
Sarah French	2559f0a3db	Update backend state file so it can describe PSS state (#37179) ... * Split code for backend state file vs backend state * Rename BackendState to BackendConfigState * Spelling error * Add `StateStorageConfigState` struct as new implementation of new `ConfigState[T any]` interface. * Split tests for backend state file vs backend config state structs * Rename StateStorageConfigState to StateStoreConfigState * Clarify test name, add comments * Add tests for StateStoreConfigState methods * Add test showing state_store in JSON is parsed correctly * Add detection of malformed backend state files that contain both backend and state_store fields * Add validation that stops a backend state file being written if it will contain state for both backend and state_store blocks * Rename `state_storage` to `state_store` * Rename `state_storage` to `state_store` in filenames * Move`ConfigState` to its own file * Fix test name, remove whitespace * Update `StateStoreConfigState` comment using review suggestion * Update error message to no longer allude to the environment TF is being run in * Update the state_store state to use `version.Version` and an adapted version of `tfaddr.Provider` for marshalling version and source data * Update test helper so it doesn't accidentally supply validation in tests * Add protection against saving an empty backend state file * Remove direct testing of (s *Source) MarshalText() and UnmarshalText() methods * Add Validate method to StateStoreConfigState, use in backend state encoding logic * Refactor to use new features in registry dependency	1 year ago
Sarah French	2d6eda2877	Update comments about backend interfaces, enhanced vs operations, and fix copy-paste inaccuracies (#36716) ... * Fix copy-pasta * Update some comments clarifying backend-related interfaces and "enhanced" versus "operations" * Fix more comments that refer to types and interfaces that have moved into the backendrun package	1 year ago
Bruno Schaatsbergen	a5d8673a41	Revert "Merge pull request #35929 from bschaatsbergen/b/fix-retry-messages-in-cloud-backend-json" ... This reverts commit f8cb25a246, reversing changes made to a25831d6ef.	2 years ago
Bruno Schaatsbergen	f1428b3388	backend/cloud: expose View to enable reuse in a (cloud) backend ... Signed-off-by: Bruno Schaatsbergen <git@bschaatsbergen.com>	2 years ago
Bruno Schaatsbergen	d8bfe7a80b	cloud: Initialize a new cloud view in the backend to render messages based on view type. ... Signed-off-by: Bruno Schaatsbergen <git@bschaatsbergen.com>	2 years ago
Bruno Schaatsbergen	f9797595e3	chore: initialize new view in meta backend ... Signed-off-by: Bruno Schaatsbergen <git@bschaatsbergen.com>	2 years ago
Brandon Croft	9d0133a541	Fix non-string elements in tags schema	2 years ago
Brandon Croft	e87335a972	cloud: Allows object tag schema for selecting kv tagged workspaces ... When key-value tags are enabled and used in the workspace, users may define the tags attribute as a map of strings in the cloud block in order to more precicely match workspaces using those tags.	2 years ago
James Bardin	2cbb6bf92f	udpate code which was failing staticckeck ... Some updates to staticcheck were catching more errors. Most of these were format functions without a format string, but rather than adding more exceptions I just fixed them all. This did also catch some incorrectly wrapped errors.	2 years ago
Sebastian Rivera	015f795ff0	Rebrand Terraform Cloud to HCP Terraform	2 years ago
Martin Atkins	88cb4aeea3	backendrun: Separate the types/etc for backends that support operations ... We previously had all of the types and helpers for all kinds of backends together in package backend. That kept things relatively simple, but it also meant that the majority of backends that only deal with remote state storage ended up still indirectly depending on the entire Terraform modules runtime, configuration loader, etc, etc, which brings into scope a bunch of external dependencies that the remote state backends don't really need. Since backends that support operations are a rare exception, we'll move the types and helpers for those into a separate package "backendrun", and then the main package backend can have a much more modest set of types and, more importantly, a modest set of dependencies on other packages in this codebase. This is part of an ongoing effort to reduce the exposure of Terraform Core and CLI code to the remote backends and vice-versa, so that in the long run we can more often treat them as separate for dependency maintenance purposes.	2 years ago
Martin Atkins	06f51dbe04	command/workdir: Model the "backend state" file format ... Historically (before there were "backends") Terraform had a single state file format used both for real state snapshots and for tracking where remote state was stored. Terraform v0.12 caused these two to be split because we adopted state snapshot version 4 for real snapshots but retained a subset of version 3 for tracking the remote backend configuration in the local working directory. Unfortunately we previously kept that working by retaining a snapshot of the entire Terraform v0.11 "terraform" package as legacy/terraform, which happened to still be around because we also needed to retain a copy of the entire legacy SDK to keep the remote state backends working. This now hoists just the tiny slice of legacy terraform package functionality needed to implement the "backend state" file format into package workdir. This package is a good home for it because it's part of the working directory state. Ideally it would be accessed through methods of the workdir.Dir type, but that's too disruptive a refactor to combine into this and so that'll need to wait for another day; for now we'll keep the existing callers doing their access through our "clistate" package that is itself a forked snapshot of what statemgr.Filesystem used to be in Terraform v0.11. This removes all but one of the uses of "legacy/terraform" aside from calls in the other packages under "legacy". We'll clean up the last one in a later commit, because it's not related to the backend state file format.	2 years ago
hashicorp-copywrite[bot]	53c34ff49c	Update copyright file headers to BUSL-1.1	3 years ago