62 Commits (ecfbdaf18d97fec6d019acdb5bbb194f12ecfe92)

Author	SHA1	Message	Date
Sarah French	d1e412fcf0	PSS: Add initial (incomplete) version of code changes to the `init` command for using pluggable state storage (#37321) ... * Store the FQN of the provider used in PSS in representations of the parsed config. This can only be done once modules have been parsed and the required providers data is available. There are multiple places where config is parsed, into either Config or Module structs, so this needs to be implemented in multiple places. * Update affected tests, improve error diagnostic * Begin enabling method to return a backend.Backend made using state_store config. State store config can now be received via BackendOpts and there is rough validation of whether the config makes sense (does the provider offer a store with the given name?). * Update code's logic to include possibility of a state store being in use At this point there are no cases that actually handle the state store scenarios though! * Add empty cases for handle all broad init scenarios involving PSS * Update default case's error to report state store variables * Improve how we resolve the builtin terraform provider's address * Add test that hits the code path for adding a state store to a new (or implied local) project * Add test for use of `-reconfigure` flag; show that it hits the code path for adding a state store for the first time * Add test that hits the code path for removing use of a state store, migrating to (implied) local backend * Add test that hits the code path for changing a state store's configuration * Update existing test names to be backend-specific * Add tests that hits the code path for migrating between PSS and backends * Consolidate PSS-related tests at end of the file * Fix log text * Add test showing that using variables is disallowed with state_store and nested provider blocks * Update test name * Fix test cases * Add TODOs so we remember to remove experiments from tests * Update state store-related tests to use t.Cleanup * Remove use of `testChdir`	9 months ago
Sarah French	145e152a57	Clarify usage of BackendOpts, rename field in struct to enable usage with PSS in future (#37085) ... * Update comments to distinguish between operations and state-storage backends more clearly * Rename `BackendOpts` field `Config` to more specific `BackendConfig`	10 months ago
Sarah French	677e5ea276	Add test coverage for Meta's determineInitReason method, fix error in diagnostic (#37303) ... * Add test coverage for Meta's determineInitReason method, fix error in diagnostic when migrating away from HCP Terraform * Add godoc comment to determineInitReason method	10 months ago
Sarah French	d02a1ed1b7	PSS: Make the state storage provider's config stateful (backend state, planfiles) (#37286)	10 months ago
Sarah French	2349481b23	Update data stored in plan files to enable using PSS with saved plans (#37246) ... * Implement `ForPlan` method on `StateStoreConfigState`, add `Planner` interface * Rename `ForPlan` method to `Plan` * Allow plan files to contain information about state stores * Add code needed for representing a state store in the Plan struct, which is used when handling plan files * Add ability to read/write either a backend or state store's data in a plan file. Add some test coverage. * Update plan's `ProviderAddrs` method to include the provider used for PSS, if present * Split interfaces * Apply feedback from code review * Refactor `SetVersion` to use appropriate constructor * Split `ProviderAddrs` method test into two * Fix method call after rename * Fix test * Remove change to `(p *Plan) ProviderAddrs()` We may re-add this when we implement PSS for use during apply commands with plan files * Remove changes to test, now that the plan doesn't report the provider used for PSS anymore.	10 months ago
Sarah French	d3b02a8469	Refactor code to use backend config state's `SetConfig` method, protect against nil receiver (#37254) ... * Add protection against nil receivers to SetConfig * Refactor code to use SetConfig method instead of setting ConfigRaw directly	10 months ago
Sarah French	2559f0a3db	Update backend state file so it can describe PSS state (#37179) ... * Split code for backend state file vs backend state * Rename BackendState to BackendConfigState * Spelling error * Add `StateStorageConfigState` struct as new implementation of new `ConfigState[T any]` interface. * Split tests for backend state file vs backend config state structs * Rename StateStorageConfigState to StateStoreConfigState * Clarify test name, add comments * Add tests for StateStoreConfigState methods * Add test showing state_store in JSON is parsed correctly * Add detection of malformed backend state files that contain both backend and state_store fields * Add validation that stops a backend state file being written if it will contain state for both backend and state_store blocks * Rename `state_storage` to `state_store` * Rename `state_storage` to `state_store` in filenames * Move`ConfigState` to its own file * Fix test name, remove whitespace * Update `StateStoreConfigState` comment using review suggestion * Update error message to no longer allude to the environment TF is being run in * Update the state_store state to use `version.Version` and an adapted version of `tfaddr.Provider` for marshalling version and source data * Update test helper so it doesn't accidentally supply validation in tests * Add protection against saving an empty backend state file * Remove direct testing of (s *Source) MarshalText() and UnmarshalText() methods * Add Validate method to StateStoreConfigState, use in backend state encoding logic * Refactor to use new features in registry dependency	11 months ago
Sarah French	2d6eda2877	Update comments about backend interfaces, enhanced vs operations, and fix copy-paste inaccuracies (#36716) ... * Fix copy-pasta * Update some comments clarifying backend-related interfaces and "enhanced" versus "operations" * Fix more comments that refer to types and interfaces that have moved into the backendrun package	1 year ago
Bruno Schaatsbergen	a5d8673a41	Revert "Merge pull request #35929 from bschaatsbergen/b/fix-retry-messages-in-cloud-backend-json" ... This reverts commit f8cb25a246, reversing changes made to a25831d6ef.	1 year ago
Bruno Schaatsbergen	f1428b3388	backend/cloud: expose View to enable reuse in a (cloud) backend ... Signed-off-by: Bruno Schaatsbergen <git@bschaatsbergen.com>	1 year ago
Bruno Schaatsbergen	d8bfe7a80b	cloud: Initialize a new cloud view in the backend to render messages based on view type. ... Signed-off-by: Bruno Schaatsbergen <git@bschaatsbergen.com>	1 year ago
Bruno Schaatsbergen	f9797595e3	chore: initialize new view in meta backend ... Signed-off-by: Bruno Schaatsbergen <git@bschaatsbergen.com>	1 year ago
Brandon Croft	9d0133a541	Fix non-string elements in tags schema	2 years ago
Brandon Croft	e87335a972	cloud: Allows object tag schema for selecting kv tagged workspaces ... When key-value tags are enabled and used in the workspace, users may define the tags attribute as a map of strings in the cloud block in order to more precicely match workspaces using those tags.	2 years ago
James Bardin	2cbb6bf92f	udpate code which was failing staticckeck ... Some updates to staticcheck were catching more errors. Most of these were format functions without a format string, but rather than adding more exceptions I just fixed them all. This did also catch some incorrectly wrapped errors.	2 years ago
Sebastian Rivera	015f795ff0	Rebrand Terraform Cloud to HCP Terraform	2 years ago
Martin Atkins	88cb4aeea3	backendrun: Separate the types/etc for backends that support operations ... We previously had all of the types and helpers for all kinds of backends together in package backend. That kept things relatively simple, but it also meant that the majority of backends that only deal with remote state storage ended up still indirectly depending on the entire Terraform modules runtime, configuration loader, etc, etc, which brings into scope a bunch of external dependencies that the remote state backends don't really need. Since backends that support operations are a rare exception, we'll move the types and helpers for those into a separate package "backendrun", and then the main package backend can have a much more modest set of types and, more importantly, a modest set of dependencies on other packages in this codebase. This is part of an ongoing effort to reduce the exposure of Terraform Core and CLI code to the remote backends and vice-versa, so that in the long run we can more often treat them as separate for dependency maintenance purposes.	2 years ago
Martin Atkins	06f51dbe04	command/workdir: Model the "backend state" file format ... Historically (before there were "backends") Terraform had a single state file format used both for real state snapshots and for tracking where remote state was stored. Terraform v0.12 caused these two to be split because we adopted state snapshot version 4 for real snapshots but retained a subset of version 3 for tracking the remote backend configuration in the local working directory. Unfortunately we previously kept that working by retaining a snapshot of the entire Terraform v0.11 "terraform" package as legacy/terraform, which happened to still be around because we also needed to retain a copy of the entire legacy SDK to keep the remote state backends working. This now hoists just the tiny slice of legacy terraform package functionality needed to implement the "backend state" file format into package workdir. This package is a good home for it because it's part of the working directory state. Ideally it would be accessed through methods of the workdir.Dir type, but that's too disruptive a refactor to combine into this and so that'll need to wait for another day; for now we'll keep the existing callers doing their access through our "clistate" package that is itself a forked snapshot of what statemgr.Filesystem used to be in Terraform v0.11. This removes all but one of the uses of "legacy/terraform" aside from calls in the other packages under "legacy". We'll clean up the last one in a later commit, because it's not related to the backend state file format.	2 years ago
hashicorp-copywrite[bot]	53c34ff49c	Update copyright file headers to BUSL-1.1	3 years ago
Liam Cervante	f8b1fe47dd	Make config errors more important during init operations (#33628) ... * make config errors more important during init operations * address comments	3 years ago
Nick Fagerlund	f9d937a4dd	Apply a confirmable run when given a saved cloud plan (#33270) ... It displays a run header with link to web UI, like starting a new plan does, then confirms the run and streams the apply logs. If you can't apply the run (it's from a different workspace, is in an unconfirmable state, etc. etc.), it displays an error instead. Notable points along the way: * Implement `WrappedPlanFile` sum type, and update planfile consumers to use it instead of a plain `planfile.Reader`. * Enable applying a saved cloud plan * Update TFC mocks — add org name to workspace, and minimal support for includes on MockRuns.ReadWithOptions.	3 years ago
Brandon Croft	c1a730314d	Relocate localterraform.com aliasing to backend configurators ... Previously, remote and cloud backends would automatically alias localterraform.com as the configured hostname during configuration. This turned out to be an issue with how backends could potentially be used within the builtin terraform_remote_state data source. Those data sources each configure the same service discovery with different targets for localterraform.com, and do so simultaneously, creating an occasional concurrent map read & write panic when multiple data sources are defined. localterraform.com is obviously not useful for every backend configuration. Therefore, I relocated the alias configuration to the callers, so they may specify when to use it. The modified design adds a new method to backend.Enhanced to allow configurators to ask which aliases should be defined.	3 years ago
Martin Atkins	7bee77bdd3	command: Start of propagating OpenTelemetry context ... Several times over the years we've considered adding tracing instrumentation to Terraform, since even when running in isolation as a CLI program it has a "distributed system-like" structure, with lots of concurrent internal work and also some work delegated to provider plugins that are essentially temporarily-running microservices. However, it's always felt a bit overwhelming to do it because much of Terraform predates the Go context.Context idiom and so it's tough to get a clean chain of context.Context values all the way down the stack without disturbing a lot of existing APIs. This commit aims to just get that process started by establishing how a context can propagate from "package main" into the command package, focusing initially on "terraform init" and some other commands that share some underlying functions with that command. OpenTelemetry has emerged as a de-facto industry standard and so this uses its API directly, without any attempt to hide it behind an abstraction. The OpenTelemetry API is itself already an adapter layer, so we should be able to swap in any backend that uses comparable concepts. For now we just discard the tracing reports by default, and allow users to opt in to delivering traces over OTLP by setting an environment variable when running Terraform (the environment variable was established in an earlier commit, so this commit builds on that.) When tracing collection is enabled, every Terraform CLI run will generate at least one overall span representing the command that was run. Some commands might also create child spans, but most currently do not.	3 years ago
hashicorp-copywrite[bot]	325d18262e	[COMPLIANCE] Add Copyright and License Headers	3 years ago
zetHannes	c70244426a	Fix for no json output of state locking actions for --json flag (#32451) ... * Add viewType to Meta object and use it at the call sites * Assign viewType passed from flags to state-locking cli commands * Remove temp files * Set correct mode for statelocker depending on json flag passed to commands * Add StateLocker interface conformation check for StateLockerJSON * Remove empty line at end of comment * Pass correct ViewType to StateLocker from Backend call chain * Pass viewType to backend migration and initialization functions * Remove json processing info in process comment * Restore documentation style of backendMigrateOpts	3 years ago
Megan Bang	4fab46749a	update persist state	4 years ago
James Bardin	953c448f9a	add simple error indicating backend removal ... There are no good options for inserting diagnostics into the backend lookup, or creating a backend which reports it's removal because none of the init or GetSchema functions return any errors. Keep a registry of the removed backend so that we can at least notify users that a backend was removed vs an invalid name.	4 years ago
uk1288	9093b487fd	fix for cloud integration panic	4 years ago
Chris Arcand	98978b3853	command/meta_backend: Allow the remote backend to have no workspaces [again] ... A regression introduced in d72a413ef8 The comment explains, but TLDR: The remote backend actually *depended* on being able to write it's backend state even though an 'error' occurred (no workspaces).	4 years ago
Barrett Clark	e08a02e7bf	Fixes Issue #29959, Apply w/o init error message ... When going from a local backend to Terraform Cloud, if you skip the `terraform init` and run `terraform apply` this will give the user more clear instructions.	4 years ago
Luces Huayhuaca	d72a413ef8	command/meta_backend: Prompt to select workspace before saving backend config (#29945) ... When terraform detects that a user has no workspaces that map to their current configuration, it will prompt the user to create a new workspace and enter a value name. If the user ignores the prompt and exits it, the legacy backend (terraform.tfstate) will be left in a awkward state: 1. This saved backend config will show a diff for the JSON attributes "serial", "tags" and "hash" 2. "Terraform workspace list" will show an empty list 3. "Terraform apply" will run successfully using the previous workspace, from the previous config, not the one from the current saved backend config 4. The cloud config is not reflective of the current working directory Solution: If the user exits the prompt, the saved backend config should not be updated because they did not select a new workspace. They are back at the beginning where they are force to re run the init cmd again before proceeding with new changes.	4 years ago
Luces Huayhuaca	f63b6198ca	Create a function for logic that assigns value to initReason var after changing backend configuration (#29967) ... * Create a function for logic that assigns value to initReason var after changing backend configuration Create func determineInitReason() for logic block that assigns value to initReason var after changing backend/cloud configuration block or migrating to a different type of backend configuration. Also clarify 'cloud' configuration block message to say 'Terraform Cloud configuration block has changed' instead of 'Terraform Cloud configuration has changed'.	4 years ago
Martin Atkins	bac59d2480	command/init: Be explicit that some options are not relevant for Cloud ... There are a few command line options for "terraform init" which are only relevant when working with traditional backends, with the Cloud integration previously just mostly ignoring them, or sometimes misbehaving slightly due to them creating an unreasonable situation. Now we'll catch these and return explicit errors, in order to be clear that these options are not needed nor supported in Cloud mode.	4 years ago
Chris Arcand	bf76cc98ef	command: Suggestions on migration copy from user feedback ... Some small edits to the TFC migration copy.	4 years ago
Chris Arcand	779c958fbf	cloud: Add streamlined 'remote' backend state migration path ... For Terraform Cloud users using the 'remote' backend, the existing 'pattern' prompt should work just fine - but because their workspaces are already present in TFC, the 'migration' here is really just realigning their local workspaces with Terraform Cloud. Instead of forcing users to do the mental gymnastics of what it means to migrate from 'prefix' - and because their remote workspaces probably already exist and already conform to Terraform Cloud's naming concerns - streamline the process for them and calculate the necessary pattern to migrate as-is, without any user intervention necessary.	5 years ago
Chris Arcand	7c0c2e952f	command: Don't always update backend hash when fetching the saved backend ... The Meta.backend_C_r_S_unchanged() method was sadly a bit of a mess. It seems to have originally been used as a method to be called when the backend is not changing, with an extra assumption that if the configured backend's hash doesn't match the one in state, surely the hash should just be updated as an option might have been moved to command line flags. However, this function was used throughout this file as 'the method to load the initialized (but not necessarily configured) backend', regardless of whether or not it is the same (unchanged). This is in addition to Meta.backendFromState(), which is used to load the same thing except in the main codepath of 'init -backend=false'. These changes separate the concerns of backend_C_r_S_unchanged() by 1) Fetching the saved backend (savedBackend()) 2) Updating the hash value in the backend cache when appropriate (either by leaving it to the caller to do themselves or by calling updateSavedupdateSavedBackendHash()) This allows migration codepaths to *not* update the hash value until after a migration has successfully taken place.	5 years ago
Nick Fagerlund	02e62c9851	Cloud: Init without erroring when no workspaces match the `tags` (#29835) ... Previously, `terraform init` was throwing an error if you configured the cloud block with `tags` and there weren't any tagged workspaces yet. Confusing and alienating, since that that's a fairly normal situation! Basically TFC was handling an empty list of workspaces worse than other backends, because it doesn't support an unnamed default workspace. This commit catches that condition during `Meta.selectBackend()` and asks the user to pick a name for their first tagged workspace. If they cancel out, we still error, but if we know what name they want, we can handle it the same way as a nonexistent workspace specified in `name` -- just pass it to `Meta.SetWorkspace()`, and let the workspace get implicitly created when `InitCommand.Run()` eventually calls `StateMgr()`.	5 years ago
Chris Arcand	511afcd43a	command: Adjust skipping of resource counts for any remote implementation ... When using the Terraform Cloud integration - like the 'remote' backend - resource count output should be suppressed if those counts are being rendered remotely. This generalizes this to the shared BackendWithRemoteTerraformVersion interface.	5 years ago
Barrett Clark	f5366468b4	Cloud Backend reference migrating away from TFC	5 years ago
Chris Arcand	2c0294c7e3	Tweak configuration copy for TFC ... This aligns more with the existing copy for backends	5 years ago
Barrett Clark	cc6de251d8	Update init reconfigure error message ... If you move from the remote backend to the cloud block you will see this error message.	5 years ago
Barrett Clark	e16c53b561	Simplify/Consolidate the cloud conditionals	5 years ago
Barrett Clark	ab304d831f	Found another path to backend init error ... There are actually a few different ways to get to this message. 1. Blank state — no previous terraform applied. Start with a cloud block. 1. Implicit local — start with no backend specified. This actually goes through the same code execution path as the first scenario. 1. Explicit local — start with a backend local block that has been applied, then change from the local backend to a cloud block. This will recognize the state, and is a different path through the code in the meta backend. This commit handles the last case. The messaging has also been tweaked. End to end test included as well.	5 years ago
Barrett Clark	d29532cfeb	tfc apply before init ... https://app.asana.com/0/1199201948575144/1201019450474843/f	5 years ago
Barrett Clark	93bfcff61a	More Terraform Cloud UX changes ... * Update e2e tests to specify Terraform Cloud * Update more of the terraform command flow to specify Terraform Cloud	5 years ago
Chris Arcand	f8256f6634	Update ux for "terraform init [-reconfigure]"	5 years ago
Chris Arcand	a4c24e3147	Add cloud {} configuration block for Terraform Cloud ... This is a replacement declaration for using Terraform Cloud as a remote backend, leaving the literal backend as an implementation detail and not a user-level concept.	5 years ago
Alisdair McDiarmid	ded4f1a0fd	Merge pull request #29805 from hashicorp/alisdair/fix-init-workspace-select-input-false ... command/init: Fail if -input=false but required	5 years ago
Alisdair McDiarmid	ecb98e1c43	command/init: Fail if -input=false but required ... When running `terraform init` against a backend with multiple workspaces, none of which are the currently indicated local workspace, Terraform prompts the user to choose a workspace from the list. In automation, using the `-input=false` argument should disable asking for input, but previously would hang instead.	5 years ago
Alisdair McDiarmid	d42d83572b	cli: Fix backend init failure with deleted cache ... When an explicit backend is configured with a configuration which has not yet been initialized, running `terraform init` performs a state migration to fetch the remotely stored state in order to operate on it. Like the previous bug introduced by the recent provider diagnostics change, this code path was not correctly configured to enable init mode for the backend, which resulted in a fatal error during init when the cache dir is deleted. Setting the `Init` backend option allows this code path to continue without error when first initializing the backend for state migration. The new e2e test fails without this change.	5 years ago