34 Commits (27770ee8053da48fbb684fd23d415aae376d2bed)

Author	SHA1	Message	Date
Radek Simko	06fd817b7c	Remove handling of ephemeral root outputs (#35809) ... * Remove handling of ephemeral root outputs This is effectively reverting ~99% of https://github.com/hashicorp/terraform/pull/35676 The only changes not being reverted are some formatting and deprecation fixes which remain relevant. The code being removed is basically dead code now in the context of root ephemeral outputs being rejected per https://github.com/hashicorp/terraform/pull/35791 * Remove unrelated changes	2 years ago
Brandon Croft	cc54f79ba7	cloud: retry state version pending gracefully	2 years ago
kmoe	a2039517a5	states: store ephemeral output values in memory (#35676) ... Ephemeral root output values must be kept in the in-memory state representation, but not written to the state file. To achieve this, we store ephemeral root outputs separately from non-ephemeral root outputs, so Terraform can access them during a single plan or apply phase. Ephemeral root outputs always have a value of null in the state file. This means that the "terraform output" command, that reads the state file, reports null values for these outputs. Consumers of 'terraform output -json' should use the presence of '"ephemeral": true' in such output to interpret the value correctly.	2 years ago
Brandon Croft	0efd586490	cloud: retry output if current state outputs are not available	2 years ago
Brandon Croft	716fcce239	chore: add command ctx to GetRootOutputValues	2 years ago
Sebastian Rivera	015f795ff0	Rebrand Terraform Cloud to HCP Terraform	2 years ago
Martin Atkins	f058de612c	statemgr: Incorporate IntermediateStateConditionalPersister ... This interface was originally in the local backend's package because that backend is the only one that _truly_ runs Terraform operations and thus needs to negotiate how to store the intermediate state snapshots that result from them. However, putting the interface and its supporting types/functions there means that anything that wants to implement this interface needs to import the local backend package, and thus indirectly depends on the entire modules runtime, etc. We already had package stagemgr as a home for the shared stuff used by most state storage managers, and so we'll just move these symbols over there and then we can shed some internal dependencies on package local. This also severs the last dependency link between the remote state backends and the modules runtime, allowing us to shed a bunch of indirect dependencies from those remote state backends.	2 years ago
Martin Atkins	cf3bbb890f	State managers use schemarepo.Schemas instead of terraform.Schemas ... schemarepo.Schemas used to be terraform.Schemas, but its presence in the terraform package was the only reason for a state manager to need to import package terraform, so we've moved it out into its own package in the parent commit. This means that none of the state-related packages need to import package terraform any longer, and so once we've untangled some other dependencies this will curtail the number of packages in this codebase that the remote state backends transitively depend on, as part of an ongoing effort to eventually separate them from this codebase entirely.	2 years ago
Martin Atkins	b0b8d4aa6f	states: Only track root module output values ... For a very long time we've had an annoying discrepancy between the in-memory state model and our state snapshot format where the in-memory format stores output values for all modules whereas the snapshot format only tracks the root module output values because those are all we actually need to preserve between runs. That design wart was a result of us using the state both as an internal and an external artifact, due to having nowhere else to store the transient values of non-root module output values while Terraform Core does its work. We now have namedvals.State to internally track all of the throwaway results from named values that don't need to persist between runs, so now we'll use that for our internal work instead and reserve the states.State model only for the data that we will preserve between runs in state snapshots. The namedvals internal model isn't really designed to support enumerating all of the output values for a particular module call, but our expression evaluator currently depends on being able to do that and so we have a temporary inefficient implementation of that which just scans the entire table of values as a stopgap just to avoid this commit growing even larger than it already is. In a future commit we'll rework the evaluator to support the PartialEval mode and at the same time move the responsiblity for enumerating all of the output values into the evaluator itself, since it should be able to determine what it's expecting by analyzing the configuration rather than just by trusting that earlier evaluation has completed correctly. Because our legacy state string serialization previously included output values for all modules, some of our context tests were accidentally depending on the implementation detail of how those got stored internally. Those tests are updated here to test only the data that is a real part of Terraform Core's result, by ensuring that the relevant data appears somewhere either in a root output value or in a resource attribute.	2 years ago
Brandon Croft	70cf13cae3	cloud: don't update snapshot interval unless response is from tfe.v2	3 years ago
Brandon Croft	f5463ce77c	cloud: fix ShouldPersistIntermediateState should be false when header absent	3 years ago
hashicorp-copywrite[bot]	53c34ff49c	Update copyright file headers to BUSL-1.1	3 years ago
Brandon Croft	9fe3f7a7b4	cloud: when saving state, create a pending state version then upload ... Create a pending state version followed by a separate state upload When this version of the endpoint fails (It is not yet generally available, or when using with Terraform Enterprise) Fall back to the original call with state content included in the request. This strategy will reduce the amount of save failures due to network latency and gateway timeouts.	3 years ago
Brandon Croft	86eed095b3	Rename disableIntermediateSnapshots > enableIntermediateSnapshots	3 years ago
Sebastian Rivera	d03fd37ee6	Add interval header check to enable snapshots	3 years ago
Martin Atkins	f6737d47e7	cloud: Allow Cloud API to lower the intermediate state snapshot interval ... Previously we just made a hard rule that the state storage for Terraform Cloud would never save any intermediate snapshots at all, as a coarse way to mitigate concerns over heightened Terraform Enterprise storage caused by saving intermediate snapshots. As a better compromise, we'll now create intermediate snapshots at the default interval unless the Terraform Cloud API responds with a special extra header field X-Terraform-Snapshot-Interval, which specifies a different number of seconds (up to 1 hour) to wait before saving the next snapshot. This will then allow Terraform Cloud and Enterprise to provide some dynamic backpressure when needed, either to reduce the disk usage in Terraform Enterprise or in situations where Terraform Cloud is under unusual load and needs to calm the periodic intermediate snapshot writes from clients. This respects the "force persist" mode so that if Terraform CLI is interrupted with SIGINT then it'll still be able to urgently persist a snapshot of whatever state it currently has, in anticipation of probably being terminated with a more aggressive signal very soon.	3 years ago
Martin Atkins	efdc6e52bc	cloud: Skip intermediate state snapshots in Terraform Cloud/Enterprise ... We've seen some concern about the additional storage usage implied by creating intermediate state snapshots for particularly long apply phases that can arise when managing a large number of resource instances together in a single workspace. This is an initial coarse approach to solving that concern, just restoring the original behavior when running inside Terraform Cloud or Enterprise for now and not creating snapshots at all. This is here as a solution of last resort in case we cannot find a better compromise before the v1.5.0 final release. Hopefully a future commit will implement a more subtle take on this which still gets some of the benefits when running in a Terraform Enterprise environment but in a way that will hopefully be less concerning for Terraform Enterprise administrators. This does not affect any other state storage implementation except the Terraform Cloud integration and the "remote" backend's state storage when running inside a TFC/TFE-driven remote execution environment.	3 years ago
hashicorp-copywrite[bot]	325d18262e	[COMPLIANCE] Add Copyright and License Headers	3 years ago
Brandon Croft	de7304cacb	Begin cloud remote state with serial > 0	3 years ago
Jarrett Spiker	060255a9d5	Use safe or force workspace delete for cloud backend	3 years ago
Megan Bang	72ba8a869e	removes EnableForcePush and other unrelated code	4 years ago
Megan Bang	5eaa4c45c0	fix imports	4 years ago
Megan Bang	de8bd5826f	first part of code review comments	4 years ago
Megan Bang	dbf99f17b1	add test and removed backend state from cloud	4 years ago
Megan Bang	bddf6a9b34	updating to use the latest version of cloud/state.go and just pass schemas along to PersistState in the remote state	4 years ago
Megan Bang	344379f5c7	fix cloud state breaking?	4 years ago
Megan Bang	021f1f69e9	updates to cloud state	4 years ago
Megan Bang	4fab46749a	update persist state	4 years ago
Brandon Croft	50d48c635e	remove unused error and fix output authorization detection	4 years ago
Brandon Croft	0139e75a1a	feature: fall back to reading entire state when detailed-type is not available	4 years ago
Brandon Croft	e1fa690879	style: goimports fixes	4 years ago
Brandon Croft	166d21b20b	refactor: extract cloud state ctyValue conversion function	4 years ago
Brandon Croft	e794efc31e	better errors from terraform output when cloud is configured	4 years ago
Brandon Croft	c33c8b013f	fix: have `terraform output` adhere to authorization w/ cloud ... Normally, `terraform output` refreshes and reads the entire state in the command package before pulling output values out of it. This doesn't give Terraform Cloud the opportunity to apply the read state outputs org permission and instead applies the read state versions permission. I decided to expand the state manager interface to provide a separate GetRootOutputValues function in order to give the cloud backend a more nuanced opportunity to fetch just the outputs. This required moving state Refresh/Read code that was previously in the command into the shared backend state as well as the filesystem state packages.	4 years ago