247 Commits (2cbb6bf92fce17e8de5e681dcc47ed85f32e9b23)

Author	SHA1	Message	Date
James Bardin	2cbb6bf92f	udpate code which was failing staticckeck ... Some updates to staticcheck were catching more errors. Most of these were format functions without a format string, but rather than adding more exceptions I just fixed them all. This did also catch some incorrectly wrapped errors.	1 year ago
kmoe	a2039517a5	states: store ephemeral output values in memory (#35676) ... Ephemeral root output values must be kept in the in-memory state representation, but not written to the state file. To achieve this, we store ephemeral root outputs separately from non-ephemeral root outputs, so Terraform can access them during a single plan or apply phase. Ephemeral root outputs always have a value of null in the state file. This means that the "terraform output" command, that reads the state file, reports null values for these outputs. Consumers of 'terraform output -json' should use the presence of '"ephemeral": true' in such output to interpret the value correctly.	1 year ago
Jeff Bonhag	8b2941455a	Upgrade go-tfe from 1.51.0 to 1.58.0 (#35495) ... * go get github.com/hashicorp/go-tfe@v1.58.0 * Switch to go.uber.org/mock for mocks	2 years ago
Liam Cervante	70b93b24b4	deferred actions: load all values from deferred resources when building eval context (#35526) ... * deferred actions: fix bug when deferred actions are missing from state * move dependency-self reference check back	2 years ago
Brandon Croft	0efd586490	cloud: retry output if current state outputs are not available	2 years ago
Brandon Croft	716fcce239	chore: add command ctx to GetRootOutputValues	2 years ago
Sebastian Rivera	fb06df80fc	Refactor generalError to use appName ... In order to inject the value of TFP-AppName, we will need a backend.Cloud ref available to use. This change modifies generalError as a receiver method for backend.Cloud types. We also duplicate this method as a receiver for TestSuiteRunner, since TestSuiteRunner does not have a backend.Cloud ref.	2 years ago
Sebastian Rivera	015f795ff0	Rebrand Terraform Cloud to HCP Terraform	2 years ago
Rose M Koron	a344e83a39	Apply suggestions from code review ... Co-authored-by: rita <8647768+ritsok@users.noreply.github.com>	2 years ago
Rose	9babc8a1ee	Fix grammar	2 years ago
Rose	0eac58dcab	Change the header names	2 years ago
Mrinali Rao	b1a40a1a65	Update the Terraform CLI output to show logical separation between OPA and Sentinel policy evaluations (#34963) ... * add sentinel policy evaluation title in the CLI results * Add Sentinel policy evaluations to the CLI * add changelog * minor design fixes	2 years ago
Martin Atkins	f058de612c	statemgr: Incorporate IntermediateStateConditionalPersister ... This interface was originally in the local backend's package because that backend is the only one that _truly_ runs Terraform operations and thus needs to negotiate how to store the intermediate state snapshots that result from them. However, putting the interface and its supporting types/functions there means that anything that wants to implement this interface needs to import the local backend package, and thus indirectly depends on the entire modules runtime, etc. We already had package stagemgr as a home for the shared stuff used by most state storage managers, and so we'll just move these symbols over there and then we can shed some internal dependencies on package local. This also severs the last dependency link between the remote state backends and the modules runtime, allowing us to shed a bunch of indirect dependencies from those remote state backends.	2 years ago
Martin Atkins	cf3bbb890f	State managers use schemarepo.Schemas instead of terraform.Schemas ... schemarepo.Schemas used to be terraform.Schemas, but its presence in the terraform package was the only reason for a state manager to need to import package terraform, so we've moved it out into its own package in the parent commit. This means that none of the state-related packages need to import package terraform any longer, and so once we've untangled some other dependencies this will curtail the number of packages in this codebase that the remote state backends transitively depend on, as part of an ongoing effort to eventually separate them from this codebase entirely.	2 years ago
Martin Atkins	88cb4aeea3	backendrun: Separate the types/etc for backends that support operations ... We previously had all of the types and helpers for all kinds of backends together in package backend. That kept things relatively simple, but it also meant that the majority of backends that only deal with remote state storage ended up still indirectly depending on the entire Terraform modules runtime, configuration loader, etc, etc, which brings into scope a bunch of external dependencies that the remote state backends don't really need. Since backends that support operations are a rare exception, we'll move the types and helpers for those into a separate package "backendrun", and then the main package backend can have a much more modest set of types and, more importantly, a modest set of dependencies on other packages in this codebase. This is part of an ongoing effort to reduce the exposure of Terraform Core and CLI code to the remote backends and vice-versa, so that in the long run we can more often treat them as separate for dependency maintenance purposes.	2 years ago
Martin Atkins	06f51dbe04	command/workdir: Model the "backend state" file format ... Historically (before there were "backends") Terraform had a single state file format used both for real state snapshots and for tracking where remote state was stored. Terraform v0.12 caused these two to be split because we adopted state snapshot version 4 for real snapshots but retained a subset of version 3 for tracking the remote backend configuration in the local working directory. Unfortunately we previously kept that working by retaining a snapshot of the entire Terraform v0.11 "terraform" package as legacy/terraform, which happened to still be around because we also needed to retain a copy of the entire legacy SDK to keep the remote state backends working. This now hoists just the tiny slice of legacy terraform package functionality needed to implement the "backend state" file format into package workdir. This package is a good home for it because it's part of the working directory state. Ideally it would be accessed through methods of the workdir.Dir type, but that's too disruptive a refactor to combine into this and so that'll need to wait for another day; for now we'll keep the existing callers doing their access through our "clistate" package that is itself a forked snapshot of what statemgr.Filesystem used to be in Terraform v0.11. This removes all but one of the uses of "legacy/terraform" aside from calls in the other packages under "legacy". We'll clean up the last one in a later commit, because it's not related to the backend state file format.	2 years ago
Martin Atkins	af44fdbbf5	cloud: Use copy.DeepCopyValue instead of copystructure dependency ... The mock implementations of the Terraform Cloud SDK types were previously using the rather-heavy external dependency mitchellh/copystructure for creating deep copies of the mock values. This package doesn't need all of the customizability and generality of that library, and so our smaller internal helper copy.DeepCopyValue is sufficient.	2 years ago
Martin Atkins	1358245617	dependencies: Remove all non-legacy uses of hashicorp/go-multierror ... This functionality is now provided by the "errors" package in the standard library, so the HashiCorp-specific library is obsolete. This also removed our one direct use of github.com/pkg/errors, which we were using in a way that was totally redundant with the stdlib package errors. However, it lives on as an indirect dependency through the Consul backend's dependency on the Consul SDK, which uses that library.	2 years ago
Lauren	6d9b3b2cd7	call Colorize() on CLI output	2 years ago
Nick Fagerlund	c02533c889	Cloud backend: Cache service discovery in public ServicesHost field ... Most of the time, the Cloud backend is the only thing that needs to read the results of this discovery, and the only thing it cares about is the URL to the TFE v2 API service. However: - Other things might need to know about services that your TFC instance provides, including services other than the TFE v2 API. - There's a lot of complexity involved in determining which TFC instance you should be talking to (as well as the credentials for contacting it, the default org/workspace/project you're interested in, etc.), and it makes sense to leave the Cloud backend as the owner of that complexity rather than re-implement it in other potential consumers! So, since other TFC consumers will likely be configuring a Cloud backend to get their information anyway, might as well cache the known services so we don't have to repeat the discovery request. As part of this commit, note that I removed some code that swallowed a particular error during discovery. This was vestigial behavior that came over from the `remote` backend, but we were no longer performing the second part of the process (i.e. producing a more descriptive error in place of the one we swallowed). See https://github.com/hashicorp/terraform/pull/19659 for discussion about the original intent.	2 years ago
Nick Fagerlund	0248b6cf90	Give cloud TestRunner its own service discovery function ... I'm about to refactor the shared one away into a form it can't re-use. This is the only other callsite, so it can just own this logic.	2 years ago
Nick Fagerlund	054fdf93cf	Publicize Cloud's token and organization fields ... Sorry... I need them.	2 years ago
Jamie Finnigan	05b0bdac99	adopt hashicorp/cli fork of mitchellh/cli (#34429)	2 years ago
Brandon Croft	2cf2943626	Update go-tfe mocks	2 years ago
Martin Atkins	b0b8d4aa6f	states: Only track root module output values ... For a very long time we've had an annoying discrepancy between the in-memory state model and our state snapshot format where the in-memory format stores output values for all modules whereas the snapshot format only tracks the root module output values because those are all we actually need to preserve between runs. That design wart was a result of us using the state both as an internal and an external artifact, due to having nowhere else to store the transient values of non-root module output values while Terraform Core does its work. We now have namedvals.State to internally track all of the throwaway results from named values that don't need to persist between runs, so now we'll use that for our internal work instead and reserve the states.State model only for the data that we will preserve between runs in state snapshots. The namedvals internal model isn't really designed to support enumerating all of the output values for a particular module call, but our expression evaluator currently depends on being able to do that and so we have a temporary inefficient implementation of that which just scans the entire table of values as a stopgap just to avoid this commit growing even larger than it already is. In a future commit we'll rework the evaluator to support the PartialEval mode and at the same time move the responsiblity for enumerating all of the output values into the evaluator itself, since it should be able to determine what it's expecting by analyzing the configuration rather than just by trusting that earlier evaluation has completed correctly. Because our legacy state string serialization previously included output values for all modules, some of our context tests were accidentally depending on the implementation detail of how those got stored internally. Those tests are updated here to test only the data that is a real part of Terraform Core's result, by ensuring that the relevant data appears somewhere either in a root output value or in a resource attribute.	2 years ago
Nick Fagerlund	4ec407af08	Disallow saved cloud plans in VCS workspaces ... By convention, Terraform Cloud workspaces connected to a VCS repo should should only apply configs that come from the designated branch, to ensure that version control remains the source of truth for that infrastructure. The cloud version of `terraform apply` already includes a guardrail for that, but we forgot to include an equivalent one for `terraform plan` when performing saved plans (which can be applied later, unlike a purely speculative plan). We suspect that TFC actually needs some additional formal access controls around assigning configuration versions to workspaces, as everyone tends to be surprised to learn that the API doesn't actually prevent this. But in the meantime, what's good for the goose (apply command) is good for the gander (plan command), so I'm copying over the check to voluntarily enforce the VCS workflow for plans that can be applied.	2 years ago
Liam Cervante	e38ca6ea60	Fix race conditions in mock provider and mock tfc client (#34306)	2 years ago
Nick Fagerlund	438e47659b	Remove some redundant test cases for resolveCloudConfig	2 years ago
Nick Fagerlund	26dac26fd1	fix configWithEnvVars tests ... Actually I don't think these _ever_ worked.	2 years ago
Nick Fagerlund	1f9c3eb405	Fix TestCloud_config ... That error happens after PrepareConfig, now.	2 years ago
Nick Fagerlund	d7dadca40f	Re-enable disabled cloud configure tests ... These got turned off in d7e07e66fc by accident	2 years ago
Nick Fagerlund	59c99b1231	Steal most test cases from setConfigurationFields and PrepareConfig ... These tests were rather chaotic, because the basic job of validating the finalized config was split across a bunch of different places. Now they're in (mostly) one spot, testing `resolveCloudConfig`. They might be somewhat excessive or redundant, and can be revised later. In a few cases, I revised test cases that had incorrect expectations, to match the fixed implementation. Also, note that the new implementation is less loosey-goosey around cty values, and expects something that definitely matches the schema (though any amount of it might be null). This commit also splits the TF_FORCE_LOCAL_BACKEND test case into a separate test function.	2 years ago
Nick Fagerlund	be71ab06a8	Add a first test for resolveCloudConfig	2 years ago
Nick Fagerlund	1ae8643201	Refactor `(*Cloud).Configure`, and fix TF_WORKSPACE bug ... This fixes issue #33976, introduced in https://github.com/hashicorp/terraform/pull/33489, which broke the intended behavior of specifying the active workspace via the TF_WORKSPACE variable when using a tag-based workspace mapping. Now that all the default and fallback value behaviors are cleanly isolated in a function, this whole flow can be a bit simpler. - Remove `setConfigurationFields`. Instead, `Configure` can just trade its Cty `obj` for a dumb struct and set a couple fields from it. The `TF_FORCE_LOCAL_BACKEND` handling can just join the relevant section of Configure directly. - Radically chop down PrepareConfig. It turns out we were violating the interface contract, which says PrepareConfig shouldn't trouble itself with the shell environment and fallback values... So, don't do that.	2 years ago
Nick Fagerlund	8e1b65ebc3	Add resolveCloudConfig function ... This will be the central location for everything involving combining environment variables with a `cloud` config block to obtain a final cloud config. It returns a plain Go value (so that nothing downstream of it ever needs to mess with Cty types), and doesn't mutate any fields on the backend, so it has a nice firm boundary of responsibilities. Also, it's quite a bit more pedantic and explicit about HOW the environment variables get consulted, in the hope of reducing future misunderstandings about our UI-level expectations.	2 years ago
Radek Simko	1d1dcc4b8b	cloud: Fix typo & align 'project' field description (#33946)	2 years ago
Liam Cervante	01c0480e77	Test framework will produce regular updates in -json mode (#33953) ... * Test framework will produce regular updates in -json mode * add tests * docs * docs tweaking	2 years ago
Liam Cervante	8360d7566e	Implement double cancel instead of force cancel for test cloud runs (#33951)	2 years ago
Liam Cervante	e37526b4e5	Refactor cloud test run cancellation logic to always intercept signals (#33930) ... * Refactor cloud test run cancellation logic to always intercept signals * add test * add more comments explaining cancellation flow	2 years ago
Liam Cervante	ae6b36b247	fix panic when tests don't finish executing (#33929)	2 years ago
Liam Cervante	ffbcaf8bef	Implement TFC integration for private module registry test runs (#33710) ... * implement module testing via TFC * ready for review * fix static checks * licence headers	2 years ago
Brandon Croft	70cf13cae3	cloud: don't update snapshot interval unless response is from tfe.v2	2 years ago
Brandon Croft	f5463ce77c	cloud: fix ShouldPersistIntermediateState should be false when header absent	2 years ago
Brandon Croft	6500b7c740	change visibility of cloud.Backend Hostname field	3 years ago
hashicorp-copywrite[bot]	53c34ff49c	Update copyright file headers to BUSL-1.1	3 years ago
Brandon Croft	b1d4f18a9b	Create provisional configuration version when saving cloud plan	3 years ago
Karl Kirch	d7e07e66fc	Add ability to specify Terraform Cloud Project in cloud block (#33489) ... * Add ability to specify Terraform Cloud Project in cloud block Adds project configuration to the workspaces section of the cloud block. Also configurable via the `TF_CLOUD_PROJECT` environment variable. When a project is configured, the following behaviors will occur: - `terraform init` with workspaces.name configured will create the workspace in the given project - `terraform workspace new <name>` with workspaces.tags configured will create workspaces in the given project - `terraform workspace list` will list workspaces only from the given project The following behaviors are NOT affected by project configuration - `terraform workspace delete <name>` does not validate the workspace's inclusion in the given project - When initializing a workspace that already exists in Terraform Cloud, the workspace's parent project is NOT validated against the given project Adds tests for cloud block configuration of project Update changelog * Update cloud block docs * Fix typos and changelog entry * Add speculative project lookup early in the cloud initialize process to capture inability to find a configured project * Add project config for alias test	3 years ago
Nick Fagerlund	f98f920b67	Add error wrapper types to highlight bad plan/state data ... This commit uses Go's error wrapping features to transparently add some optional info to certain planfile/state read errors. Specifically, we wrap errors when we think we've identified the file type but are somehow unable to use it. Callers that aren't interested in what we think about our input can just ignore the wrapping; callers that ARE interested can use `errors.As()`.	3 years ago
Nick Fagerlund	1f35173192	Tests: Add tests for `(*Cloud).ShowPlanForRun()`	3 years ago
Nick Fagerlund	7f6b827987	Modify tfe client mocks to meet some new requirements ... - Add plausible unredacted plan json for `plan-json-{basic,full}` testdata -- Created by just running the relevant terraform commands locally. - Add plan-json-no-changes testdata -- The unredacted json was organically grown, but I edited the log and redacted json by hand to match what I observed from a real but unrelated planned-and-finished run in TFC. - Add plan-json-basic-no-unredacted testdata -- This mimics a lack of admin permissions, resulting in a 404. - Hook up `MockPlans.ReadJSONOutput` to test fixtures, when present. This method has been implemented for ages, and has had a backing store for unredacted plan json, but has been effectively a no-op since nothing ever fills that backing store. So, when creating a mock plan, make an attempt to read unredacted json and stow it in the mocks on success. - Make it possible to get the entire MockClient for a test backend In order to test some things, I'm going to need to mess with the internal state of runs and plans beyond what the go-tfe client API allows. I could add magic special-casing to the mock API methods, or I could locate the shenanigans next to the test that actually exploits it. The latter seems more comprehensible, but I need access to the full mock client struct in order to mess with its interior. - Fill in some missing expectations around HasChanges when retrieving a run + plan.	3 years ago