terraform

Commit Graph

Author	SHA1	Message	Date
Nick Fagerlund	c02533c889	Cloud backend: Cache service discovery in public ServicesHost field Most of the time, the Cloud backend is the only thing that needs to read the results of this discovery, and the only thing it cares about is the URL to the TFE v2 API service. However: - Other things might need to know about services that your TFC instance provides, including services other than the TFE v2 API. - There's a lot of complexity involved in determining which TFC instance you should be talking to (as well as the credentials for contacting it, the default org/workspace/project you're interested in, etc.), and it makes sense to leave the Cloud backend as the owner of that complexity rather than re-implement it in other potential consumers! So, since other TFC consumers will likely be configuring a Cloud backend to get their information anyway, might as well cache the known services so we don't have to repeat the discovery request. As part of this commit, note that I removed some code that swallowed a particular error during discovery. This was vestigial behavior that came over from the `remote` backend, but we were no longer performing the second part of the process (i.e. producing a more descriptive error in place of the one we swallowed). See https://github.com/hashicorp/terraform/pull/19659 for discussion about the original intent.	2 years ago
Nick Fagerlund	0248b6cf90	Give cloud TestRunner its own service discovery function I'm about to refactor the shared one away into a form it can't re-use. This is the only other callsite, so it can just own this logic.	2 years ago
Nick Fagerlund	054fdf93cf	Publicize Cloud's token and organization fields Sorry... I need them.	2 years ago
Jamie Finnigan	05b0bdac99	adopt hashicorp/cli fork of mitchellh/cli (#34429 )	2 years ago
Brandon Croft	2cf2943626	Update go-tfe mocks	2 years ago
Martin Atkins	b0b8d4aa6f	states: Only track root module output values For a very long time we've had an annoying discrepancy between the in-memory state model and our state snapshot format where the in-memory format stores output values for all modules whereas the snapshot format only tracks the root module output values because those are all we actually need to preserve between runs. That design wart was a result of us using the state both as an internal and an external artifact, due to having nowhere else to store the transient values of non-root module output values while Terraform Core does its work. We now have namedvals.State to internally track all of the throwaway results from named values that don't need to persist between runs, so now we'll use that for our internal work instead and reserve the states.State model only for the data that we will preserve between runs in state snapshots. The namedvals internal model isn't really designed to support enumerating all of the output values for a particular module call, but our expression evaluator currently depends on being able to do that and so we have a temporary inefficient implementation of that which just scans the entire table of values as a stopgap just to avoid this commit growing even larger than it already is. In a future commit we'll rework the evaluator to support the PartialEval mode and at the same time move the responsiblity for enumerating all of the output values into the evaluator itself, since it should be able to determine what it's expecting by analyzing the configuration rather than just by trusting that earlier evaluation has completed correctly. Because our legacy state string serialization previously included output values for all modules, some of our context tests were accidentally depending on the implementation detail of how those got stored internally. Those tests are updated here to test only the data that is a real part of Terraform Core's result, by ensuring that the relevant data appears somewhere either in a root output value or in a resource attribute.	2 years ago
Nick Fagerlund	4ec407af08	Disallow saved cloud plans in VCS workspaces By convention, Terraform Cloud workspaces connected to a VCS repo should should only apply configs that come from the designated branch, to ensure that version control remains the source of truth for that infrastructure. The cloud version of `terraform apply` already includes a guardrail for that, but we forgot to include an equivalent one for `terraform plan` when performing saved plans (which can be applied later, unlike a purely speculative plan). We suspect that TFC actually needs some additional formal access controls around assigning configuration versions to workspaces, as everyone tends to be surprised to learn that the API doesn't actually prevent this. But in the meantime, what's good for the goose (apply command) is good for the gander (plan command), so I'm copying over the check to voluntarily enforce the VCS workflow for plans that can be applied.	2 years ago
Liam Cervante	e38ca6ea60	Fix race conditions in mock provider and mock tfc client (#34306 )	2 years ago
Nick Fagerlund	438e47659b	Remove some redundant test cases for resolveCloudConfig	3 years ago
Nick Fagerlund	26dac26fd1	fix configWithEnvVars tests Actually I don't think these _ever_ worked.	3 years ago
Nick Fagerlund	1f9c3eb405	Fix TestCloud_config That error happens after PrepareConfig, now.	3 years ago
Nick Fagerlund	d7dadca40f	Re-enable disabled cloud configure tests These got turned off in `d7e07e66fc` by accident	3 years ago
Nick Fagerlund	59c99b1231	Steal most test cases from setConfigurationFields and PrepareConfig These tests were rather chaotic, because the basic job of validating the finalized config was split across a bunch of different places. Now they're in (mostly) one spot, testing `resolveCloudConfig`. They might be somewhat excessive or redundant, and can be revised later. In a few cases, I revised test cases that had incorrect expectations, to match the fixed implementation. Also, note that the new implementation is less loosey-goosey around cty values, and expects something that definitely matches the schema (though any amount of it might be null). This commit also splits the TF_FORCE_LOCAL_BACKEND test case into a separate test function.	3 years ago
Nick Fagerlund	be71ab06a8	Add a first test for resolveCloudConfig	3 years ago
Nick Fagerlund	1ae8643201	Refactor `(*Cloud).Configure`, and fix TF_WORKSPACE bug This fixes issue #33976, introduced in https://github.com/hashicorp/terraform/pull/33489, which broke the intended behavior of specifying the active workspace via the TF_WORKSPACE variable when using a tag-based workspace mapping. Now that all the default and fallback value behaviors are cleanly isolated in a function, this whole flow can be a bit simpler. - Remove `setConfigurationFields`. Instead, `Configure` can just trade its Cty `obj` for a dumb struct and set a couple fields from it. The `TF_FORCE_LOCAL_BACKEND` handling can just join the relevant section of Configure directly. - Radically chop down PrepareConfig. It turns out we were violating the interface contract, which says PrepareConfig shouldn't trouble itself with the shell environment and fallback values... So, don't do that.	3 years ago
Nick Fagerlund	8e1b65ebc3	Add resolveCloudConfig function This will be the central location for everything involving combining environment variables with a `cloud` config block to obtain a final cloud config. It returns a plain Go value (so that nothing downstream of it ever needs to mess with Cty types), and doesn't mutate any fields on the backend, so it has a nice firm boundary of responsibilities. Also, it's quite a bit more pedantic and explicit about HOW the environment variables get consulted, in the hope of reducing future misunderstandings about our UI-level expectations.	3 years ago
Radek Simko	1d1dcc4b8b	cloud: Fix typo & align 'project' field description (#33946 )	3 years ago
Liam Cervante	01c0480e77	Test framework will produce regular updates in -json mode (#33953 ) * Test framework will produce regular updates in -json mode * add tests * docs * docs tweaking	3 years ago
Liam Cervante	8360d7566e	Implement double cancel instead of force cancel for test cloud runs (#33951 )	3 years ago
Liam Cervante	e37526b4e5	Refactor cloud test run cancellation logic to always intercept signals (#33930 ) * Refactor cloud test run cancellation logic to always intercept signals * add test * add more comments explaining cancellation flow	3 years ago
Liam Cervante	ae6b36b247	fix panic when tests don't finish executing (#33929 )	3 years ago
Liam Cervante	ffbcaf8bef	Implement TFC integration for private module registry test runs (#33710 ) * implement module testing via TFC * ready for review * fix static checks * licence headers	3 years ago
Brandon Croft	70cf13cae3	cloud: don't update snapshot interval unless response is from tfe.v2	3 years ago
Brandon Croft	f5463ce77c	cloud: fix ShouldPersistIntermediateState should be false when header absent	3 years ago
Brandon Croft	6500b7c740	change visibility of cloud.Backend Hostname field	3 years ago
hashicorp-copywrite[bot]	53c34ff49c	Update copyright file headers to BUSL-1.1	3 years ago
Brandon Croft	b1d4f18a9b	Create provisional configuration version when saving cloud plan	3 years ago
Karl Kirch	d7e07e66fc	Add ability to specify Terraform Cloud Project in cloud block (#33489 ) * Add ability to specify Terraform Cloud Project in cloud block Adds project configuration to the workspaces section of the cloud block. Also configurable via the `TF_CLOUD_PROJECT` environment variable. When a project is configured, the following behaviors will occur: - `terraform init` with workspaces.name configured will create the workspace in the given project - `terraform workspace new <name>` with workspaces.tags configured will create workspaces in the given project - `terraform workspace list` will list workspaces only from the given project The following behaviors are NOT affected by project configuration - `terraform workspace delete <name>` does not validate the workspace's inclusion in the given project - When initializing a workspace that already exists in Terraform Cloud, the workspace's parent project is NOT validated against the given project Adds tests for cloud block configuration of project Update changelog * Update cloud block docs * Fix typos and changelog entry * Add speculative project lookup early in the cloud initialize process to capture inability to find a configured project * Add project config for alias test	3 years ago
Nick Fagerlund	f98f920b67	Add error wrapper types to highlight bad plan/state data This commit uses Go's error wrapping features to transparently add some optional info to certain planfile/state read errors. Specifically, we wrap errors when we think we've identified the file type but are somehow unable to use it. Callers that aren't interested in what we think about our input can just ignore the wrapping; callers that ARE interested can use `errors.As()`.	3 years ago
Nick Fagerlund	1f35173192	Tests: Add tests for `(*Cloud).ShowPlanForRun()`	3 years ago
Nick Fagerlund	7f6b827987	Modify tfe client mocks to meet some new requirements - Add plausible unredacted plan json for `plan-json-{basic,full}` testdata -- Created by just running the relevant terraform commands locally. - Add plan-json-no-changes testdata -- The unredacted json was organically grown, but I edited the log and redacted json by hand to match what I observed from a real but unrelated planned-and-finished run in TFC. - Add plan-json-basic-no-unredacted testdata -- This mimics a lack of admin permissions, resulting in a 404. - Hook up `MockPlans.ReadJSONOutput` to test fixtures, when present. This method has been implemented for ages, and has had a backing store for unredacted plan json, but has been effectively a no-op since nothing ever fills that backing store. So, when creating a mock plan, make an attempt to read unredacted json and stow it in the mocks on success. - Make it possible to get the entire MockClient for a test backend In order to test some things, I'm going to need to mess with the internal state of runs and plans beyond what the go-tfe client API allows. I could add magic special-casing to the mock API methods, or I could locate the shenanigans next to the test that actually exploits it. The latter seems more comprehensible, but I need access to the full mock client struct in order to mess with its interior. - Fill in some missing expectations around HasChanges when retrieving a run + plan.	3 years ago
Nick Fagerlund	d5938f6b45	Add `(*Cloud).ShowPlanForRun` method to support `terraform show` To do the "human" version of showing a plan, we need more than just the redacted plan JSON itself; we also need a bunch of extra information that only the Cloud backend is in a position to find out (since it's the only one holding a configured go-tfe client instance). So, this method takes a run ID and hostname, finds out everything we're going to need, and returns it wrapped up in a RemotePlanJSON struct.	3 years ago
Nick Fagerlund	e0af3e25e0	Add `cloudplan.RemotePlanJSON` wrapper struct for keeping plan metadata together	3 years ago
Nick Fagerlund	2a08a5b46e	Cloud: Split private `readRedactedPlan` func into two Since `terraform show -json` needs to get a raw hunk of json bytes and sling it right back out again, it's going to be more convenient if plain `show` can ALSO take in raw json. In order for that to happen, I need a function that basically acts like `client.Plans.ReadJSONOutput()`, without eagerly unmarshalling that `jsonformat.Plan` struct. As a slight bonus, this also lets us make the tfe client mocks slightly stupider.	3 years ago
Nick Fagerlund	da963a13b9	Implement plan -out for Cloud - Don't save errored plans. - Call op.View.PlanNextStep for terraform plan in cloud mode (We never used to show this footer, because we didn't support -out.) - Create non-speculative config version if saving plan - Rewrite TestCloud_planWithPath to expect success!	3 years ago
Nick Fagerlund	f9d937a4dd	Apply a confirmable run when given a saved cloud plan (#33270 ) It displays a run header with link to web UI, like starting a new plan does, then confirms the run and streams the apply logs. If you can't apply the run (it's from a different workspace, is in an unconfirmable state, etc. etc.), it displays an error instead. Notable points along the way: * Implement `WrappedPlanFile` sum type, and update planfile consumers to use it instead of a plain `planfile.Reader`. * Enable applying a saved cloud plan * Update TFC mocks — add org name to workspace, and minimal support for includes on MockRuns.ReadWithOptions.	3 years ago
Nick Fagerlund	9d85f18930	Add NewSavedPlanBookmark function so we don't have to remember the format version number	3 years ago
Lauren	dcccd3b266	Add error handling and tests for saved plan bookmark load/save (#33268 )	3 years ago
Nick Fagerlund	df7a1c821a	Move SavedPlanBookmark to dedicated `internal/cloud/cloudplan` package Having this sitting loose in `cloud` proved problematic because of circular import dependencies. Putting it in a sub-package under cloud frees us up to reference the type from places like `internal/backend`!	3 years ago
Lauren	717a36036d	codify saved plan artifact contents (#33185 )	3 years ago
Brandon Croft	c1a730314d	Relocate localterraform.com aliasing to backend configurators Previously, remote and cloud backends would automatically alias localterraform.com as the configured hostname during configuration. This turned out to be an issue with how backends could potentially be used within the builtin terraform_remote_state data source. Those data sources each configure the same service discovery with different targets for localterraform.com, and do so simultaneously, creating an occasional concurrent map read & write panic when multiple data sources are defined. localterraform.com is obviously not useful for every backend configuration. Therefore, I relocated the alias configuration to the callers, so they may specify when to use it. The modified design adds a new method to backend.Enhanced to allow configurators to ask which aliases should be defined.	3 years ago
Liam Cervante	6882dd9530	testing framework: introduce test command optional flags (#33504 ) * testing framework: introduce test command optional flags * address consistency checks	3 years ago
James Bardin	cf5ebaf685	s/Schemas/ProviderSchema/	3 years ago
James Bardin	36d34135ad	replace remaining types in tests	3 years ago
Brandon Croft	9fe3f7a7b4	cloud: when saving state, create a pending state version then upload Create a pending state version followed by a separate state upload When this version of the endpoint fails (It is not yet generally available, or when using with Terraform Enterprise) Fall back to the original call with state content included in the request. This strategy will reduce the amount of save failures due to network latency and gateway timeouts.	3 years ago
kmoe	5900d1177c	cloud: assert import block compatibility (#33282 ) * cloud: assert import block compatibility * check for import <> TFC compatibility during init * imports are not in alphabetical order 🙃 --------- Co-authored-by: CJ Horton <cjhorton@hashicorp.com>	3 years ago
CJ Horton	8213513e2b	Merge pull request #33278 from hashicorp/radditude/cloud-config-generation plannable import: allow writing generated config when using the cloud integration	3 years ago
CJ Horton	6ca13bfc1e	fix static check errors	3 years ago
Brandon Croft	86eed095b3	Rename disableIntermediateSnapshots > enableIntermediateSnapshots	3 years ago
CJ Horton	cdce4c4a6d	write generated config when using the cloud integration	3 years ago

1 2 3 4 5

228 Commits (1298fcd41259c2cf66306cce2453efd2c19f445d)