128 Commits (c99f5972dee6f62eaaf96ba74ab3cd4e71d19caa)

Author	SHA1	Message	Date
Omar Ismail	1ff7827416	Multi-state migration prompt fix	4 years ago
Barrett Clark	cc6de251d8	Update init reconfigure error message ... If you move from the remote backend to the cloud block you will see this error message.	4 years ago
Barrett Clark	e16c53b561	Simplify/Consolidate the cloud conditionals	4 years ago
Barrett Clark	ab304d831f	Found another path to backend init error ... There are actually a few different ways to get to this message. 1. Blank state — no previous terraform applied. Start with a cloud block. 1. Implicit local — start with no backend specified. This actually goes through the same code execution path as the first scenario. 1. Explicit local — start with a backend local block that has been applied, then change from the local backend to a cloud block. This will recognize the state, and is a different path through the code in the meta backend. This commit handles the last case. The messaging has also been tweaked. End to end test included as well.	4 years ago
Barrett Clark	d29532cfeb	tfc apply before init ... https://app.asana.com/0/1199201948575144/1201019450474843/f	4 years ago
Barrett Clark	e49f271eb5	Update copy for terraform init with cloud block	4 years ago
Chris Arcand	09d5b70b7a	command/meta_backend_migrate:Tweak multi-to-single To TFC copy	4 years ago
Omar Ismail	dfb4609be2	Backend State Migration from cloud to cloud. ... * Add test for tfc to tfc mgiration * Fix old tests, and remove unused code.	4 years ago
Barrett Clark	93bfcff61a	More Terraform Cloud UX changes ... * Update e2e tests to specify Terraform Cloud * Update more of the terraform command flow to specify Terraform Cloud	4 years ago
Barrett Clark	261a2b49d3	UX for terraform init that copies state to TFC	4 years ago
Barrett Clark	0cd9490838	PR Feedback: cleanup variable setter	4 years ago
Chris Arcand	f8256f6634	Update ux for "terraform init [-reconfigure]"	4 years ago
Omar Ismail	dc76bbee73	Backend State Migration: Add remote backend test	4 years ago
Omar Ismail	3fedd6898c	Backend State Migration to `cloud`: Multiple Workspaces ... * Handle when there are multiple workspaces migrating to cloud, using both the cloud name strategy and cloud tags strategy. * Add e2e tests	4 years ago
Barrett Clark	edbc84420c	Run apply -refresh-state instead of refresh ... When a user runs `terraform refresh` we give them an error message that tells them to run `terraform apply -refresh-state`. We could just run that command for them, though. That is what this PR does.	4 years ago
Omar Ismail	55fc590904	Teraform Cloud Backend State Migration ... * determining source or destination to cloud * handling single to single state migrations to cloud, using a name strategy or a tags strategy * Add end-to-end tests for state migration.	4 years ago
Chris Arcand	fccc873a3d	Remove ability to declare a 'cloud' backend ... With the alternative block introduced in 7bf9b2c7b, this removes the ability to explicitly declare the 'cloud' backend. The literal backend interface is an implementation detail and no longer a user-level concept when using Terraform Cloud.	4 years ago
Chris Arcand	a4c24e3147	Add cloud {} configuration block for Terraform Cloud ... This is a replacement declaration for using Terraform Cloud as a remote backend, leaving the literal backend as an implementation detail and not a user-level concept.	4 years ago
Alisdair McDiarmid	ded4f1a0fd	Merge pull request #29805 from hashicorp/alisdair/fix-init-workspace-select-input-false ... command/init: Fail if -input=false but required	4 years ago
Alisdair McDiarmid	ecb98e1c43	command/init: Fail if -input=false but required ... When running `terraform init` against a backend with multiple workspaces, none of which are the currently indicated local workspace, Terraform prompts the user to choose a workspace from the list. In automation, using the `-input=false` argument should disable asking for input, but previously would hang instead.	4 years ago
Alisdair McDiarmid	6d0521e6be	Merge pull request #29803 from hashicorp/alisdair/init-backend-provider-errors ... cli: Fix backend init failure with deleted cache	4 years ago
Alisdair McDiarmid	22923f9873	Merge pull request #29793 from hashicorp/alisdair/fix-remote-backend-migrate-version-check ... backend/remote: Fix version check when migrating	4 years ago
Alisdair McDiarmid	d42d83572b	cli: Fix backend init failure with deleted cache ... When an explicit backend is configured with a configuration which has not yet been initialized, running `terraform init` performs a state migration to fetch the remotely stored state in order to operate on it. Like the previous bug introduced by the recent provider diagnostics change, this code path was not correctly configured to enable init mode for the backend, which resulted in a fatal error during init when the cache dir is deleted. Setting the `Init` backend option allows this code path to continue without error when first initializing the backend for state migration. The new e2e test fails without this change.	4 years ago
Alisdair McDiarmid	1729431520	backend/remote: Fix version check when migrating ... When migrating state to an existing Terraform Cloud workspace using the remote backend, we check the remote version is compatible with the local one by default. This commit fixes two bugs in this code: - If using the "name" strategy for the remote backend, the list of destination workspaces is empty. This resulted in no version checking of the remote workspace, and we fell back to the string equality check. - The user-specified CLI flag `-ignore-remote-version` was not being applied for the state migration version checking.	4 years ago
Alisdair McDiarmid	39de3ebec7	cli: Fix init failure with deleted cache ... The init command needs to initialize a backend, in order to access state, in turn to derive provider requirements from state. The backend initialization step requires building provider factories, which previously would fail if a lockfile was present without a corresponding local provider cache. This commit ensures that in this situation only, errors with the provider factories are temporarily ignored. This allows us to continue to initialize the backend, fetch providers, and then report any errors as necessary.	4 years ago
Alisdair McDiarmid	1190b95fe2	command/e2etest: Ensure init fixes missing cache ... We test that a deleted provider cache results in an error when running terraform plan, but previously did not test that running init (as instructed) would resolve the issue. This (failing) e2e test adds that step.	4 years ago
Martin Atkins	5b266dd5ca	command: Remove the experimental "terraform add" command ... We introduced this experiment to gather feedback, and the feedback we saw led to us deciding to do another round of design work before we move forward with something to meet this use-case. In addition to being experimental, this has only been included in alpha releases so far, and so on both counts it is not protected by the Terraform v1.0 Compatibility Promises.	4 years ago
Alisdair McDiarmid	cdd5ee6fb3	Merge pull request #29773 from hashicorp/alisdair/init-lock-flags ... cli: Restore -lock and -lock-timeout init flags	4 years ago
Alisdair McDiarmid	c587384dff	cli: Restore -lock and -lock-timeout init flags ... The -lock and -lock-timeout flags were removed prior to the release of 1.0 as they were thought to have no effect. This is not true in the case of state migrations when changing backends. This commit restores these flags, and adds test coverage for locking during backend state migration. Also update the help output describing other boolean flags, showing the argument as the user would type it rather than the default behavior.	4 years ago
Alisdair McDiarmid	fb58f9e6d2	cli: Fix flaky init cancel test ... There is a race between the MockSource and ShutdownCh which sometimes causes this test to fail. Add a HangingSource implementation of Source which hangs until the context is cancelled, so that there is always time for a user-initiated shutdown to trigger the cancellation code path under test.	4 years ago
James Bardin	ef3c98466d	Merge pull request #29755 from hashicorp/jbardin/first-plan-lineage ... Check for stale plan with no state metadata	4 years ago
James Bardin	9c80574417	test planfile may need to have a specific lineage ... In order to test applying a plan from an existing state, we need to be able to inject the state meta into the planfile.	4 years ago
Martin Atkins	bee7403f3e	command/workspace_delete: Allow deleting a workspace with empty husks ... Previously we would reject attempts to delete a workspace if its state contained any resources at all, even if none of the resources had any resource instance objects associated with it. Nowadays there isn't any situation where the normal Terraform workflow will leave behind resource husks, and so this isn't as problematic as it might've been in the v0.12 era, but nonetheless what we actually care about for this check is whether there might be any remote objects that this state is tracking, and for that it's more precise to look for non-nil resource instance objects, rather than whole resources. This also includes some adjustments to our error messaging to give more information about the problem and to use terminology more consistent with how we currently talk about this situation in our documentation and elsewhere in the UI. We were also using the old State.HasResources method as part of some of our tests. I considered preserving it to avoid changing the behavior of those tests, but the new check seemed close enough to the intent of those tests that it wasn't worth maintaining this method that wouldn't be used in any main code anymore. I've therefore updated those tests to use the new HasResourceInstanceObjects method instead.	4 years ago
Alisdair McDiarmid	b9f3dab035	json-output: Release format version 1.0	5 years ago
Omar Ismail	bea7e3ebce	Backend State Migration: change variable names from one/two to source/destination (#29699)	5 years ago
Martin Atkins	01b22f4b76	command/e2etest: TestProviderTampering ... We have various mechanisms that aim to ensure that the installed provider plugins are consistent with the lock file and that the lock file is consistent with the provider requirements, and we do have existing unit tests for them, but all of those cases mock our fake out at least part of the process and in the past that's caused us to miss usability regressions, where we still catch the error but do so at the wrong layer and thus generate error message lacking useful additional context. Here we'll add some new end-to-end tests to supplement the existing unit tests, making sure things work as expected when we assemble the system together as we would in a release. These tests cover a number of different ways in which the plugin selections can grow inconsistent. These new tests all run only when we're in a context where we're allowed to access the network, because they exercise the real plugin installer codepath. We could technically build this to use a local filesystem mirror or other such override to avoid that, but the point here is to make sure we see the expected behavior in the main case, and so it's worth the small additional cost of downloading the null provider from the real registry.	5 years ago
Martin Atkins	d09510a8fb	command: Early error message for missing cache entries of locked providers ... In the original incarnation of Meta.providerFactories we were returning into a Meta.contextOpts whose signature didn't allow it to return an error directly, and so we had compromised by making the provider factory functions themselves return errors once called. Subsequent work made Meta.contextOpts need to return an error anyway, but at the time we neglected to update our handling of the providerFactories result, having it still defer the error handling until we finally instantiate a provider. Although that did ultimately get the expected result anyway, the error ended up being reported from deep in the guts of a Terraform Core graph walk, in whichever concurrently-visited graph node happened to try to instantiate the plugin first. This meant that the exact phrasing of the error message would vary between runs and the reporting codepath didn't have enough context to given an actionable suggestion on how to proceed. In this commit we make Meta.contextOpts pass through directly any error that Meta.providerFactories produces, and then make Meta.providerFactories produce a special error type so that Meta.Backend can ultimately return a user-friendly diagnostic message containing a specific suggestion to run "terraform init", along with a short explanation of what a provider plugin is. The reliance here on an implied contract between two functions that are not directly connected in the callstack is non-ideal, and so hopefully we'll revisit this further in future work on the overall architecture of the CLI layer. To try to make this robust in the meantime though, I wrote it to use the errors.As function to potentially unwrap a wrapped version of our special error type, in case one of the intervening layers is changed at some point to wrap the downstream error before returning it.	5 years ago
Martin Atkins	df578afd7e	backend/local: Check dependency lock consistency before any operations ... In historical versions of Terraform the responsibility to check this was inside the terraform.NewContext function, along with various other assorted concerns that made that function particularly complicated. More recently, we reduced the responsibility of the "terraform" package only to instantiating particular named plugins, assuming that its caller is responsible for selecting appropriate versions of any providers that _are_ external. However, until this commit we were just assuming that "terraform init" had correctly selected appropriate plugins and recorded them in the lock file, and so nothing was dealing with the problem of ensuring that there haven't been any changes to the lock file or config since the most recent "terraform init" which would cause us to need to re-evaluate those decisions. Part of the game here is to slightly extend the role of the dependency locks object to also carry information about a subset of provider addresses whose lock entries we're intentionally disregarding as part of the various little edge-case features we have for overridding providers: dev_overrides, "unmanaged providers", and the testing overrides in our own unit tests. This is an in-memory-only annotation, never included in the serialized plan files on disk. I had originally intended to create a new package to encapsulate all of this plugin-selection logic, including both the version constraint checking here and also the handling of the provider factory functions, but as an interim step I've just made version constraint consistency checks the responsibility of the backend/local package, which means that we'll always catch problems as part of preparing for local operations, while not imposing these additional checks on commands that _don't_ run local operations, such as "terraform apply" when in remote operations mode.	5 years ago
Martin Atkins	6a98e4720c	plans/planfile: Create takes most arguments via a struct type ... Previously the planfile.Create function had accumulated probably already too many positional arguments, and I'm intending to add another one in a subsequent commit and so this is preparation to make the callsites more readable (subjectively) and make it clearer how we can extend this function's arguments to include further components in a plan file. There's no difference in observable functionality here. This is just passing the same set of arguments in a slightly different way.	5 years ago
Martin Atkins	8d193ad268	core: Simplify and centralize plugin availability checks ... Historically the responsibility for making sure that all of the available providers are of suitable versions and match the appropriate checksums has been split rather inexplicably over multiple different layers, with some of the checks happening as late as creating a terraform.Context. We're gradually iterating towards making that all be handled in one place, but in this step we're just cleaning up some old remnants from the main "terraform" package, which is now no longer responsible for any version or checksum verification and instead just assumes it's been provided with suitable factory functions by its caller. We do still have a pre-check here to make sure that we at least have a factory function for each plugin the configuration seems to depend on, because if we don't do that up front then it ends up getting caught instead deep inside the Terraform runtime, often inside a concurrent graph walk and thus it's not deterministic which codepath will happen to catch it on a particular run. As of this commit, this actually does leave some holes in our checks: the command package is using the dependency lock file to make sure we have exactly the provider packages we expect (exact versions and checksums), which is the most crucial part, but we don't yet have any spot where we make sure that the lock file is consistent with the current configuration, and we are no longer preserving the provider checksums as part of a saved plan. Both of those will come in subsequent commits. While it's unusual to have a series of commits that briefly subtracts functionality and then adds back in equivalent functionality later, the lock file checking is the only part that's crucial for security reasons, with everything else mainly just being to give better feedback when folks seem to be using Terraform incorrectly. The other bits are therefore mostly cosmetic and okay to be absent briefly as we work towards a better design that is clearer about where that responsibility belongs.	5 years ago
Zach Whaley	c9a5fdb366	cliconfig: Fix error message about invalid credentials helper type	5 years ago
James Bardin	ab0322e406	remove debugging println	5 years ago
James Bardin	c2e0d265cf	LoadModule now always returns the module ... We don't need to load the configuration twice, since configload can return the module for us.	5 years ago
James Bardin	a53faf43f6	return partial config from LoadConfig with errors ... LoadConfig should return any parsed configuration in order for the caller to verify `required_version`.	5 years ago
James Bardin	625e768678	make sure required_version is checked before diags ... We must ensure that the terraform required_version is checked as early as possible, so that new configuration constructs don't cause init to fail without indicating the version is incompatible. The loadConfig call before the earlyconfig parsing seems to be unneeded, and we can delay that to de-tangle it from installing the modules which may have their own constraints. TODO: it seems that loadConfig should be able to handle returning the version constraints in the same manner as loadSingleModule.	5 years ago
Alisdair McDiarmid	b699391d04	json-output: Add change reasons to explain deletes ... The extra feedback information for why resource instance deletion is planned is now included in the streaming JSON UI output. We also add an explicit case for no-op actions to switch statements in this package to ensure exhaustiveness, for future linting.	5 years ago
Martin Atkins	04f9e7148c	command/format: Include deletion reasons in plan report ... The core runtime is now able to specify a reason for some situations when Terraform plans to delete a resource instance. This commit makes that information visible in the human-oriented UI. A previous commit already made the underlying data informing these new hints visible as part of the machine-oriented (JSON) plan output. This also removes the bold formatting from the existing "has moved to" hints, because subjectively it seemed like the result was emphasizing too many parts of the output and thus somewhat defeating the benefit of the emphasis in trying to create additional visual hierarchy for sighted users running Terraform in a terminal. Now only the first line containing the main action statement will be in bold, and all of the parenthesized follow-up notes will be unformatted.	5 years ago
Martin Atkins	a1a713cf28	core: Report ActionReasons when we plan to delete "orphans" ... There are a few different reasons why a resource instance tracked in the prior state might be considered an "orphan", but previously we reported them all identically in the planned changes. In order to help users understand the reason for a surprising planned delete, we'll now try to specify an additional reason for the planned deletion, covering all of the main reasons why that could happen. This commit only introduces the new detail to the plans.Changes result, though it also incidentally exposes it as part of the JSON plan result in order to keep that working without returning errors in these new cases. We'll expose this information in the human-oriented UI output in a subsequent commit.	5 years ago
Chris Arcand	171cdbbf93	command: Clean up testInputResponseMap before failing on unused answers ... If you don't, the unused answers will persist in the package-level var and bleed in to other tests.	5 years ago
Chris Arcand	60bc7aa05d	command: Auto-select single workspace if necessary ... When initializing a backend, if the currently selected workspace does not exist, the user is prompted to select from the list of workspaces the backend provides. Instead, we should automatically select the only workspace available _if_ that's all that's there. Although with being a nice bit of polish, this enables future improvments with Terraform Cloud in potentially removing the implicit depenency on always using the 'default' workspace when the current configuration is mapped to a single TFC workspace.	5 years ago