92 Commits (f-debugger)

Author	SHA1	Message	Date
Alisdair McDiarmid	2d1976bbda	clistate: Update clistate.Locker for command views ... The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.	5 years ago
Alisdair McDiarmid	536c80da23	backend: Add per-operation diagnostic rendering ... The enhanced backends (local and remote) need to be able to render diagnostics during operations. Prior to this commit, this functionality was supported with a per-backend `ShowDiagnostics` function pointer. In order to allow users of these backends to control how diagnostics are rendered, this commit moves that function pointer to the `Operation` type. This means that a diagnostic renderer is configured for each operation, rather than once per backend initialization. Some secondary consequences of this change: - The `ReportResult` method on the backend is now moved to the `Operation` type, as it needs to access the `ShowDiagnostics` callback (and nothing else from the backend); - Tests which assumed that diagnostics would be written to the backend's `cli.Ui` instance are migrated to using a new record/playback diags helper function; - Apply, plan, and refresh commands now pass a pointer to the `Meta` struct's `showDiagnostics` method. This commit should not change how Terraform works, and is refactoring in preparation for more changes which move UI code out of the backend.	5 years ago
Jonathan Hall	49ee3d3ef8	Grammar nit: "setup" as a verb should be spelled "set up"	5 years ago
Alisdair McDiarmid	a15c5c84a3	Merge pull request #27498 from hashicorp/alisdair/fix-remote-terraform-version-check-again ... cli: Fix double check of remote workspace version	5 years ago
Martin Atkins	d2c3403ab6	command: Use the new terminal.Streams object ... Here we propagate in the initialized terminal.Streams from package main, and then onwards to backends running in CLI mode. This also replaces our use of helper/wrappedstreams to determine whether stdin is a terminal or a pipe. helper/wrappedstreams returns incorrect file descriptors on Windows, causing StdinPiped to always return false on that platform and thus causing one of the odd behaviors discussed in Finally, this includes some wrappers around the ability to look up the number of columns in the terminal in preparation for use elsewhere. These wrappers deal with the fact that our unit tests typically won't populate meta.Streams.	5 years ago
Alisdair McDiarmid	ddac282553	cli: Fix double check of remote workspace version ... After verifying the remote backend workspace version is compatible with the local Terraform version, we need to record that this check was successful. Otherwise the fallback error handling path will run a strict version check, even if the versions are compatible, which will cause an unexpected failure.	5 years ago
James Bardin	1c58c6ba48	command staticcheck	6 years ago
James Bardin	e998882824	remove legacy types from command package	6 years ago
Alisdair McDiarmid	c5c1f31db3	backend: Validate remote backend Terraform version ... When using the enhanced remote backend, a subset of all Terraform operations are supported. Of these, only plan and apply can be executed on the remote infrastructure (e.g. Terraform Cloud). Other operations run locally and use the remote backend for state storage. This causes problems when the local version of Terraform does not match the configured version from the remote workspace. If the two versions are incompatible, an `import` or `state mv` operation can cause the remote workspace to be unusable until a manual fix is applied. To prevent this from happening accidentally, this commit introduces a check that the local Terraform version and the configured remote workspace Terraform version are compatible. This check is skipped for commands which do not write state, and can also be disabled by the use of a new command-line flag, `-ignore-remote-version`. Terraform version compatibility is defined as: - For all releases before 0.14.0, local must exactly equal remote, as two different versions cannot share state; - 0.14.0 to 1.0.x are compatible, as we will not change the state version number until at least Terraform 1.1.0; - Versions after 1.1.0 must have the same major and minor versions, as we will not change the state version number in a patch release. If the two versions are incompatible, a diagnostic is displayed, advising that the error can be suppressed with `-ignore-remote-version`. When this flag is used, the diagnostic is still displayed, but as a warning instead of an error. Commands which will not write state can assert this fact by calling the helper `meta.ignoreRemoteBackendVersionConflict`, which will disable the checks. Those which can write state should instead call the helper `meta.remoteBackendVersionCheck`, which will return diagnostics for display. In addition to these explicit paths for managing the version check, we have an implicit check in the remote backend's state manager initialization method. Both of the above helpers will disable this check. This fallback is in place to ensure that future code paths which access state cannot accidentally skip the remote version check.	6 years ago
Kristin Laemmert	d2e999ba1f	remove unused code (#26503) ... * remove unused code I've removed the provider-specific code under registry, and unused nil backend, and replaced a call to helper from backend/oss (the other callers of that func are provisioners scheduled to be deprecated). I also removed the Dockerfile, as our build process uses a different file. Finally I removed the examples directory, which had outdated examples and links. There are better, actively maintained examples available. * command: remove various unused bits * test wasn't running * backend: remove unused err	6 years ago
Alisdair McDiarmid	b239570abb	command: Always validate workspace name ... The workspace name can be overridden by setting a TF_WORKSPACE environment variable. If this is done, we should still validate the resulting workspace name; otherwise, we could end up with an invalid and unselectable workspace. This change updates the Meta.Workspace function to return an error, and handles that error wherever necessary.	6 years ago
Kristin Laemmert	6621501ae3	state: remove deprecated state package (#25490) ... Most of the state package has been deprecated by the states package. This PR replaces all the references to the old state package that can be done simply - the low-hanging fruit. * states: move state.Locker to statemgr The state.Locker interface was a wrapper around a statemgr.Full, so moving this was relatively straightforward. * command: remove unnecessary use of state package for writing local terraform state files * move state.LocalState into terraform package state.LocalState is responsible for managing terraform.States, so it made sense (to me) to move it into the terraform package. * slight change of heart: move state.LocalState into clistate instead of terraform	6 years ago
Kristin Laemmert	4d8fde3d6f	command: use backend config from state when backend=false is used. (#23802) ... * command: use backend config from state when backend=false is used. When a user runs `terraform init --backend=false`, terraform should inspect the state for a previously-configured backend, and use that backend, ignoring any backend config in the current configuration. If no backend is configured or there is no state, return a local backend. Fixes #16593	6 years ago
Martin Atkins	d0f50ff83f	command: Don't treat config warnings as errors ... Meta.backendConfig was incorrectly treating the second return value from loadBackendConfig as if it were go "error" rather than tfdiags.Diagnostics, which in turn meant that it would treat warnings like errors. This had confusing results because it still returned that tfdiags.Diagnostics value in its own diagnostics return value, causing the caller to see warnings even though the backendConfig function had taken the error codepath.	7 years ago
Martin Atkins	39e609d5fd	vendor: switch to HCL 2.0 in the HCL repository ... Previously we were using the experimental HCL 2 repository, but now we'll shift over to the v2 import path within the main HCL repository as part of actually releasing HCL 2.0 as stable. This is a mechanical search/replace to the new import paths. It also switches to the v2.0.0 release of HCL, which includes some new code that Terraform didn't previously have but should not change any behavior that matters for Terraform's purposes. For the moment the experimental HCL2 repository is still an indirect dependency via terraform-config-inspect, so it remains in our go.sum and vendor directories for the moment. Because terraform-config-inspect uses a much smaller subset of the HCL2 functionality, this does still manage to prune the vendor directory a little. A subsequent release of terraform-config-inspect should allow us to completely remove that old repository in a future commit.	7 years ago
James Bardin	7f5ffbfe90	Merge pull request #21439 from hashicorp/jbardin/init-backend ... init always prompts for migration with most remote backend configurations	7 years ago
James Bardin	c017149b31	don't store prepared backend config ... The backend gets to "prepare" the configuration before Configure is called, in order to validate the values and insert defaults. We don't want to store this value in the "config state", because it will often not match the raw config after it is prepared, forcing unecessary backend migrations during init. Since PrepareConfig is always called before Configure, we can store the config value directly, and assume that it will be prepared in the same manner each time.	7 years ago
James Bardin	ee9a618369	don't migrate backend during init without override ... If the backend config hashes match during init, and there are no new backend override options, then we assume the existing config is OK. Since init should be idempotent, we should be able to run init with no options or config changes, and not effect the backends at all.	7 years ago
James Bardin	cefc927e48	failing test for backend re-init	7 years ago
Sander van Harmelen	d9ba0c69ea	Always try to select a workspace after initialization ... There are a number of use cases that can require a user to select a workspace after initializing Terraform. To make sure we cover all these use cases, we will always call the selectWorkspace method to verify a valid workspace is already selected or (if needed) offer to select one before moving on.	7 years ago
Sander van Harmelen	973e2a7cf9	core: add a context to the UIInput interface	7 years ago
James Bardin	2adf5801d9	don't panic of the users aborts backend input ... When the user aborts input, it may end up as an unknown value, which needs to be converted to null for PrepareConfig. Allow PrepareConfig to accept null config values in order to fill in missing defaults.	7 years ago
James Bardin	c814f2da37	Change backend.ValidateConfig to PrepareConfig ... This mirrors the change made for providers, so that default values can be inserted into the config by the backend implementation. This is only the interface and method name changes, it does not yet add any default values.	7 years ago
James Bardin	9667e06a03	decode backend hash as uint64 ... Older versions of terraform could save the backend hash number in a value larger than an int. While we could conditionally decode the state into an intermediary data structure for upgrade, or detect the specific decode error and modify the json, it seems simpler to just decode into the most flexible value for now, which is a uint64.	8 years ago
Martin Atkins	27abd9c6b8	command: Fix TestMetaBackend_localDoesNotDeleteLocal ... The changes to how we handle setting the state path on the local backend broke the heuristic we were using here for detecting migration from one local backend to another with the same state path, which would by default end up deleting the state altogether after migration. We now use the StatePaths method to do this, which takes into account both the default values and any settings that have been set. Additionally this addresses a flaw in the old method which could potentially have deleted all non-default workspace state files if the "path" setting were changed without also changing the "workspace_dir" setting. This new approach is conservative because it will preserve all of the files if any one overlaps.	8 years ago
Martin Atkins	be79bf0412	command: Fix TestPlan_outBackend ... In an earlier change we fixed the "backendFromConfig" codepath to be able to properly detect changes to the -backend-config arguments during "terraform init", but this detection is too strict for the normal case of running an operation in a previously-initialized directory. Before any of the recent changes, the logic here was to selectively update the hash to include -backend-config settings in the init case. Since that late hash recalculation was confusing, here we take the alternative path of using the hash only in the normal case and full value comparison in the init case. Treating both of these cases separately makes things marginally easier to follow here.	8 years ago
Martin Atkins	2b9f92be31	command: Partially fix TestMetaBackend_planLocalStatePath ... This test is testing some strange implementation details of the old local backend which do not hold with the new filesystem state manager. Specifically, it was expecting state to be read from the stateOutPath rather than the statePath, which makes no sense here because the backend is configured to read from the default terraform.tfstate file (which does not exist.) There is another problem with this test which will be addressed in a subsequent commit.	8 years ago
Martin Atkins	24046ab833	command: More TRACE logging for the Backend instantiation codepaths	8 years ago
Martin Atkins	c0b7f58143	command: Fix detection of necessary backend migration ... In the refactoring for new HCL this codepath stopped taking into account changes to the CLI -backend-config options when deciding if a backend migration is required. This restores that behavior in a different way than it used to be: rather than re-hashing the merged config and comparing the hashes, we instead just compare directly the configuration values, which must be exactly equal in order to skip migration. This change is covered by the test TestInit_inputFalse, although as of this commit it is still not passing due a downstream problem within the migration code itself.	8 years ago
Sander van Harmelen	5e9414bf03	backend/migrations: only select workspaces if supported ... If the backend does not support workspaces, we don't have to try to select a workspace and we should not return an error.	8 years ago
Martin Atkins	b5547f00f6	command: backendConfig must mutate its copy, not the original ... Here we were going to the trouble of copying the body so we could mutate it, but then ended up mutating the original anyway and then returning the unmodified copy. Whoops! This fix is verified by a number of "init" command tests that exercise the -backend-config option, including TestInit_backendConfigFile and several others whose names have the prefix TestInit_backendConfig .	8 years ago
Sander van Harmelen	52a1b22f7a	Implement the remote enhanced backend ... This is a refactored version of the `remote` backend that was initially added to Terraform v0.11.8 which should now be compatible with v0.12.0.	8 years ago
Sander van Harmelen	5944e8e34f	Fix the ability to ask for and save user input	8 years ago
Martin Atkins	541952bb8f	Revert some work that happened since v0.12-dev branched ... This work was done against APIs that were already changed in the branch before work began, and so it doesn't apply to the v0.12 development work. To allow v0.12 to merge down to master, we'll revert this work out for now and then re-introduce equivalent functionality in later commits that works against the new APIs.	8 years ago
Martin Atkins	e8240087fe	command: Populate backend configuration in plan files ... In previous work we didn't quite connect these dots. The connection here is sub-awesome since the existing interfaces here had some unfortunate assumptions that we'd like to move away from (like the idea of a "nil backend" implying the local backend) but we're accepting this for now to avoid another big round of refactoring. The main implication of this is that we will now always include a backend configuration in the plan, though it might just be a placeholder config for the local backend in the remaining cases where that's still implicitly set. Later we will change this so that there is no implicit local backend at all (terraform init is always required, _it_ will deal with setting implicitly setting the local backend when appropriate), which will allow us to rework this to be more straightforward and less "spooky".	8 years ago
Martin Atkins	cbc548eb36	command: Do CLI init of backend loaded from plan ... If we don't do this, we can't produce any output when applying a saved plan file. Here we also introduce a check to the local backend's ReportResult function so that it won't panic if CLI init is skipped, although that will no longer happen in the apply-from-file case due to the change described in the previous paragraph.	8 years ago
Martin Atkins	a3403f2766	terraform: Ugly huge change to weave in new State and Plan types ... Due to how often the state and plan types are referenced throughout Terraform, there isn't a great way to switch them out gradually. As a consequence, this huge commit gets us from the old world to a _compilable_ new world, but still has a large number of known test failures due to key functionality being stubbed out. The stubs here are for anything that interacts with providers, since we now need to do the follow-up work to similarly replace the old terraform.ResourceProvider interface with its replacement in the new "providers" package. That work, along with work to fix the remaining failing tests, will follow in subsequent commits. The aim here was to replace all references to terraform.State and its downstream types with states.State, terraform.Plan with plans.Plan, state.State with statemgr.State, and switch to the new implementations of the state and plan file formats. However, due to the number of times those types are used, this also ended up affecting numerous other parts of core such as terraform.Hook, the backend.Backend interface, and most of the CLI commands. Just as with 5861dbf3fc before, I apologize in advance to the person who inevitably just found this huge commit while spelunking through the commit history.	8 years ago
Martin Atkins	ebafa51723	command: Various updates for the new backend package API ... This is a rather-messy, complex change to get the "command" package building again against the new backend API that was updated for the new configuration loader. A lot of this is mechanical rewriting to the new API, but meta_config.go and meta_backend.go in particular saw some major changes to interface with the new loader APIs and to deal with the change in order of steps in the backend API.	8 years ago
Martin Atkins	5782357c28	backend: Update interface and implementations for new config loader ... The new config loader requires some steps to happen in a different order, particularly in regard to knowing the schema in order to decode the configuration. Here we lean directly on the configschema package, rather than on helper/schema.Backend as before, because it's generally sufficient for our needs here and this prepares us for the helper/schema package later moving out into its own repository to seed a "plugin SDK".	8 years ago
Sander van Harmelen	67db9da000	Add checks for all flags we currently don’t support ... For Plan only: -module-depth=n For Plan & Apply -parallelism=m -refresh=false -var “foo=bar” and -var-file=foo	8 years ago
Sander van Harmelen	1696ade924	backend/migrations: check all workspaces ... This commit fixes a bug that (in the case of the `local` backend) would only check if the selected workspace had a state when deciding to preform a migration. When the selected workspace didn’t have a state (but other existing workspace(s) did), the migration would not be preformed and the other workspaces would be ignored.	8 years ago
Sander van Harmelen	495d1ea350	Use New() instead of `once.Do(b.init)`	8 years ago
Sander van Harmelen	97d1c46602	Update the backend import names ... It’s a purely cosmetic change, but I find it easier to read them like this.	8 years ago
Kristin Laemmert	b8a1a77f6b	command: fix spelling in error message - fixes #18065 (#18241)	8 years ago
James Bardin	bdd475e149	use the new clistate.Locker in commands ... Use the new StateLocker field to provide a wrapper for locking the state during terraform.Context creation in the commands that directly manipulate the state.	8 years ago
James Bardin	e9a76808df	create clistate.Locker interface ... Simplify the use of clistate.Lock by creating a clistate.Locker instance, which stores the context of locking a state, to allow unlock to be called without knowledge of how the state was locked. This alows the backend code to bring the needed UI methods to the point where the state is locked, and still unlock the state from an outer scope. Provide a NoopLocker as well, so that callers can always call Unlock without verifying the status of the lock. Add the StateLocker field to the backend.Operation, so that the state lock can be carried between the different function scopes of the backend code. This will allow the backend context to lock the state before it's read, while allowing the different operations to unlock the state when they complete.	8 years ago
Masayuki Morita	f440dba137	Standardize on log level "WARN" rather than "WARNING"	8 years ago
James Bardin	f45205feb7	re-word state migration prompts ... The existing prompts were worded as if backend configurations were named, but they can only really be referenced by their type. Change the wording to reference them as type "X backend". When migrating state, refer to the backends as the "previously configured" and "newly configured", since they will often have the same type.	8 years ago
James Bardin	7c93b2e5e6	remove duplicate backend migration prompts ... The duplicate prompts can be confusing when the user confirms that a migration should happen and we immediately prompt a second time for the same thing with slightly different wording. The extra hand-holding that this provides for legacy remote states is less critical now, since it's been 2 major release cycles since those were removed.	9 years ago
Martin Atkins	0fe43c8977	cli: allow disabling "next steps" message in terraform plan ... In #15884 we adjusted the plan output to give an explicit command to run to apply a plan, whereas before this command was just alluded to in the prose. Since releasing that, we've got good feedback that it's confusing to include such instructions when Terraform is running in a workflow automation tool, because such tools usually abstract away exactly what commands are run and require users to take different actions to proceed through the workflow. To accommodate such environments while retaining helpful messages for normal CLI usage, here we introduce a new environment variable TF_IN_AUTOMATION which, when set to a non-empty value, is a hint to Terraform that it isn't being run in an interactive command shell and it should thus tone down the "next steps" messaging. The documentation for this setting is included as part of the "...in automation" guide since it's not generally useful in other cases. We also intentionally disclaim comprehensive support for this since we want to avoid creating an extreme number of "if running in automation..." codepaths that would increase the testing matrix and hurt maintainability. The focus is specifically on the output of the three commands we give in the automation guide, which at present means the following two situations: * "terraform init" does not include the final paragraphs that suggest running "terraform plan" and tell you in what situations you might need to re-run "terraform init". * "terraform plan" does not include the final paragraphs that either warn about not specifying "-out=..." or instruct to run "terraform apply" with the generated plan file.	9 years ago