108 Commits (3c8a4e6e05afefb7639bf30efa852cf59f14ade6)

Author	SHA1	Message	Date
Martin Atkins	3c8a4e6e05	command+backend/local: -refresh-only and drift detection ... This is a light revamp of our plan output to make use of Terraform core's new ability to report both the previous run state and the refreshed state, allowing us to explicitly report changes made outside of Terraform. Because whether a plan has "changes" or not is no longer such a straightforward matter, this now merges views.Operation.Plan with views.Operation.PlanNoChanges to produce a single function that knows how to report all of the various permutations. This was also an opportunity to fill some holes in our previous logic which caused it to produce some confusing messages, including a new tailored message for when "terraform destroy" detects that nothing needs to be destroyed. This also allows users to request the refresh-only planning mode using a new -refresh-only command line option. In that case, Terraform _only_ performs drift detection, and so applying a refresh-only plan only involves writing a new state snapshot, without changing any real infrastructure objects.	5 years ago
Martin Atkins	42e0985839	command: use -lock=false consistently in -help output ... Previously the docs for this were rather confusing because they showed an option to turn _on_ state locking, even though it's on by default. Instead, we'll now show -lock=false in all cases and document it as _disabling_ the default locking. While working on this I also noticed that the equivalent docs on the website were differently inconsistent. I've not made them fully consistent here but at least moreso than they were before.	5 years ago
Martin Atkins	b402fd9d3a	command/views: Remove command-specific runningInAutomation ... We now have RunningInAutomation has a general concern in views.View, so we no longer need to specify it for each command-specific constructor separately. For this initial change I focused only on changing the exported interface of the views package and let the command-specific views go on having their own unexported fields containing a copy of the flag because it made this change less invasive and I wasn't feeling sure yet about whether we ought to have code within command-specific views directly access the internals of views.View. However, maybe we'll simplify this further in a later commit if we conclude that these copies of the flag are burdensome. The general version of this gets set directly inside the main package, which might at some future point allow us to make the command package itself unaware of this "running in automation" idea and thus reinforce that it's intended as a presentation-only thing rather than as a behavioral thing, but we'll save more invasive refactoring for another day.	5 years ago
Martin Atkins	1d3e34e35e	command: New -replace=... planning option ... This allows a similar effect to pre-tainting an object but does the action within the context of a normal plan and apply, avoiding the need for an intermediate state where the old object still exists but is marked as tainted. The core functionality for this was already present, so this commit is just the UI-level changes to make that option available for use and to explain how it contributed to the resulting plan in Terraform's output.	5 years ago
Martin Atkins	89f986ded6	command+backend: generalized "plan mode" ... So far we've only had "normal mode" and "destroy mode", where the latter is activated either by "terraform plan -destroy" or "terraform destroy". In preparation for introducing a third mode "refresh only" this generalizes how we handle modes so we can potentially deal with an arbitrary number of modes, although for now we only intend to have three. Mostly this is just a different implementation of the same old behavior, but there is one small user-visible difference here: the "terraform apply" command now accepts a -destroy option, mirroring the option of the same name on "terraform plan", which in turn makes "terraform destroy" effectively a shorthand for "terraform apply -destroy". This is intended to make us consistent that "terraform apply" without a plan file argument accepts all of the same plan-customization options that "terraform plan" does, which will in turn avoid us having to add a new alias of "terraform plan" for each new plan mode we might add. The -help output is changed in that vein here, although we'll wait for subsequent commit to make a similar change to the website documentation just so we can deal with the "refresh only mode" docs at the same time.	5 years ago
Alisdair McDiarmid	fad305f884	cli: Fix remote backend UI issues ... Fix two bugs which surface when using the remote backend: - When migrating to views, we removed the call to `(*Meta).process` which initialized the color boolean. This resulted in the legacy UI calls in the remote backend stripping color codes. To fix this, we populate this boolean from the common arguments. - Remote apply will output the resource summary and output changes, and these are rendered via the remote backend streaming. We need to special case this in the apply command and prevent displaying a zero-change summary line. Neither of these are coverable by automated tests, as we don't have any command-package level testing for the remote backend. Manually verified.	5 years ago
Martin Atkins	6f35c2847b	command: Reorganize docs of the local backend's legacy CLI options ... We have these funny extra options that date back to before Terraform even had remote state, which we've preserved along the way by most recently incorporating them as special-case overrides for the local backend. The documentation we had for these has grown less accurate over time as the details have shifted, and was in many cases missing the requisite caveats that they are only for the local backend and that backend configuration is the modern, preferred way to deal with the use-cases they were intended for. We always have a bit of a tension with this sort of legacy option because we want to keep them documented just enough to be useful to someone who finds an existing script/etc using them and wants to know what they do, but not to take up so much space that they might distract users from finding the modern alternative they should consider instead. As a compromise in that vein here I've created a new section about these options under the local backend documentation, which then gives us the space to go into some detail about the various behaviors and interactions and also to discuss their history and our recommended alternatives. I then simplified all of the other mentions of these in command documentation to just link to or refer to the local backend documentation. My hope then is that folks who need to know what these do can still find the docs, but that information can be kept out of the direct path of new users so they can focus on learning about remote backends instead. This is certainly not the most ideal thing ever, but it seemed like the best compromise between the competing priorities I described above.	5 years ago
Alisdair McDiarmid	86495f93cb	views: Fix missing source in diagnostic output ... The previous implementation of views was copying and embedding the base View struct in each individual view. While this allowed for easy access to the interface of that struct (both in the view and externally), it more importantly completely broke the ability of the diagnostic printer to output source code snippets. This is because the `configSources` field on the base view is lazily set after the config loader is initialized. In the commands ported to use views, this happens after the base View struct is copied, so we are updating the wrong copy of the struct. This commit fixes this with a simple mechanical refactor: keep a pointer to the base View struct instead, and update all of the individual views to explicitly refer to that struct to access its fields and methods. This is not a particularly satisfying solution, but I can't find anything clearly better. It might be worth exploring the alternative approach in the view for the new test command, which explicitly pulls its dependencies out of the base view, rather than retaining a full reference. Maybe there's a third way which is better still.	5 years ago
Alisdair McDiarmid	7c0ec0107f	backend: Replace ShowDiagnostics with view.Diagnostics	5 years ago
Alisdair McDiarmid	7a03c3153e	Merge pull request #27861 from hashicorp/alisdair/help-global-options ... cli: Add reference to global options to help text	5 years ago
Alisdair McDiarmid	8d9a08e3a1	cli: Migrate plan to command views	5 years ago
Alisdair McDiarmid	94340b5940	cli: Add reference to global options to help text	5 years ago
Alisdair McDiarmid	b976178018	cli: Remove no-op plan backend setup ... This code does not appear to have any effect. The operation request has its PlanOutBackend field populated directly in the Meta.Operation method, by calling m.backendForState.ForPlan(). I tested a trivial null-resource config with a Consul backend, and the saved plans with and without this code present were identical.	5 years ago
Alisdair McDiarmid	68558ccd54	backend/local: Replace CLI with view instance ... This commit extracts the remaining UI logic from the local backend, and removes access to the direct CLI output. This is replaced with an instance of a `views.Operation` interface, which codifies the current requirements for the local backend to interact with the user. The exception to this at present is interactivity: approving a plan still depends on the `UIIn` field for the backend. This is out of scope for this commit and can be revisited separately, at which time the `UIOut` field can also be removed. Changes in support of this: - Some instances of direct error output have been replaced with diagnostics, most notably in the emergency state backup handler. This requires reformatting the error messages to allow the diagnostic renderer to line-wrap them; - The "in-automation" logic has moved out of the backend and into the view implementation; - The plan, apply, refresh, and import commands instantiate a view and set it on the `backend.Operation` struct, as these are the only code paths which call the `local.Operation()` method that requires it; - The show command requires the plan rendering code which is now in the views package, so there is a stub implementation of a `views.Show` interface there. Other refactoring work in support of migrating these commands to the common views code structure will come in follow-up PRs, at which point we will be able to remove the UI instances from the unit tests for those commands.	5 years ago
Alisdair McDiarmid	a7b7cd29fc	cli: Migrate Terraform UI hook to command views ... Move the code which renders Terraform hook callbacks as UI into the views package, backed by a views.View instead of a cli.Ui. Update test setup accordingly. To allow commands to control this hook, we add a hooks member on the backend Operation struct. This supersedes the hooks in the Terraform context, which is not directly controlled by the command logic. This commit should not change how Terraform works, and is refactoring in preparation for more changes which move UI code out of the backend.	5 years ago
Alisdair McDiarmid	536c80da23	backend: Add per-operation diagnostic rendering ... The enhanced backends (local and remote) need to be able to render diagnostics during operations. Prior to this commit, this functionality was supported with a per-backend `ShowDiagnostics` function pointer. In order to allow users of these backends to control how diagnostics are rendered, this commit moves that function pointer to the `Operation` type. This means that a diagnostic renderer is configured for each operation, rather than once per backend initialization. Some secondary consequences of this change: - The `ReportResult` method on the backend is now moved to the `Operation` type, as it needs to access the `ShowDiagnostics` callback (and nothing else from the backend); - Tests which assumed that diagnostics would be written to the backend's `cli.Ui` instance are migrated to using a new record/playback diags helper function; - Apply, plan, and refresh commands now pass a pointer to the `Meta` struct's `showDiagnostics` method. This commit should not change how Terraform works, and is refactoring in preparation for more changes which move UI code out of the backend.	5 years ago
Alisdair McDiarmid	4991cc4835	cli: Improve error for invalid -target flags ... Errors encountered when parsing flags for apply, plan, and refresh were being suppressed. This resulted in a generic usage error when using an invalid `-target` flag. This commit makes several changes to address this. First, these commands now output the flag parse error before exiting, leaving at least some hint about the error. You can verify this manually with something like: terraform apply -invalid-flag We also change how target attributes are parsed, moving the responsibility from the flags instance to the command. This allows us to customize the diagnostic output to be more user friendly. The diagnostics now look like: ```shellsession $ terraform apply -no-color -target=foo Error: Invalid target "foo" Resource specification must include a resource type and name. ``` Finally, we add test coverage for both parsing of target flags, and at the command level for successful use of resource targeting. These tests focus on the UI output (via the change summary and refresh logs), as the functionality of targeting is covered by the context tests in the terraform package.	5 years ago
Alisdair McDiarmid	ca23a096d8	cli: Remove legacy positional path arguments ... Several commands continued to support the legacy positional path argument to specify a working directory. This functionality has been replaced with the global -chdir flag, which is specified before any other arguments, including the sub-command name. This commit removes support for the trailing path parameter from most commands. The only command which still supports a path argument is fmt, which also supports "-" to indicate receiving configuration from standard input. Any invocation of a command with an invalid trailing path parameter will result in a short error message, pointing at the -chdir alternative. There are many test updates in this commit, almost all of which are migrations from using positional arguments to specify a working directory. Because of the layer at which these tests run, we are unable to use the -chdir argument, so the churn in test files is larger than ideal. Sorry!	5 years ago
Martin Atkins	c94a6102df	command: Improve consistency of the command short descriptions ... The short description of our commands (as shown in the main help output from "terraform") was previously very inconsistent, using different tense/mood for different commands. Some of the commands were also using some terminology choices inconsistent with how we currently talk about the related ideas in our documentation. Here I've tried to add some consistency by first rewriting them all in the imperative mood (except the ones that just are just subcommand groupings), and tweaking some of the terminology to hopefully gel better with how we present similar ideas in our recently-updated docs. While working on this I inevitably spotted some similar inconsistencies in the longer-form help output of some of the commands. I've not reviewed all of these for consistency, but I did update some where the wording was either left inconsstent with the short form changes I'd made or where the prose stood out to me as particularly inconsistent with our current usual documentation language style. All of this is subjective, so I expect we'll continue to tweak these over time as we continue to develop our documentation writing style based on user questions and feedback.	6 years ago
Alisdair McDiarmid	b239570abb	command: Always validate workspace name ... The workspace name can be overridden by setting a TF_WORKSPACE environment variable. If this is done, we should still validate the resulting workspace name; otherwise, we could end up with an invalid and unselectable workspace. This change updates the Meta.Workspace function to return an error, and handles that error wherever necessary.	6 years ago
Alisdair McDiarmid	67203dade8	command: Simplify Meta.process helper method ... After some refactoring, this helper method had an unused argument (vars) and an always-nil error return value. This commit cleans this up.	6 years ago
Martin Atkins	c06675c616	command: New -compact-warnings option ... When warnings appear in isolation (not accompanied by an error) it's reasonable to want to defer resolving them for a while because they are not actually blocking immediate work. However, our warning messages tend to be long by default in order to include all of the necessary context to understand the implications of the warning, and that can make them overwhelming when combined with other output. As a compromise, this adds a new CLI option -compact-warnings which is supported for all the main operation commands and which uses a more compact format to print out warnings as long as they aren't also accompanied by errors. The default remains unchanged except that the threshold for consolidating warning messages is reduced to one so that we'll now only show one of each distinct warning summary. Full warning messages are always shown if there's at least one error included in the diagnostic set too, because in that case the warning message could contain additional context to help understand the error.	6 years ago
Jeff Green	e505845a63	command/apply: remove duplicate statement (#23011)	7 years ago
Kristin Laemmert	21f6e3dffd	Mildwonkey/012 docs updated (#20542) ... * docs: update plan command documentation. Fixes #19235 * docs: added a missing reserved variable name. Fixes #19159. * website: add note that resource names cannot start with a number * website: add some notes to the 0.12 upgrade guide	7 years ago
Sander van Harmelen	ef9054562e	commands: make sure the correct flagset is used ... A lot of commands used `c.Meta.flagSet()` to create the initial flagset for the command, while quite a few of them didn’t actually use or support the flags that are then added. So I updated a few commands to use `flag.NewFlagSet()` instead to only add the flags that are actually needed/supported. Additionally this prevents a few commands from using locking while they actually don’t need locking (as locking is enabled as a default in `c.Meta.flagSet()`.	7 years ago
Sander van Harmelen	52a1b22f7a	Implement the remote enhanced backend ... This is a refactored version of the `remote` backend that was initially added to Terraform v0.11.8 which should now be compatible with v0.12.0.	8 years ago
Sander van Harmelen	178ec8f7b4	Remove support for the -module-depth flag ... # Conflicts: # backend/backend.go	8 years ago
Martin Atkins	de3944b9bf	command: collect root module variable values for apply/plan/refresh ... This connects a missing link left by earlier refactoring: the command package is responsible for gathering up variable values provided by the user and passing them through to the backend to use in operations.	8 years ago
Martin Atkins	55b9c7eacd	command: Actually include the error when backend config decoding fails	8 years ago
Martin Atkins	e8240087fe	command: Populate backend configuration in plan files ... In previous work we didn't quite connect these dots. The connection here is sub-awesome since the existing interfaces here had some unfortunate assumptions that we'd like to move away from (like the idea of a "nil backend" implying the local backend) but we're accepting this for now to avoid another big round of refactoring. The main implication of this is that we will now always include a backend configuration in the plan, though it might just be a placeholder config for the local backend in the remaining cases where that's still implicitly set. Later we will change this so that there is no implicit local backend at all (terraform init is always required, _it_ will deal with setting implicitly setting the local backend when appropriate), which will allow us to rework this to be more straightforward and less "spooky".	8 years ago
Martin Atkins	a3403f2766	terraform: Ugly huge change to weave in new State and Plan types ... Due to how often the state and plan types are referenced throughout Terraform, there isn't a great way to switch them out gradually. As a consequence, this huge commit gets us from the old world to a _compilable_ new world, but still has a large number of known test failures due to key functionality being stubbed out. The stubs here are for anything that interacts with providers, since we now need to do the follow-up work to similarly replace the old terraform.ResourceProvider interface with its replacement in the new "providers" package. That work, along with work to fix the remaining failing tests, will follow in subsequent commits. The aim here was to replace all references to terraform.State and its downstream types with states.State, terraform.Plan with plans.Plan, state.State with statemgr.State, and switch to the new implementations of the state and plan file formats. However, due to the number of times those types are used, this also ended up affecting numerous other parts of core such as terraform.Hook, the backend.Backend interface, and most of the CLI commands. Just as with 5861dbf3fc before, I apologize in advance to the person who inevitably just found this huge commit while spelunking through the commit history.	8 years ago
Martin Atkins	ebafa51723	command: Various updates for the new backend package API ... This is a rather-messy, complex change to get the "command" package building again against the new backend API that was updated for the new configuration loader. A lot of this is mechanical rewriting to the new API, but meta_config.go and meta_backend.go in particular saw some major changes to interface with the new loader APIs and to deal with the change in order of steps in the backend API.	8 years ago
Sander van Harmelen	b1fdbd7db8	Allow enhanced backends to pass custom exit codes ... In some cases this is needed to keep the UX clean and to make sure any remote exit codes are passed through to the local process. The most obvious example for this is when using the "remote" backend. This backend runs Terraform remotely and stream the output back to the local terminal. When an error occurs during the remote execution, all the needed error information will already be in the streamed output. So if we then return an error ourselves, users will get the same errors twice. By allowing the backend to specify the correct exit code, the UX remains the same while preserving the correct exit codes.	8 years ago
Sander van Harmelen	67db9da000	Add checks for all flags we currently don’t support ... For Plan only: -module-depth=n For Plan & Apply -parallelism=m -refresh=false -var “foo=bar” and -var-file=foo	8 years ago
James Bardin	67a6152091	move backend operation cancellation into meta ... Create a single command method for running and operation with cancellation.	8 years ago
James Bardin	7cba68326a	always wait for a RunningOperation to return ... If the user wishes to interrupt the running operation, only the first interrupt was communicated to the operation by canceling the provided context. A second interrupt would start the shutdown process, but not communicate this to the running operation. This order of event could cause partial writes of state. What would happen is that once the command returns, the plugin system would stop the provider processes. Once the provider processes dies, all pending Eval operations would return return with an error, and quickly cause the operation to complete. Since the backend code didn't know that the process was shutting down imminently, it would continue by attempting to write out the last known state. Under the right conditions, the process would exit part way through the writing of the state file. Add Stop and Cancel CancelFuncs to the RunningOperation, to allow it to easily differentiate between the two signals. The backend will then be able to detect a shutdown and abort more gracefully. In order to ensure that the backend is not in the process of writing the state out, the command will always attempt to wait for the process to complete after cancellation.	8 years ago
Martin Atkins	9a5c865040	command: validate config as part of loading it ... Previously we required callers to separately call .Validate on the root module to determine if there were any value errors, but we did that inconsistently and would thus see crashes in some cases where later code would try to use invalid configuration as if it were valid. Now we run .Validate automatically after config loading, returning the resulting diagnostics. Since we return a diagnostics here, it's possible to return both warnings and errors. We return the loaded module even if it's invalid, so callers are free to ignore returned errors and try to work with the config anyway, though they will need to be defensive against invalid configuration themselves in that case. As a result of this, all of the commands that load configuration now need to use diagnostic printing to signal errors. For the moment this just allows us to return potentially-multiple config errors/warnings in full fidelity, but also sets us up for later when more subsystems are able to produce rich diagnostics so we can show them all together. Finally, this commit also removes some stale, commented-out code for the "legacy" (pre-0.8) graph implementation, which has not been available for some time.	8 years ago
James Bardin	2941ed464c	replace the testShutdownHook with a check for Stop ... Now that the local backend can be cancelled during plan and refresh, we don't really need the testShutdownHook. Simplify the tests by just checking for Stop being called on the provider.	8 years ago
James Bardin	e2501d7830	make apply shutdown test completely deterministic ... Add a shutdown hook to verify that a context has been correctly cancelled, so we can remove the sleep and stop guessing. Add a plan version of the shutdown test as well.	8 years ago
James Bardin	3aaa1e9d04	make plans cancellable ... There was no cancellation context for a plan, so it would always have to run to completion as SIGINT was being swallowed. Move the shutdown channel to the command Meta since it's used in multiple commands.	8 years ago
Martin Atkins	6f7bc4f5d7	command: use c.showDiagnostics for backend operation errors ... This allows richer diagnostics produced by some subsystems to be displayed in full-fidelity to the user.	9 years ago
Martin Atkins	5cd00a13ec	command: use new diagnostics output for config errors ... This uses the new diagnostics printer for config-related errors in the main five commands that deal with config. The immediate motivation for this is to allow HCL2-produced diagnostics to be printed out in their full fidelity, though it also slightly changes the presentation of other errors so that they are not presented in all red text, which can be hard to read on some terminals.	9 years ago
Robert Liebowitz	8d98fdecac	Autoload only .auto.tfvars files	9 years ago
Robert Liebowitz	006744bfe0	Use all tfvars files in working directory ... As a side effect, several commands that previously did not have a failure state can now fail during meta-parameter processing.	9 years ago
James Bardin	000e860706	Add plugin dir scaffolding ... add pluginDir to command.Meta, the flag to initialize it, and the methods to save and restore it.	9 years ago
James Bardin	718ede0636	have Meta.Backend use a Config rather than loading ... Instead of providing the a path in BackendOpts, provide a loaded *config.Config instead. This reduces the number of places where configuration is loaded.	9 years ago
James Bardin	5eca913b14	add cli flags for -lock-timeout ... Add the -lock-timeout flag to the appropriate commands. Add the -lock flag to `init` and `import` which were missing it. Set both stateLock and stateLockTimeout in Meta.flagsSet, and remove the extra references for clarity.	9 years ago
James Bardin	eb8e5ac739	Change CLI flag to '-lock'	9 years ago
James Bardin	07903189f1	s/Meta.lockState/Meta.stateLock/g	9 years ago
James Bardin	a157ebbccd	add -lock-state usage to plan/refresh/apply/destr	9 years ago