74 Commits (3679de0630ffbe67fa8d2ff299891dfdcb01cd6a)

Author	SHA1	Message	Date
Martin Atkins	6f35c2847b	command: Reorganize docs of the local backend's legacy CLI options ... We have these funny extra options that date back to before Terraform even had remote state, which we've preserved along the way by most recently incorporating them as special-case overrides for the local backend. The documentation we had for these has grown less accurate over time as the details have shifted, and was in many cases missing the requisite caveats that they are only for the local backend and that backend configuration is the modern, preferred way to deal with the use-cases they were intended for. We always have a bit of a tension with this sort of legacy option because we want to keep them documented just enough to be useful to someone who finds an existing script/etc using them and wants to know what they do, but not to take up so much space that they might distract users from finding the modern alternative they should consider instead. As a compromise in that vein here I've created a new section about these options under the local backend documentation, which then gives us the space to go into some detail about the various behaviors and interactions and also to discuss their history and our recommended alternatives. I then simplified all of the other mentions of these in command documentation to just link to or refer to the local backend documentation. My hope then is that folks who need to know what these do can still find the docs, but that information can be kept out of the direct path of new users so they can focus on learning about remote backends instead. This is certainly not the most ideal thing ever, but it seemed like the best compromise between the competing priorities I described above.	5 years ago
Alisdair McDiarmid	86495f93cb	views: Fix missing source in diagnostic output ... The previous implementation of views was copying and embedding the base View struct in each individual view. While this allowed for easy access to the interface of that struct (both in the view and externally), it more importantly completely broke the ability of the diagnostic printer to output source code snippets. This is because the `configSources` field on the base view is lazily set after the config loader is initialized. In the commands ported to use views, this happens after the base View struct is copied, so we are updating the wrong copy of the struct. This commit fixes this with a simple mechanical refactor: keep a pointer to the base View struct instead, and update all of the individual views to explicitly refer to that struct to access its fields and methods. This is not a particularly satisfying solution, but I can't find anything clearly better. It might be worth exploring the alternative approach in the view for the new test command, which explicitly pulls its dependencies out of the base view, rather than retaining a full reference. Maybe there's a third way which is better still.	5 years ago
Alisdair McDiarmid	7c0ec0107f	backend: Replace ShowDiagnostics with view.Diagnostics	5 years ago
Alisdair McDiarmid	7a03c3153e	Merge pull request #27861 from hashicorp/alisdair/help-global-options ... cli: Add reference to global options to help text	5 years ago
Alisdair McDiarmid	50d58b592a	cli: Migrate refresh to command views	5 years ago
Alisdair McDiarmid	94340b5940	cli: Add reference to global options to help text	5 years ago
Alisdair McDiarmid	68558ccd54	backend/local: Replace CLI with view instance ... This commit extracts the remaining UI logic from the local backend, and removes access to the direct CLI output. This is replaced with an instance of a `views.Operation` interface, which codifies the current requirements for the local backend to interact with the user. The exception to this at present is interactivity: approving a plan still depends on the `UIIn` field for the backend. This is out of scope for this commit and can be revisited separately, at which time the `UIOut` field can also be removed. Changes in support of this: - Some instances of direct error output have been replaced with diagnostics, most notably in the emergency state backup handler. This requires reformatting the error messages to allow the diagnostic renderer to line-wrap them; - The "in-automation" logic has moved out of the backend and into the view implementation; - The plan, apply, refresh, and import commands instantiate a view and set it on the `backend.Operation` struct, as these are the only code paths which call the `local.Operation()` method that requires it; - The show command requires the plan rendering code which is now in the views package, so there is a stub implementation of a `views.Show` interface there. Other refactoring work in support of migrating these commands to the common views code structure will come in follow-up PRs, at which point we will be able to remove the UI instances from the unit tests for those commands.	5 years ago
Alisdair McDiarmid	a7b7cd29fc	cli: Migrate Terraform UI hook to command views ... Move the code which renders Terraform hook callbacks as UI into the views package, backed by a views.View instead of a cli.Ui. Update test setup accordingly. To allow commands to control this hook, we add a hooks member on the backend Operation struct. This supersedes the hooks in the Terraform context, which is not directly controlled by the command logic. This commit should not change how Terraform works, and is refactoring in preparation for more changes which move UI code out of the backend.	5 years ago
Alisdair McDiarmid	536c80da23	backend: Add per-operation diagnostic rendering ... The enhanced backends (local and remote) need to be able to render diagnostics during operations. Prior to this commit, this functionality was supported with a per-backend `ShowDiagnostics` function pointer. In order to allow users of these backends to control how diagnostics are rendered, this commit moves that function pointer to the `Operation` type. This means that a diagnostic renderer is configured for each operation, rather than once per backend initialization. Some secondary consequences of this change: - The `ReportResult` method on the backend is now moved to the `Operation` type, as it needs to access the `ShowDiagnostics` callback (and nothing else from the backend); - Tests which assumed that diagnostics would be written to the backend's `cli.Ui` instance are migrated to using a new record/playback diags helper function; - Apply, plan, and refresh commands now pass a pointer to the `Meta` struct's `showDiagnostics` method. This commit should not change how Terraform works, and is refactoring in preparation for more changes which move UI code out of the backend.	5 years ago
Alisdair McDiarmid	c5a6aa31d3	cli: Add initial command views abstraction ... Terraform supports multiple output formats for several sub-commands. The default format is user-readable text, but many sub-commands support a `-json` flag to output a machine-readable format for the result. The output command also supports a `-raw` flag for a simpler, scripting- focused machine readable format. This commit adds a "views" abstraction, intended to help ensure consistency between the various output formats. This extracts the render specific code from the command package, and moves it into a views package. Each command is expected to create an interface for its view, and one or more implementations of that interface. By doing so, we separate the concerns of generating the sub-command result from rendering the result in the specified output format. This should make it easier to ensure that all output formats will be updated together when changes occur in the result-generating phase. There are some other consequences of this restructuring: - Views now directly access the terminal streams, rather than the now-redundant cli.Ui instance; - With the reorganization of commands, parsing CLI arguments is now the responsibility of a separate "arguments" package. For now, views are added only for the output sub-command, as an example. Because this command uses code which is shared with the apply and refresh commands, those are also partially updated.	5 years ago
Alisdair McDiarmid	4991cc4835	cli: Improve error for invalid -target flags ... Errors encountered when parsing flags for apply, plan, and refresh were being suppressed. This resulted in a generic usage error when using an invalid `-target` flag. This commit makes several changes to address this. First, these commands now output the flag parse error before exiting, leaving at least some hint about the error. You can verify this manually with something like: terraform apply -invalid-flag We also change how target attributes are parsed, moving the responsibility from the flags instance to the command. This allows us to customize the diagnostic output to be more user friendly. The diagnostics now look like: ```shellsession $ terraform apply -no-color -target=foo Error: Invalid target "foo" Resource specification must include a resource type and name. ``` Finally, we add test coverage for both parsing of target flags, and at the command level for successful use of resource targeting. These tests focus on the UI output (via the change summary and refresh logs), as the functionality of targeting is covered by the context tests in the terraform package.	5 years ago
Alisdair McDiarmid	ca23a096d8	cli: Remove legacy positional path arguments ... Several commands continued to support the legacy positional path argument to specify a working directory. This functionality has been replaced with the global -chdir flag, which is specified before any other arguments, including the sub-command name. This commit removes support for the trailing path parameter from most commands. The only command which still supports a path argument is fmt, which also supports "-" to indicate receiving configuration from standard input. Any invocation of a command with an invalid trailing path parameter will result in a short error message, pointing at the -chdir alternative. There are many test updates in this commit, almost all of which are migrations from using positional arguments to specify a working directory. Because of the layer at which these tests run, we are unable to use the -chdir argument, so the churn in test files is larger than ideal. Sorry!	5 years ago
Alisdair McDiarmid	2e62e03b64	cli: Remove dead code for output -module flag ... The -module flag to terraform output has been unimplemented since 0.12. This commit removes some dead code and the specific error message for this flag. The website documentation for output does not mention this flag, so it is unchanged.	5 years ago
Martin Atkins	c94a6102df	command: Improve consistency of the command short descriptions ... The short description of our commands (as shown in the main help output from "terraform") was previously very inconsistent, using different tense/mood for different commands. Some of the commands were also using some terminology choices inconsistent with how we currently talk about the related ideas in our documentation. Here I've tried to add some consistency by first rewriting them all in the imperative mood (except the ones that just are just subcommand groupings), and tweaking some of the terminology to hopefully gel better with how we present similar ideas in our recently-updated docs. While working on this I inevitably spotted some similar inconsistencies in the longer-form help output of some of the commands. I've not reviewed all of these for consistency, but I did update some where the wording was either left inconsstent with the short form changes I'd made or where the prose stood out to me as particularly inconsistent with our current usual documentation language style. All of this is subjective, so I expect we'll continue to tweak these over time as we continue to develop our documentation writing style based on user questions and feedback.	5 years ago
Shunsuke Miyoshi	dc3ce1bbf7	command/refresh: fix state file path (#22885) ... * fix default state file path for refresh command	6 years ago
Alisdair McDiarmid	67203dade8	command: Simplify Meta.process helper method ... After some refactoring, this helper method had an unused argument (vars) and an always-nil error return value. This commit cleans this up.	6 years ago
Martin Atkins	c06675c616	command: New -compact-warnings option ... When warnings appear in isolation (not accompanied by an error) it's reasonable to want to defer resolving them for a while because they are not actually blocking immediate work. However, our warning messages tend to be long by default in order to include all of the necessary context to understand the implications of the warning, and that can make them overwhelming when combined with other output. As a compromise, this adds a new CLI option -compact-warnings which is supported for all the main operation commands and which uses a more compact format to print out warnings as long as they aren't also accompanied by errors. The default remains unchanged except that the threshold for consolidating warning messages is reduced to one so that we'll now only show one of each distinct warning summary. Full warning messages are always shown if there's at least one error included in the diagnostic set too, because in that case the warning message could contain additional context to help understand the error.	6 years ago
Brian Flad	20a814028d	command: Consistency implement and document parallelism default of 10 ... References: * f4da82a023/command/command.go (L41) * b9d8e96e0c/terraform/context.go (L149-L155)	7 years ago
Sander van Harmelen	ef9054562e	commands: make sure the correct flagset is used ... A lot of commands used `c.Meta.flagSet()` to create the initial flagset for the command, while quite a few of them didn’t actually use or support the flags that are then added. So I updated a few commands to use `flag.NewFlagSet()` instead to only add the flags that are actually needed/supported. Additionally this prevents a few commands from using locking while they actually don’t need locking (as locking is enabled as a default in `c.Meta.flagSet()`.	7 years ago
Martin Atkins	e54848b86f	command: Fix many (but not all) "terraform refresh" tests	8 years ago
Martin Atkins	de3944b9bf	command: collect root module variable values for apply/plan/refresh ... This connects a missing link left by earlier refactoring: the command package is responsible for gathering up variable values provided by the user and passing them through to the backend to use in operations.	8 years ago
Martin Atkins	a3403f2766	terraform: Ugly huge change to weave in new State and Plan types ... Due to how often the state and plan types are referenced throughout Terraform, there isn't a great way to switch them out gradually. As a consequence, this huge commit gets us from the old world to a _compilable_ new world, but still has a large number of known test failures due to key functionality being stubbed out. The stubs here are for anything that interacts with providers, since we now need to do the follow-up work to similarly replace the old terraform.ResourceProvider interface with its replacement in the new "providers" package. That work, along with work to fix the remaining failing tests, will follow in subsequent commits. The aim here was to replace all references to terraform.State and its downstream types with states.State, terraform.Plan with plans.Plan, state.State with statemgr.State, and switch to the new implementations of the state and plan file formats. However, due to the number of times those types are used, this also ended up affecting numerous other parts of core such as terraform.Hook, the backend.Backend interface, and most of the CLI commands. Just as with 5861dbf3fc before, I apologize in advance to the person who inevitably just found this huge commit while spelunking through the commit history.	8 years ago
Martin Atkins	ebafa51723	command: Various updates for the new backend package API ... This is a rather-messy, complex change to get the "command" package building again against the new backend API that was updated for the new configuration loader. A lot of this is mechanical rewriting to the new API, but meta_config.go and meta_backend.go in particular saw some major changes to interface with the new loader APIs and to deal with the change in order of steps in the backend API.	8 years ago
James Bardin	67a6152091	move backend operation cancellation into meta ... Create a single command method for running and operation with cancellation.	8 years ago
James Bardin	7cba68326a	always wait for a RunningOperation to return ... If the user wishes to interrupt the running operation, only the first interrupt was communicated to the operation by canceling the provided context. A second interrupt would start the shutdown process, but not communicate this to the running operation. This order of event could cause partial writes of state. What would happen is that once the command returns, the plugin system would stop the provider processes. Once the provider processes dies, all pending Eval operations would return return with an error, and quickly cause the operation to complete. Since the backend code didn't know that the process was shutting down imminently, it would continue by attempting to write out the last known state. Under the right conditions, the process would exit part way through the writing of the state file. Add Stop and Cancel CancelFuncs to the RunningOperation, to allow it to easily differentiate between the two signals. The backend will then be able to detect a shutdown and abort more gracefully. In order to ensure that the backend is not in the process of writing the state out, the command will always attempt to wait for the process to complete after cancellation.	8 years ago
Martin Atkins	9a5c865040	command: validate config as part of loading it ... Previously we required callers to separately call .Validate on the root module to determine if there were any value errors, but we did that inconsistently and would thus see crashes in some cases where later code would try to use invalid configuration as if it were valid. Now we run .Validate automatically after config loading, returning the resulting diagnostics. Since we return a diagnostics here, it's possible to return both warnings and errors. We return the loaded module even if it's invalid, so callers are free to ignore returned errors and try to work with the config anyway, though they will need to be defensive against invalid configuration themselves in that case. As a result of this, all of the commands that load configuration now need to use diagnostic printing to signal errors. For the moment this just allows us to return potentially-multiple config errors/warnings in full fidelity, but also sets us up for later when more subsystems are able to produce rich diagnostics so we can show them all together. Finally, this commit also removes some stale, commented-out code for the "legacy" (pre-0.8) graph implementation, which has not been available for some time.	8 years ago
Martin Atkins	6f7bc4f5d7	command: use c.showDiagnostics for backend operation errors ... This allows richer diagnostics produced by some subsystems to be displayed in full-fidelity to the user.	9 years ago
Martin Atkins	5cd00a13ec	command: use new diagnostics output for config errors ... This uses the new diagnostics printer for config-related errors in the main five commands that deal with config. The immediate motivation for this is to allow HCL2-produced diagnostics to be printed out in their full fidelity, though it also slightly changes the presentation of other errors so that they are not presented in all red text, which can be hard to read on some terminals.	9 years ago
Robert Liebowitz	8d98fdecac	Autoload only .auto.tfvars files	9 years ago
Robert Liebowitz	006744bfe0	Use all tfvars files in working directory ... As a side effect, several commands that previously did not have a failure state can now fail during meta-parameter processing.	9 years ago
James Bardin	000e860706	Add plugin dir scaffolding ... add pluginDir to command.Meta, the flag to initialize it, and the methods to save and restore it.	9 years ago
James Bardin	718ede0636	have Meta.Backend use a Config rather than loading ... Instead of providing the a path in BackendOpts, provide a loaded *config.Config instead. This reduces the number of places where configuration is loaded.	9 years ago
James Bardin	5eca913b14	add cli flags for -lock-timeout ... Add the -lock-timeout flag to the appropriate commands. Add the -lock flag to `init` and `import` which were missing it. Set both stateLock and stateLockTimeout in Meta.flagsSet, and remove the extra references for clarity.	9 years ago
James Bardin	eb8e5ac739	Change CLI flag to '-lock'	9 years ago
James Bardin	07903189f1	s/Meta.lockState/Meta.stateLock/g	9 years ago
James Bardin	a157ebbccd	add -lock-state usage to plan/refresh/apply/destr	9 years ago
James Bardin	dd19cb202d	add locking to plan and refresh commands	9 years ago
Mitchell Hashimoto	ad7b063262	command: convert to use backends	9 years ago
Mitchell Hashimoto	d429e82661	command: show shadow errors to the user	10 years ago
James Nugent	0e4e94a86f	core: Fix -module for terraform output command ... The behaviour whereby outputs for a particular nested module can be output was broken by the changes for lists and maps. This commit restores the previous behaviour by passing the module path into the outputsAsString function. We also add a new test of this since the code path for indivdual output vs all outputs for a module has diverged.	10 years ago
James Nugent	6a20e8927d	core: Fix issues from rebasing dev-0.7 onto master ... - Fix sensitive outputs for lists and maps - Fix test prelude which was missed during conflict resolution - Fix `terraform output` to match old behaviour and not have outputs header and colouring - Bump timeout on TestAtlasClient_UnresolvableConflict	10 years ago
James Nugent	b62f6af158	core: Add support for marking outputs as sensitive (#6559) ... * core: Add support for marking outputs as sensitive This commit allows an output to be marked "sensitive", in which case the value is redacted in the post-refresh and post-apply list of outputs. For example, the configuration: ``` variable "input" { default = "Hello world" } output "notsensitive" { value = "${var.input}" } output "sensitive" { sensitive = true value = "${var.input}" } ``` Would result in the output: ``` terraform apply Apply complete! Resources: 0 added, 0 changed, 0 destroyed. Outputs: notsensitive = Hello world sensitive = <sensitive> ``` The `terraform output` command continues to display the value as before. Limitations: Note that sensitivity is not tracked internally, so if the output is interpolated in another module into a resource, the value will be displayed. The value is still present in the state.	10 years ago
James Nugent	50b5e7c1a5	Validate context after input of vars on refresh ... Fixes #4013.	10 years ago
Kevin Nuckolls	fc60b2858c	Added -parallelism to refresh and plan cli UI	11 years ago
Paul Hinze	ad680b1832	core: show outputs after `terraform refresh` ... closes #25	11 years ago
Paul Hinze	97acccd3ed	core: targeted operations ... Add `-target=resource` flag to core operations, allowing users to target specific resources in their infrastructure. When `-target` is used, the operation will only apply to that resource and its dependencies. The calculated dependencies are different depending on whether we're running a normal operation or a `terraform destroy`. Generally, "dependencies" refers to ancestors: resources falling _before_ the target in the graph, because their changes are required to accurately act on the target. For destroys, "dependencies" are descendents: those resources which fall _after_ the target. These resources depend on our target, which is going to be destroyed, so they should also be destroyed.	11 years ago
Mitchell Hashimoto	0c6d133d61	command/refresh: get rid of remote package	11 years ago
Phil Kates	b211d69dde	Fix typo in refresh command error ... Noticed this while I was digging around to see how the commands work.	11 years ago
Kevin van Zonneveld	07505ed159	Fix typo: expacts -> expects	11 years ago
Armon Dadgar	b6265cf1a2	command/refresh: Remote enable	11 years ago