177 Commits (25cebfbd8be2c43a25dfafdcf882e79b9ccc3d3d)

Author	SHA1	Message	Date
Chris Arcand	4b97411890	Add purple to the list of colorstring codes ... This is the first color code introduced for 256 color terminals, and should be a safe addition when considering the oldest supported platform.	5 years ago
Alisdair McDiarmid	43028c4912	cli: Migrate apply to command views	5 years ago
Alisdair McDiarmid	a7b7cd29fc	cli: Migrate Terraform UI hook to command views ... Move the code which renders Terraform hook callbacks as UI into the views package, backed by a views.View instead of a cli.Ui. Update test setup accordingly. To allow commands to control this hook, we add a hooks member on the backend Operation struct. This supersedes the hooks in the Terraform context, which is not directly controlled by the command logic. This commit should not change how Terraform works, and is refactoring in preparation for more changes which move UI code out of the backend.	5 years ago
Alisdair McDiarmid	57879bfb71	cli: Add global view arguments parser ... Rather than modifying and relying on the existing Meta.process argument extractor, we can more clearly handle global CLI flags using a separate parser step. This allows us to explicitly configure the view in the command.	5 years ago
Alisdair McDiarmid	c5a6aa31d3	cli: Add initial command views abstraction ... Terraform supports multiple output formats for several sub-commands. The default format is user-readable text, but many sub-commands support a `-json` flag to output a machine-readable format for the result. The output command also supports a `-raw` flag for a simpler, scripting- focused machine readable format. This commit adds a "views" abstraction, intended to help ensure consistency between the various output formats. This extracts the render specific code from the command package, and moves it into a views package. Each command is expected to create an interface for its view, and one or more implementations of that interface. By doing so, we separate the concerns of generating the sub-command result from rendering the result in the specified output format. This should make it easier to ensure that all output formats will be updated together when changes occur in the result-generating phase. There are some other consequences of this restructuring: - Views now directly access the terminal streams, rather than the now-redundant cli.Ui instance; - With the reorganization of commands, parsing CLI arguments is now the responsibility of a separate "arguments" package. For now, views are added only for the output sub-command, as an example. Because this command uses code which is shared with the apply and refresh commands, those are also partially updated.	5 years ago
Alisdair McDiarmid	4991cc4835	cli: Improve error for invalid -target flags ... Errors encountered when parsing flags for apply, plan, and refresh were being suppressed. This resulted in a generic usage error when using an invalid `-target` flag. This commit makes several changes to address this. First, these commands now output the flag parse error before exiting, leaving at least some hint about the error. You can verify this manually with something like: terraform apply -invalid-flag We also change how target attributes are parsed, moving the responsibility from the flags instance to the command. This allows us to customize the diagnostic output to be more user friendly. The diagnostics now look like: ```shellsession $ terraform apply -no-color -target=foo Error: Invalid target "foo" Resource specification must include a resource type and name. ``` Finally, we add test coverage for both parsing of target flags, and at the command level for successful use of resource targeting. These tests focus on the UI output (via the change summary and refresh logs), as the functionality of targeting is covered by the context tests in the terraform package.	5 years ago
Pam Selle	29dd334fe8	Merge pull request #27611 from hashicorp/pselle/ignore-remote-version-docstring ... Update docstring for ignore-remote-version	5 years ago
Pam Selle	c1ede28a15	Update docstring for ignore-remote-version ... Update docstring for ignore-remote-version to be more accurate, as it's not whether the versions differ, but if they are incompatible	5 years ago
Katy Moe	51c687c2db	command: no visual warning hierarchy in -no-color ... Commit e865faf adds visual indentation for diagnostic messages using various vertical line characters. The present commit disables this behaviour when running with colourised output disabled. While the contents of stderr are not intended to be part of the Terraform API, this is currently how the hashicorp/terraform-exec library detects certain error types in order to present them as well-known Go errors to the user. Such detection is complicated when vertical lines are added to the CLI output at unpredictable points. I expect this change will also be helpful for screen reader users.	5 years ago
Martin Atkins	e865faf318	command: Better visual hierarchy for diagnostics ... I frequently see people attempting to ask questions about Terraform's error and warning messages but either only copying part of the message or accidentally copying a surrounding paragraph that isn't part of the message. While I'm sure some of these are just "careless" mistakes, I've also noticed that this has sometimes overlapped with someone asking a question whose answer is written directly in the part of the message they didn't include when copying, and so I have a theory that our current output doesn't create a good enough visual hierarchy for sighted users to understand where the diagnostic messages start and end when we show them in close proximity to other content, or to other diagnostic messages. As a result, some folks fail to notice the relevant message that might've answered their question. I tried a few different experiments for different approaches here, such as adding more horizontal rules to the output and coloring the detail text differently, but the approach that felt like the nicest compromise to me was what's implemented here, which is to add a vertical line along the left edge of each diagnostic message, colored to match with the typical color we use for each diagnostic severity. This means that the diagnostics end up slightly indented from what's around them, and the vertical line seems to help subtly signal how we intended the content to be grouped together.	5 years ago
Martin Atkins	e6a516d87e	backend/local: Use terminal properties to tweak the plan output ... We now require the output to accept UTF-8 and we can determine how wide the terminal (if any) is, so here we begin to make use of that for the "terraform plan" command. The horizontal rule is now made of box drawing characters instead of hyphens and fills the whole terminal width. The paragraphs of text in the output are now also wrapped to fill the terminal width, instead of the hard-wrapping we did before. This is just a start down the road of making better use of the terminal capabilities. Lots of other commands could benefit from updates like these too.	5 years ago
Martin Atkins	d2c3403ab6	command: Use the new terminal.Streams object ... Here we propagate in the initialized terminal.Streams from package main, and then onwards to backends running in CLI mode. This also replaces our use of helper/wrappedstreams to determine whether stdin is a terminal or a pipe. helper/wrappedstreams returns incorrect file descriptors on Windows, causing StdinPiped to always return false on that platform and thus causing one of the odd behaviors discussed in Finally, this includes some wrappers around the ability to look up the number of columns in the terminal in preparation for use elsewhere. These wrappers deal with the fact that our unit tests typically won't populate meta.Streams.	5 years ago
James Bardin	dcf0dba6f4	Merge pull request #27081 from hashicorp/jbardin/staticcheck ... Fixes to pass static analysis	5 years ago
Kristin Laemmert	3fa063b8dc	command/format: concise diff is now the default (#27079) ... * command/format: concise diff is no longer an experiment Since state formatting goes through the "diff" printer, I have repurposed the concise flag as a verbose flag, used only when printing state. It's silly but it works! * remove helper/experiment With this experiment concluded, we no longer need helper/experiment. The shadow experiment had not been touched in many years, so I removed all references, and removed the package entirely. Any new experiments are expected to be configuration experiments handled by our (other) experiments package. * check for the verbose flag consistently, in case we end up using it in plans in the future	5 years ago
James Bardin	1c58c6ba48	command staticcheck	5 years ago
James Bardin	e998882824	remove legacy types from command package	5 years ago
Alisdair McDiarmid	c5c1f31db3	backend: Validate remote backend Terraform version ... When using the enhanced remote backend, a subset of all Terraform operations are supported. Of these, only plan and apply can be executed on the remote infrastructure (e.g. Terraform Cloud). Other operations run locally and use the remote backend for state storage. This causes problems when the local version of Terraform does not match the configured version from the remote workspace. If the two versions are incompatible, an `import` or `state mv` operation can cause the remote workspace to be unusable until a manual fix is applied. To prevent this from happening accidentally, this commit introduces a check that the local Terraform version and the configured remote workspace Terraform version are compatible. This check is skipped for commands which do not write state, and can also be disabled by the use of a new command-line flag, `-ignore-remote-version`. Terraform version compatibility is defined as: - For all releases before 0.14.0, local must exactly equal remote, as two different versions cannot share state; - 0.14.0 to 1.0.x are compatible, as we will not change the state version number until at least Terraform 1.1.0; - Versions after 1.1.0 must have the same major and minor versions, as we will not change the state version number in a patch release. If the two versions are incompatible, a diagnostic is displayed, advising that the error can be suppressed with `-ignore-remote-version`. When this flag is used, the diagnostic is still displayed, but as a warning instead of an error. Commands which will not write state can assert this fact by calling the helper `meta.ignoreRemoteBackendVersionConflict`, which will disable the checks. Those which can write state should instead call the helper `meta.remoteBackendVersionCheck`, which will return diagnostics for display. In addition to these explicit paths for managing the version check, we have an implicit check in the remote backend's state manager initialization method. Both of the above helpers will disable this check. This fallback is in place to ensure that future code paths which access state cannot accidentally skip the remote version check.	5 years ago
Alisdair McDiarmid	10cc25fc21	terraform: Compare locks and provider requirements ... When building a context, we read the dependency locks and ensure that the provider requirements from the configuration can be satisfied. If the configured requirements change such that the locks need to be updated, we explain this and recommend running "terraform init". This check is ignored for any providers which are locally marked as in development. This includes unmanaged providers and those listed in the provider installation `dev_overrides` block.	6 years ago
Martin Atkins	30204ecded	command/cliconfig: Allow development overrides for providers ... For normal provider installation we want to associate each provider with a selected version number and find a suitable package for that version that conforms to the official hashes for that release. Those requirements are very onerous for a provider developer currently testing a not-yet-released build, though. To allow for that case this new CLI configuration feature allows overriding specific providers to refer to give local filesystem directories. Any provider overridden in this way is not subject to the usual restrictions about selected versions or checksum conformance, and activating an override won't cause any changes to the selections recorded in the lock file because it's intended to be a temporary setting for one developer only. This is, in a sense, a spiritual successor of an old capability we had to override specific plugins in the CLI configuration file. There were some vestiges of that left in the main package and CLI config package but nothing has actually been honoring them for several versions now and so this commit removes them to avoid confusion with the new mechanism.	6 years ago
James Bardin	1ecd86d08e	Merge pull request #26155 from hashicorp/unused-env-var ... Remove unused env var TF_SKIP_PROVIDER_VERIFY	6 years ago
Martin Atkins	b3f5c7f1e6	command/init: Read, respect, and update provider dependency locks ... This changes the approach used by the provider installer to remember between runs which selections it has previously made, using the lock file format implemented in internal/depsfile. This means that version constraints in the configuration are considered only for providers we've not seen before or when -upgrade mode is active.	6 years ago
Martin Atkins	0b734a2803	command: Make provider installation interruptible ... In earlier commits we started to make the installation codepath context-aware so that it could be canceled in the event of a SIGINT, but we didn't complete wiring that through the API of the getproviders package. Here we make the getproviders.Source interface methods, along with some other functions that can make network requests, take a context.Context argument and act appropriately if that context is cancelled. The main providercache.Installer.EnsureProviderVersions method now also has some context-awareness so that it can abort its work early if its context reports any sort of error. That avoids waiting for the process to wind through all of the remaining iterations of the various loops, logging each request failure separately, and instead returns just a single aggregate "canceled" error. We can then set things up in the "terraform init" and "terraform providers mirror" commands so that the context will be cancelled if we get an interrupt signal, allowing provider installation to abort early while still atomically completing any local-side effects that may have started.	6 years ago
Paul Tyng	f3ff843ffd	Remove unused env var TF_SKIP_PROVIDER_VERIFY	6 years ago
Martin Atkins	efe78b2910	main: new global option -chdir ... This new option is intended to address the previous inconsistencies where some older subcommands supported partially changing the target directory (where Terraform would use the new directory inconsistently) where newer commands did not support that override at all. Instead, now Terraform will accept a -chdir command at the start of the command line (before the subcommand) and will interpret it as a request to direct all actions that would normally be taken in the current working directory into the target directory instead. This is similar to options offered by some other similar tools, such as the -C option in "make". The new option is only accepted at the start of the command line (before the subcommand) as a way to reflect that it is a global command (not specific to a particular subcommand) and that it takes effect _before_ executing the subcommand. This also means it'll be forced to appear before any other command-specific arguments that take file paths, which hopefully communicates that those other arguments are interpreted relative to the overridden path. As a measure of pragmatism for existing uses, the path.cwd object in the Terraform language will continue to return the _original_ working directory (ignoring -chdir), in case that is important in some exceptional workflows. The path.root object gives the root module directory, which will always match the overriden working directory unless the user simultaneously uses one of the legacy directory override arguments, which is not a pattern we intend to support in the long run. As a first step down the deprecation path, this commit adjusts the documentation to de-emphasize the inconsistent old command line arguments, including specific guidance on what to use instead for the main three workflow commands, but all of those options remain supported in the same way as they were before. In a later commit we'll make those arguments produce a visible deprecation warning in Terraform's output, and then in an even later commit we'll remove them entirely so that -chdir is the single supported way to run Terraform from a directory other than the one containing the root module configuration.	6 years ago
Alisdair McDiarmid	35125717b2	Merge pull request #25847 from zachwhaley/fix-tf-cli-args-no-color ... Fix error when multiple -no-color arguments are used	6 years ago
Zachary Whaley	af8d5a69d8	Fix error when TF_CLI_ARGS adds -no-color argument ... Fixes #25845	6 years ago
Alisdair McDiarmid	b239570abb	command: Always validate workspace name ... The workspace name can be overridden by setting a TF_WORKSPACE environment variable. If this is done, we should still validate the resulting workspace name; otherwise, we could end up with an invalid and unselectable workspace. This change updates the Meta.Workspace function to return an error, and handles that error wherever necessary.	6 years ago
Paddy	5127f1ef8b	command: Unmanaged providers ... This adds supports for "unmanaged" providers, or providers with process lifecycles not controlled by Terraform. These providers are assumed to be started before Terraform is launched, and are assumed to shut themselves down after Terraform has finished running. To do this, we must update the go-plugin dependency to v1.3.0, which added support for the "test mode" plugin serving that powers all this. As a side-effect of not needing to manage the process lifecycle anymore, Terraform also no longer needs to worry about the provider's binary, as it won't be used for anything anymore. Because of this, we can disable the init behavior that concerns itself with downloading that provider's binary, checking its version, and otherwise managing the binary. This is all managed on a per-provider basis, so managed providers that Terraform downloads, starts, and stops can be used in the same commands as unmanaged providers. The TF_REATTACH_PROVIDERS environment variable is added, and is a JSON encoding of the provider's address to the information we need to connect to it. This change enables two benefits: first, delve and other debuggers can now be attached to provider server processes, and Terraform can connect. This allows for attaching debuggers to provider processes, which before was difficult to impossible. Second, it allows the SDK test framework to host the provider in the same process as the test driver, while running a production Terraform binary against the provider. This allows for Go's built-in race detector and test coverage tooling to work as expected in provider tests. Unmanaged providers are expected to work in the exact same way as managed providers, with one caveat: Terraform kills provider processes and restarts them once per graph walk, meaning multiple times during most Terraform CLI commands. As unmanaged providers can't be killed by Terraform, and have no visibility into graph walks, unmanaged providers are likely to have differences in how their global mutable state behaves when compared to managed providers. Namely, unmanaged providers are likely to retain global state when managed providers would have reset it. Developers relying on global state should be aware of this.	6 years ago
Martin Atkins	549aede792	Remove terraform.ResourceProvider, use providercache.Installer instead ... Back when we first introduced provider versioning in Terraform 0.10, we did the provider version resolution in terraform.NewContext because we weren't sure yet how exactly our versioning model was going to play out (whether different versions could be selected per provider configuration, for example) and because we were building around the limitations of our existing filesystem-based plugin discovery model. However, the new installer codepath is new able to do all of the selections up front during installation, so we don't need such a heavy inversion of control abstraction to get this done: the command package can select the exact provider versions and pass their factories directly to terraform.NewContext as a simple static map. The result of this commit is that CLI commands other than "init" are now able to consume the local cache directory and selections produced by the installation process in "terraform init", passing all of the selected providers down to the terraform.NewContext function for use in implementing the main operations. This commit is just enough to get the providers passing into the terraform.Context. There's still plenty more to do here, including to repair all of the tests this change has additionally broken.	6 years ago
Alisdair McDiarmid	67203dade8	command: Simplify Meta.process helper method ... After some refactoring, this helper method had an unused argument (vars) and an always-nil error return value. This commit cleans this up.	6 years ago
Martin Atkins	e9d0822b2a	command: Accept a "provider source" from the main package ... Following the same approach we use for other CLI-Config-able objects like the service discovery system, the main package is responsible for producing a suitable implementation of this interface which the command package can then use. When unit testing in the command package we can then substitute mocks as necessary, following the dependency inversion principle.	6 years ago
Martin Atkins	c06675c616	command: New -compact-warnings option ... When warnings appear in isolation (not accompanied by an error) it's reasonable to want to defer resolving them for a while because they are not actually blocking immediate work. However, our warning messages tend to be long by default in order to include all of the necessary context to understand the implications of the warning, and that can make them overwhelming when combined with other output. As a compromise, this adds a new CLI option -compact-warnings which is supported for all the main operation commands and which uses a more compact format to print out warnings as long as they aren't also accompanied by errors. The default remains unchanged except that the threshold for consolidating warning messages is reduced to one so that we'll now only show one of each distinct warning summary. Full warning messages are always shown if there's at least one error included in the diagnostic set too, because in that case the warning message could contain additional context to help understand the error.	6 years ago
Martin Atkins	7081c26e54	command: Don't show more than two of the same warning ... Some of our warnings are produced in response to particular configuration constructs which might appear many times across a Terraform configuration. To avoid the warning output dwarfing all of the other output, we'll use ConsolidateWarnings to limit each distinct warning summary to appear at most twice, and annotate the final one in the sequence with an additional paragraph noting that some number of them have been hidden. This is intended as a compromise to ensure that these warnings are still seen and noted but to help ensure that we won't produce so many of them as to distract from other output that appears alongside them. This applies only to warnings relating to specific configuration ranges; errors will continue to be shown individually, and sourceless warnings (which are rare in Terraform today) will likewise remain ungrouped because they are less likely to be repeating the same statement about different instances of the same problem throughout the configuration.	7 years ago
Radek Simko	32f9722d9d	Replace import paths & set UA string where necessary	7 years ago
Martin Atkins	e21f0fa61e	backend/local: Handle interactive prompts for variables in UI layer ... During the 0.12 work we intended to move all of the variable value collection logic into the UI layer (command package and backend packages) and present them all together as a unified data structure to Terraform Core. However, we didn't quite succeed because the interactive prompts for unset required variables were still being handled _after_ calling into Terraform Core. Here we complete that earlier work by moving the interactive prompts for variables out into the UI layer too, thus allowing us to handle final validation of the variables all together in one place and do so in the UI layer where we have the most context still available about where all of these values are coming from. This allows us to fix a problem where previously disabling input with -input=false on the command line could cause Terraform Core to receive an incomplete set of variable values, and fail with a bad error message. As a consequence of this refactoring, the scope of terraform.Context.Input is now reduced to only gathering provider configuration arguments. Ideally that too would move into the UI layer somehow in a future commit, but that's a problem for another day.	7 years ago
Martin Atkins	7ccd6204c4	command: Swappable implementation of launching web browsers ... For unit testing in particular we can't launch a real browser for testing, so this indirection is primarily to allow us to substitute a mock when testing a command that can launch a browser. This includes a simple mock implementation that expects to interact with a running web server directly.	7 years ago
Martin Atkins	6bba3ceb42	command: "terraform login" command	7 years ago
Kristin Laemmert	c9d62bb2f6	command: discard output from flags package and return errs directly (#22373) ... Any command using meta.defaultFlagSet *might* occasionally exit before the flag package's output got written. This caused flag error messages to get lost. This PR discards the flag package output in favor of directly returning the error to the end user.	7 years ago
Alex Pilon	d5997e05db	prune dead code from command/meta	7 years ago
Masayuki Morita	ca9b643c73	docs: Fix typo in (*Meta) RunOperation comments	7 years ago
Sander van Harmelen	973e2a7cf9	core: add a context to the UIInput interface	7 years ago
Martin Atkins	2cf63d068f	command: Always normalize config path before operations ... Previously we were doing this rather inconsistently: some commands would do it and others would not. By doing it here we ensure we always apply the same normalization, regardless of which operation we're running. This normalization is mostly for cosmetic purposes in error messages, but it also ends up being used to populate path.module and path.root and so it's important that we always produce consistent results here so that we don't produce flappy changes as users work with different commands. The fact that thus mutates a data structure as a side-effect is not ideal but this is the best place to ensure it always gets applied without doing any significant refactoring, since everything after this point happens in the backend package where the normalizePath method is not available.	7 years ago
Sander van Harmelen	ef9054562e	commands: make sure the correct flagset is used ... A lot of commands used `c.Meta.flagSet()` to create the initial flagset for the command, while quite a few of them didn’t actually use or support the flags that are then added. So I updated a few commands to use `flag.NewFlagSet()` instead to only add the flags that are actually needed/supported. Additionally this prevents a few commands from using locking while they actually don’t need locking (as locking is enabled as a default in `c.Meta.flagSet()`.	8 years ago
Martin Atkins	e20346bf4f	command: fix TestMeta_process ... The mission of this process method used to include dealing with auto-loaded tfvars files, but it doesn't do that anymore. It does still deal with the -no-color option, but the test wasn't exercising that part before. Now the test here focuses on the -no-color behavior. The process method still has a "vars" flag argument which is no longer used. Since this is an unexported method we could potentially address this but this commit is intentionally limited only to fixing the test.	8 years ago
Sander van Harmelen	52a1b22f7a	Implement the remote enhanced backend ... This is a refactored version of the `remote` backend that was initially added to Terraform v0.11.8 which should now be compatible with v0.12.0.	8 years ago
Sander van Harmelen	178ec8f7b4	Remove support for the -module-depth flag ... # Conflicts: # backend/backend.go	8 years ago
Martin Atkins	541952bb8f	Revert some work that happened since v0.12-dev branched ... This work was done against APIs that were already changed in the branch before work began, and so it doesn't apply to the v0.12 development work. To allow v0.12 to merge down to master, we'll revert this work out for now and then re-introduce equivalent functionality in later commits that works against the new APIs.	8 years ago
Martin Atkins	d9051be66d	command: Use new provider and provisioner types in test helpers ... This moves our build errors from the test helpers out into the tests themselves. We'll update the tests in subsequent commits.	8 years ago
Martin Atkins	44bc7519a6	terraform: More wiring in of new provider types ... This doesn't actually work yet, but it builds and then panics in a pretty satisfying way.	8 years ago
Martin Atkins	a3403f2766	terraform: Ugly huge change to weave in new State and Plan types ... Due to how often the state and plan types are referenced throughout Terraform, there isn't a great way to switch them out gradually. As a consequence, this huge commit gets us from the old world to a _compilable_ new world, but still has a large number of known test failures due to key functionality being stubbed out. The stubs here are for anything that interacts with providers, since we now need to do the follow-up work to similarly replace the old terraform.ResourceProvider interface with its replacement in the new "providers" package. That work, along with work to fix the remaining failing tests, will follow in subsequent commits. The aim here was to replace all references to terraform.State and its downstream types with states.State, terraform.Plan with plans.Plan, state.State with statemgr.State, and switch to the new implementations of the state and plan file formats. However, due to the number of times those types are used, this also ended up affecting numerous other parts of core such as terraform.Hook, the backend.Backend interface, and most of the CLI commands. Just as with 5861dbf3fc before, I apologize in advance to the person who inevitably just found this huge commit while spelunking through the commit history.	8 years ago