115 Commits (94de72c63834acea811c27a585c19eae29374a4b)

Author	SHA1	Message	Date
Martin Atkins	fda0579537	Experiments supported only in alpha/dev builds ... We originally introduced the idea of language experiments as a way to get early feedback on not-yet-proven feature ideas, ideally as part of the initial exploration of the solution space rather than only after a solution has become relatively clear. Unfortunately, our tradeoff of making them available in normal releases behind an explicit opt-in in order to make it easier to participate in the feedback process had the unintended side-effect of making it feel okay to use experiments in production and endure the warnings they generate. This in turn has made us reluctant to make use of the experiments feature lest experiments become de-facto production features which we then feel compelled to preserve even though we aren't yet ready to graduate them to stable features. In an attempt to tweak that compromise, here we make the availability of experiments _at all_ a build-time flag which will not be set by default, and therefore experiments will not be available in most release builds. The intent (not yet implemented in this PR) is for our release process to set this flag only when it knows it's building an alpha release or a development snapshot not destined for release at all, which will therefore allow us to still use the alpha releases as a vehicle for giving feedback participants access to a feature (without needing to install a Go toolchain) but will not encourage pretending that these features are production-ready before they graduate from experimental. Only language experiments have an explicit framework for dealing with them which outlives any particular experiment, so most of the changes here are to that generalized mechanism. However, the intent is that non-language experiments, such as experimental CLI commands, would also in future check Meta.AllowExperimentalFeatures and gate the use of those experiments too, so that we can be consistent that experimental features will never be available unless you explicitly choose to use an alpha release or a custom build from source code. Since there are already some experiments active at the time of this commit which were not previously subject to this restriction, we'll pragmatically leave those as exceptions that will remain generally available for now, and so this new approach will apply only to new experiments started in the future. Once those experiments have all concluded, we will be left with no more exceptions unless we explicitly choose to make an exception for some reason we've not imagined yet. It's important that we be able to write tests that rely on experiments either being available or not being available, so here we're using our typical approach of making "package main" deal with the global setting that applies to Terraform CLI executables while making the layers below all support fine-grain selection of this behavior so that tests with different needs can run concurrently without trampling on one another. As a compromise, the integration tests in the terraform package will run with experiments enabled _by default_ since we commonly need to exercise experiments in those tests, but they can selectively opt-out if they need to by overriding the loader setting back to false again.	4 years ago
Martin Atkins	5b266dd5ca	command: Remove the experimental "terraform add" command ... We introduced this experiment to gather feedback, and the feedback we saw led to us deciding to do another round of design work before we move forward with something to meet this use-case. In addition to being experimental, this has only been included in alpha releases so far, and so on both counts it is not protected by the Terraform v1.0 Compatibility Promises.	5 years ago
Martin Atkins	65e0c448a0	workdir: Start of a new package for working directory state management ... Thus far our various interactions with the bits of state we keep associated with a working directory have all been implemented directly inside the "command" package -- often in the huge command.Meta type -- and not managed collectively via a single component. There's too many little codepaths reading and writing from the working directory and data directory to refactor it all in one step, but this is an attempt at a first step towards a future where everything that reads and writes from the current working directory would do so via an object that encapsulates the implementation details and offers a high-level API to read and write all of these session-persistent settings. The design here continues our gradual path towards using a dependency injection style where "package main" is solely responsible for directly interacting with the OS command line, the OS environment, the OS working directory, the stdio streams, and the CLI configuration, and then communicating the resulting information to the rest of Terraform by wiring together objects. It seems likely that eventually we'll have enough wiring code in package main to justify a more explicit organization of that code, but for this commit the new "workdir.Dir" object is just wired directly in place of its predecessors, without any significant change of code organization at that top layer. This first commit focuses on the main files and directories we use to find provider plugins, because a subsequent commit will lightly reorganize the separation of concerns for plugin launching with a similar goal of collecting all of the relevant logic together into one spot.	5 years ago
Kristin Laemmert	583859e510	commands: `terraform add` (#28874) ... * command: new command, terraform add, generates resource templates terraform add ADDRESS generates a resource configuration template with all required (and optionally optional) attributes set to null. This can optionally also pre-populate nonsesitive attributes with values from an existing resource of the same type in state (sensitive vals will be populated with null and a comment indicating sensitivity) * website: terraform add documentation	5 years ago
Martin Atkins	b40a4fb741	Move plugin/ and plugin6/ to internal/plugin{,6}/ ... This is part of a general effort to move all of Terraform's non-library package surface under internal in order to reinforce that these are for internal use within Terraform only. If you were previously importing packages under this prefix into an external codebase, you could pin to an earlier release tag as an interim solution until you've make a plan to achieve the same functionality some other way.	5 years ago
Martin Atkins	ffe056bacb	Move command/ to internal/command/ ... This is part of a general effort to move all of Terraform's non-library package surface under internal in order to reinforce that these are for internal use within Terraform only. If you were previously importing packages under this prefix into an external codebase, you could pin to an earlier release tag as an interim solution until you've make a plan to achieve the same functionality some other way.	5 years ago
Martin Atkins	b9a93a0fe7	Move addrs/ to internal/addrs/ ... This is part of a general effort to move all of Terraform's non-library package surface under internal in order to reinforce that these are for internal use within Terraform only. If you were previously importing packages under this prefix into an external codebase, you could pin to an earlier release tag as an interim solution until you've make a plan to achieve the same functionality some other way.	5 years ago
Martin Atkins	bf396b5f1b	command/views: main View type aware if it's running in automation ... This "running in automation" idea is a best effort thing where we try to avoid printing out specific suggestions of commands to run in the main workflow when the user is running Terraform inside a wrapper script or other automation, because they probably don't want to bypass that automation. This just makes that information available to the main views.View type, so we can then make use of it in the implementation of more specialized view types that embed views.View. However, nothing is using it as of this commit. We'll use it in later commits.	5 years ago
Martin Atkins	7f78342953	command: Experimental "terraform test" command ... This is just a prototype to gather some feedback in our ongoing research on integration testing of Terraform modules. The hope is that by having a command integrated into Terraform itself it'll be easier for interested module authors to give it a try, and also easier for us to iterate quickly based on feedback without having to coordinate across multiple codebases. Everything about this is subject to change even in future patch releases. Since it's a CLI command rather than a configuration language feature it's not using the language experiments mechanism, but generates a warning similar to the one language experiments generate in order to be clear that backward compatibility is not guaranteed.	5 years ago
Pam Selle	08b649b6f9	Remove removed upgrade commands ... Delete commands that are no longer in Terraform and have been removed for a full release cycle.	5 years ago
Alisdair McDiarmid	c5a6aa31d3	cli: Add initial command views abstraction ... Terraform supports multiple output formats for several sub-commands. The default format is user-readable text, but many sub-commands support a `-json` flag to output a machine-readable format for the result. The output command also supports a `-raw` flag for a simpler, scripting- focused machine readable format. This commit adds a "views" abstraction, intended to help ensure consistency between the various output formats. This extracts the render specific code from the command package, and moves it into a views package. Each command is expected to create an interface for its view, and one or more implementations of that interface. By doing so, we separate the concerns of generating the sub-command result from rendering the result in the specified output format. This should make it easier to ensure that all output formats will be updated together when changes occur in the result-generating phase. There are some other consequences of this restructuring: - Views now directly access the terminal streams, rather than the now-redundant cli.Ui instance; - With the reorganization of commands, parsing CLI arguments is now the responsibility of a separate "arguments" package. For now, views are added only for the output sub-command, as an example. Because this command uses code which is shared with the apply and refresh commands, those are also partially updated.	5 years ago
Nick Fagerlund	a60120477c	Update links to CLI docs in code comments, messages, and readme	5 years ago
Martin Atkins	d2c3403ab6	command: Use the new terminal.Streams object ... Here we propagate in the initialized terminal.Streams from package main, and then onwards to backends running in CLI mode. This also replaces our use of helper/wrappedstreams to determine whether stdin is a terminal or a pipe. helper/wrappedstreams returns incorrect file descriptors on Windows, causing StdinPiped to always return false on that platform and thus causing one of the odd behaviors discussed in Finally, this includes some wrappers around the ability to look up the number of columns in the terminal in preparation for use elsewhere. These wrappers deal with the fact that our unit tests typically won't populate meta.Streams.	5 years ago
Martin Atkins	15c0645bd5	main: initialize the terminal (if any) using internal/terminal ... We need to call into terminal.Init in early startup to make sure that we either have a suitable Terminal or that we disable attempts to use virtual terminal escape sequences. This commit gets the terminal initialized but doesn't do much with it after that. Subsequent commits will make more use of this.	5 years ago
Pam Selle	83e6703bf7	Remove revision from version command ... The revision field is only populated on dev builds so this means most releases of Terraform have an empty "terraform_revision" field in the JSON output. Since we recommend developers use go tooling to `go build` this tool when developing, the revision is not useful data and so it is removed.	5 years ago
James Bardin	bdfea50cc8	remove unused	5 years ago
James Bardin	5e089c2c09	run built-in provisioners in-process ... Use the new provisioner interfaces, and run the built-in provisioners in-process.	5 years ago
Martin Atkins	0a596d2a12	command/version: Report the current platform ... Along with all of the other information we previously reported in the "terraform version" output, we'll now include the name of the current platform as our provider mechanisms represent it. This is addressing a long-standing minor annoyance where we often can't tell from an incomplete bug report which platform Terraform was running on, and incomplete bug reporters do tend to at least include the "terraform version" output even if they don't also include the requested full trace log. However, what motivated doing it _now_ is that anyone building a provider registry or mirror needs to have some awareness of these platform identifiers which have been, until v0.13, mostly an implementation detail. This additional information is a small thing we can do to help registry builders find out what the platform identifier ought to be for each of the platforms they aim to support, even if some of them are platforms which the Go compiler allows but which HashiCorp doesn't officially support. The new information is on a line of its own in the output as a pragmatic way to avoid breaking anyone who might be using something like $(terraform version | head -n1) to print a brief Terraform version identifier into some logs. That's not an interface we officially support for machine consumption, but it's easy to avoid breaking it here and so we won't do so.	5 years ago
Martin Atkins	6a44586a8f	website: Update the CLI commands index page for latest help output ... My initial motivation here was to update the example output from Terraform's top-level help list to match recent updates in the layout and language used. However, while here I took the opportunity to update some dated language that was not consistent with our modern documentation writing style, in particular including a totally unnecessary and potentially-alienating claim that Terraform is "very easy to use". Our modern writing style discourages this sort of "boastful" language and encourages us to focus on the facts at hand.	6 years ago
Martin Atkins	9665901e8e	main: Emphasize only the primary workflow commands in our help ... A long time ago we introduced this separation between "common commands" and "all other commands", but over the intervening years we've not really done a good job of classifying new commands we've added and so by default most of them ended up being classified as "common". In the interests of making this output more useful for those getting started, this switches the two categories so that "Main commands" is now the curated list, and "all other commands" is the bucket for everything else. The intent here is that the "main commands" are the ones users are likely to try as part of their initial learning of Terraform, while the other commands are for less common situations where the user is more likely to learn about a specific command in some other context, like a tutorial about a special situation. The "main commands" are also now ordered by the sequence users will typically run them in, rather than alphabetical order. That's a subjective readability tradeoff, but I think as long as the list stays relatively short (which it should) it's still relatively easy to scan and find a particular command in the shortlist.	6 years ago
Martin Atkins	f44265e59e	main: Hide several commands from our help ... These are commands that either no longer do anything aside from emitting an error message or are just backward-compatibility aliases for other commands. This generalizes our previous situation where we were specifically hiding "internal-plugin", and does so in a way that fixes the long-standing cosmetic bug that the column width in the help output was chosen based on the hidden command "internal-plugin", which is unfortunately also the longest command in our command set.	6 years ago
Martin Atkins	39504ede05	command: Remove the useless "debug" subcommand ... This is just a husk of a container command that has no nested commands under it, so it isn't serving any purpose.	6 years ago
Martin Atkins	30204ecded	command/cliconfig: Allow development overrides for providers ... For normal provider installation we want to associate each provider with a selected version number and find a suitable package for that version that conforms to the official hashes for that release. Those requirements are very onerous for a provider developer currently testing a not-yet-released build, though. To allow for that case this new CLI configuration feature allows overriding specific providers to refer to give local filesystem directories. Any provider overridden in this way is not subject to the usual restrictions about selected versions or checksum conformance, and activating an override won't cause any changes to the selections recorded in the lock file because it's intended to be a temporary setting for one developer only. This is, in a sense, a spiritual successor of an old capability we had to override specific plugins in the CLI configuration file. There were some vestiges of that left in the main package and CLI config package but nothing has actually been honoring them for several versions now and so this commit removes them to avoid confusion with the new mechanism.	6 years ago
Martin Atkins	e270291f19	command: terraform providers lock ... This command is intended to help support situations where Terraform is configured to use only local mirrors for provider installation and so the normal "terraform init" flow would not have direct access to the official package checksums published in the origin registry. The intended workflow here is to use this command only when adding a new provider or changing an existing provider's version in the configuration, to augment the lock file with all of the checksums required to verify the provider across a variety of different platforms. Once this command has recorded all of the official checksums, future runs of "terraform init" will verify that provider packages obtained from a local mirror match with those upstream checksums.	6 years ago
Martin Atkins	efe78b2910	main: new global option -chdir ... This new option is intended to address the previous inconsistencies where some older subcommands supported partially changing the target directory (where Terraform would use the new directory inconsistently) where newer commands did not support that override at all. Instead, now Terraform will accept a -chdir command at the start of the command line (before the subcommand) and will interpret it as a request to direct all actions that would normally be taken in the current working directory into the target directory instead. This is similar to options offered by some other similar tools, such as the -C option in "make". The new option is only accepted at the start of the command line (before the subcommand) as a way to reflect that it is a global command (not specific to a particular subcommand) and that it takes effect _before_ executing the subcommand. This also means it'll be forced to appear before any other command-specific arguments that take file paths, which hopefully communicates that those other arguments are interpreted relative to the overridden path. As a measure of pragmatism for existing uses, the path.cwd object in the Terraform language will continue to return the _original_ working directory (ignoring -chdir), in case that is important in some exceptional workflows. The path.root object gives the root module directory, which will always match the overriden working directory unless the user simultaneously uses one of the legacy directory override arguments, which is not a pattern we intend to support in the long run. As a first step down the deprecation path, this commit adjusts the documentation to de-emphasize the inconsistent old command line arguments, including specific guidance on what to use instead for the main three workflow commands, but all of those options remain supported in the same way as they were before. In a later commit we'll make those arguments produce a visible deprecation warning in Terraform's output, and then in an even later commit we'll remove them entirely so that -chdir is the single supported way to run Terraform from a directory other than the one containing the root module configuration.	6 years ago
Martin Atkins	49e2e00231	command: terraform providers mirror ... This new command is intended to make it easy to create or update a mirror directory containing suitable providers for the current configuration, producing a layout that is appropriate both for a filesystem mirror or, if copied into the document root of an HTTP server, a network mirror. This initial version is not customizable aside from being able to select multiple platforms to install packages for. Future iterations of this could include commands to turn the JSON index generation on and off, or to instruct it to produce the unpacked directory layout instead of the packed directory layout as it currently does. Both of those options would make the generated directory unsuitable to be a network mirror, but it would still work as a filesystem mirror. In the long run this will hopefully form part of a replacement workflow to terraform-bundle as a way to put copies of providers somewhere so we don't need to re-download them every time, but some other changes will be needed outside of just this command before that'd be true, such as adding support for network and/or filesystem mirrors in Terraform Enterprise.	6 years ago
Alisdair McDiarmid	5e2b11657e	command: Fix 0.12upgrade stub	6 years ago
Paddy	5127f1ef8b	command: Unmanaged providers ... This adds supports for "unmanaged" providers, or providers with process lifecycles not controlled by Terraform. These providers are assumed to be started before Terraform is launched, and are assumed to shut themselves down after Terraform has finished running. To do this, we must update the go-plugin dependency to v1.3.0, which added support for the "test mode" plugin serving that powers all this. As a side-effect of not needing to manage the process lifecycle anymore, Terraform also no longer needs to worry about the provider's binary, as it won't be used for anything anymore. Because of this, we can disable the init behavior that concerns itself with downloading that provider's binary, checking its version, and otherwise managing the binary. This is all managed on a per-provider basis, so managed providers that Terraform downloads, starts, and stops can be used in the same commands as unmanaged providers. The TF_REATTACH_PROVIDERS environment variable is added, and is a JSON encoding of the provider's address to the information we need to connect to it. This change enables two benefits: first, delve and other debuggers can now be attached to provider server processes, and Terraform can connect. This allows for attaching debuggers to provider processes, which before was difficult to impossible. Second, it allows the SDK test framework to host the provider in the same process as the test driver, while running a production Terraform binary against the provider. This allows for Go's built-in race detector and test coverage tooling to work as expected in provider tests. Unmanaged providers are expected to work in the exact same way as managed providers, with one caveat: Terraform kills provider processes and restarts them once per graph walk, meaning multiple times during most Terraform CLI commands. As unmanaged providers can't be killed by Terraform, and have no visibility into graph walks, unmanaged providers are likely to have differences in how their global mutable state behaves when compared to managed providers. Namely, unmanaged providers are likely to retain global state when managed providers would have reset it. Developers relying on global state should be aware of this.	6 years ago
Alisdair McDiarmid	7165d6c429	command: Add state replace-provider subcommand ... Terraform 0.13 will allow the installation of providers from various sources. If a user updates their configuration to change the source of an in-use provider (for example, if the provider namespace changes), they will also need to update the state file accordingly. This commit introduces a new `state replace-provider` subcommand which supports this. All resources using the `from` provider will be updated to use the `to` provider.	6 years ago
Kristin Laemmert	5f313a65ad	command: remove 0.12upgrade (#24403) ... * command: remove 0.12upgrade and related `configupgrade` library * leave deprecation warning for 0.12upgrade to point users to v0.12	6 years ago
Alisdair McDiarmid	3b0b29ef52	command: Add scaffold for 0.13upgrade command	6 years ago
James Bardin	1b45b744c3	remove json2dot command ... There's no need for the json2dot command since we can't create json debug graphs.	6 years ago
Alisdair McDiarmid	081f02971d	command/logout: Add terraform logout command ... Use terraform logout to remove stored credentials for a remote service host.	6 years ago
Alisdair McDiarmid	b75201acc2	Enable login subcommand, add manual token support	6 years ago
Martin Atkins	e9d0822b2a	command: Accept a "provider source" from the main package ... Following the same approach we use for other CLI-Config-able objects like the service discovery system, the main package is responsible for producing a suitable implementation of this interface which the command package can then use. When unit testing in the command package we can then substitute mocks as necessary, following the dependency inversion principle.	6 years ago
Pam Selle	78b1220558	Remove config.go and update things using its aliases	6 years ago
Radek Simko	32f9722d9d	Replace import paths & set UA string where necessary	7 years ago
Martin Atkins	67d6f58f31	main: use cliconfig.ConfigDir instead of just ConfigDir ... main.ConfigDir is just a wrapper around cliconfig.ConfigDir to allow us to gradually clean up the old calls here, but since this is new code we might as well do it right from the start.	7 years ago
Martin Atkins	131656a237	main: Temporarily disable "terraform login" as a command ... We're not ready to ship this in a release yet because there's still some remaining work to do on the Terraform Cloud side, but we want to get the implementation work behind this into the master branch so it's easier to maintain it in the mean time, rather than letting this long-lived branch live even longer. We'll continue to iterate on UX polish and other details in subsequent commits, and eventually enable this.	7 years ago
Martin Atkins	7ccd6204c4	command: Swappable implementation of launching web browsers ... For unit testing in particular we can't launch a real browser for testing, so this indirection is primarily to allow us to substitute a mock when testing a command that can launch a browser. This includes a simple mock implementation that expects to interact with a running web server directly.	7 years ago
Martin Atkins	6bba3ceb42	command: "terraform login" command	7 years ago
Martin Atkins	22a2580e93	main: Use the new cliconfig package credentials source ... This should not cause any change in behavior yet, but using this new implementation will allow the "terraform login" and "terraform logout" commands to store and forget credentials when they are implemented in subsequent commits.	7 years ago
Martin Atkins	e1590d0a70	command/cliconfig: Factor out CLI config handling ... This is just a wholesale move of the CLI configuration types and functions from the main package into its own package, leaving behind some type aliases and wrappers for now to keep existing callers working. This commit alone doesn't really achieve anything, but in future commits we'll expand the functionality in this package.	7 years ago
Kristin Laemmert	16823f43de	command/jsonprovider: export providers schemas to json (#20446) ... * command/jsonprovider: a new package for exporting providers schemas as JSON	7 years ago
Martin Atkins	ffe5f7c4e6	command: 0.12upgrade command ... This is the frontend to the work-in-progress codepath for upgrading the source code for a module written for Terraform v0.11 or earlier to use the new syntax and idiom of v0.12. The underlying upgrade code is not yet complete as of this commit, and so the command is not yet very useful. We will continue to iterate on the upgrade code in subsequent commits.	8 years ago
Martin Atkins	618883596a	command: remove "terraform push" ... The remote API this talks to will be going away very soon, before our next major release, and so we'll remove the command altogether in that release. This also removes the "encodeHCL" function, which was used only for adding a .tfvars-formatted file to the uploaded archive.	8 years ago
Sander van Harmelen	179b32d426	Add a `CredentialsForHost` method to disco.Disco ... By adding this method you now only have to pass a `*disco.Disco` object around in order to do discovery and use any configured credentials for the discovered hosts. Of course you can also still pass around both a `*disco.Disco` and a `auth.CredentialsSource` object if there is a need or a reason for that!	8 years ago
James Bardin	6884a07bba	use the new Meta.ShutdownCh when building commands	8 years ago
Martin Atkins	3da5fefdc1	command: Allow TF_DATA_DIR env var to override data directory ... This allows the user to customize the location where Terraform stores the files normally placed in the ".terraform" subdirectory, if e.g. the current working directory is not writable.	9 years ago
Martin Atkins	e9816c60f1	main: allow overriding host-based discovery in CLI config ... For situations where the default network-based discovery is inappropriate or inconvenient, this allows users to provide a hard-coded discovery document for a particular hostname in the CLI config. This is a new config block, rather than combined with the existing "credentials" block, because credentials should ideally live in separate files from other config so that they can be managed more carefully. However, this new "host" block _is_ designed to have room for additional host-specific configuration _other than_ credentials in future, which might include TLS certificate overrides or other such things used during the discovery step.	9 years ago