176 Commits (3679de0630ffbe67fa8d2ff299891dfdcb01cd6a)

Author	SHA1	Message	Date
Alisdair McDiarmid	786d99207e	cli: Only rewrite provider locks file if changed ... If the provider locks have not changed, there is no need to rewrite the locks file. Preventing this needless rewrite should allow Terraform to operate in a read-only directory, so long as the provider requirements don't change.	5 years ago
Pam Selle	242f319638	Give suggestions & remind users to use required_providers when provider not in registry (#28014) ... * Add helper suggestion when failed registry err When someone has a failed registry error on init, remind them that they should have required_providers in every module * Give suggestion for a provider based on reqs Suggest another provider on a registry error, from the list of requirements we have on init. This skips the legacy lookup process if there is a similar provider existing in requirements.	5 years ago
Masayuki Morita	31a5aa1878	command/init: Add a new flag `-lockfile=readonly` (#27630) ... Fixes #27506 Add a new flag `-lockfile=readonly` to `terraform init`. It would be useful to allow us to suppress dependency lockfile changes explicitly. The type of the `-lockfile` flag is string rather than bool, leaving room for future extensions to other behavior variants. The readonly mode suppresses lockfile changes, but should verify checksums against the information already recorded. It should conflict with the `-upgrade` flag. Note: In the original use-case described in #27506, I would like to suppress adding zh hashes, but a test code here suppresses adding h1 hashes because it's easy for testing. Co-authored-by: Alisdair McDiarmid <alisdair@users.noreply.github.com>	5 years ago
Alisdair McDiarmid	94340b5940	cli: Add reference to global options to help text	5 years ago
Alisdair McDiarmid	ca23a096d8	cli: Remove legacy positional path arguments ... Several commands continued to support the legacy positional path argument to specify a working directory. This functionality has been replaced with the global -chdir flag, which is specified before any other arguments, including the sub-command name. This commit removes support for the trailing path parameter from most commands. The only command which still supports a path argument is fmt, which also supports "-" to indicate receiving configuration from standard input. Any invocation of a command with an invalid trailing path parameter will result in a short error message, pointing at the -chdir alternative. There are many test updates in this commit, almost all of which are migrations from using positional arguments to specify a working directory. Because of the layer at which these tests run, we are unable to use the -chdir argument, so the churn in test files is larger than ideal. Sorry!	5 years ago
Brian Flad	70aac9c551	command: Use different warning messaging in init command and move above potentially confusing output (#27514) ... Reference: https://github.com/hashicorp/terraform/issues/27459 Reference: https://github.com/hashicorp/terraform/pull/27507 This follows the change in the plan command to move the development overrides warning above other output from the command that may exhibit unexpected behavior.	5 years ago
Jonathan Hall	49ee3d3ef8	Grammar nit: "setup" as a verb should be spelled "set up"	5 years ago
Nick Fagerlund	a60120477c	Update links to CLI docs in code comments, messages, and readme	5 years ago
Pam Selle	0d87c66211	Merge pull request #27463 from hashicorp/pselle/get-plugins ... Remove get-plugins flag	5 years ago
Pam Selle	801f8aa0b7	Merge pull request #27461 from hashicorp/pselle/verify-plugins ... Remove verify-plugins flag	5 years ago
Pam Selle	bbcf6ae20b	Remove state lock flags from init	5 years ago
Pam Selle	5ad6100ff3	Remove get-plugins ... Remove the no-op get-plugins flag	5 years ago
Pam Selle	da5a28e6ae	Remove verify-plugins flag ... This flag does not do anything, and so this removes the mention in the CLI docs and the parsing of the flag variable	5 years ago
Martin Atkins	bab4979128	command/init: Remove the warnings about the "legacy" cache directory ... We included these warnings in v0.14 after noticing that we'd accidentally published some incorrect documentation about the purpose of the plugin cache directory under .terraform/plugins. We switched to using .terraform/providers instead so that we could treat any missing providers that appear in the legacy directory as likely to be a result of following that documentation, and thus produce this extra warning. However, the further we get from v0.13 the more likely it is for this warning to be a confusing false positive rather than something helpful, and this is a non-trivial codepath requiring us to retain a concept that we otherwise don't need (the "legacy cache dir"), so here we'll remove those warnings and support code for v0.15 onwards. These warnings were always accompanied by an error message saying that a provider could not be found, and that error message remains after this change. This just removes the "by the way..."-style warning we had been emitting alongside the errors.	5 years ago
Martin Atkins	4b3e237668	command/init: Hint about providers in other namespaces ... If a user forgets to specify the source address for a provider, Terraform will assume they meant a provider in the registry.terraform.io/hashicorp/ namespace. If that ultimately doesn't exist, we'll now try to see if there's some other provider source address recorded in the registry's legacy provider lookup table, and suggest it if so. The error message here is a terse one addressed primarily to folks who are already somewhat familiar with provider source addresses and how to specify them. Terraform v0.13 had a more elaborate version of this error message which directed the user to try the v0.13 automatic upgrade tool, but we no longer have that available in v0.14 and later so the user must make the fix themselves.	5 years ago
Pam Selle	b963ea8594	Update docs and add warning for -get-plugins ... As of Terraform 0.13+, the get-plugins command has been superceded by new provider installation mechanisms, and general philosophy (providers are always installed, but the sources may be customized). Updat the init command to give users a warning if they are setting this flag, to encourage them to remove it from their workflow, and update relevant docs and docstrings as well	5 years ago
James Bardin	1c58c6ba48	command staticcheck	5 years ago
Alisdair McDiarmid	c5c1f31db3	backend: Validate remote backend Terraform version ... When using the enhanced remote backend, a subset of all Terraform operations are supported. Of these, only plan and apply can be executed on the remote infrastructure (e.g. Terraform Cloud). Other operations run locally and use the remote backend for state storage. This causes problems when the local version of Terraform does not match the configured version from the remote workspace. If the two versions are incompatible, an `import` or `state mv` operation can cause the remote workspace to be unusable until a manual fix is applied. To prevent this from happening accidentally, this commit introduces a check that the local Terraform version and the configured remote workspace Terraform version are compatible. This check is skipped for commands which do not write state, and can also be disabled by the use of a new command-line flag, `-ignore-remote-version`. Terraform version compatibility is defined as: - For all releases before 0.14.0, local must exactly equal remote, as two different versions cannot share state; - 0.14.0 to 1.0.x are compatible, as we will not change the state version number until at least Terraform 1.1.0; - Versions after 1.1.0 must have the same major and minor versions, as we will not change the state version number in a patch release. If the two versions are incompatible, a diagnostic is displayed, advising that the error can be suppressed with `-ignore-remote-version`. When this flag is used, the diagnostic is still displayed, but as a warning instead of an error. Commands which will not write state can assert this fact by calling the helper `meta.ignoreRemoteBackendVersionConflict`, which will disable the checks. Those which can write state should instead call the helper `meta.remoteBackendVersionCheck`, which will return diagnostics for display. In addition to these explicit paths for managing the version check, we have an implicit check in the remote backend's state manager initialization method. Both of the above helpers will disable this check. This fallback is in place to ensure that future code paths which access state cannot accidentally skip the remote version check.	5 years ago
Martin Atkins	c94a6102df	command: Improve consistency of the command short descriptions ... The short description of our commands (as shown in the main help output from "terraform") was previously very inconsistent, using different tense/mood for different commands. Some of the commands were also using some terminology choices inconsistent with how we currently talk about the related ideas in our documentation. Here I've tried to add some consistency by first rewriting them all in the imperative mood (except the ones that just are just subcommand groupings), and tweaking some of the terminology to hopefully gel better with how we present similar ideas in our recently-updated docs. While working on this I inevitably spotted some similar inconsistencies in the longer-form help output of some of the commands. I've not reviewed all of these for consistency, but I did update some where the wording was either left inconsstent with the short form changes I'd made or where the prose stood out to me as particularly inconsistent with our current usual documentation language style. All of this is subjective, so I expect we'll continue to tweak these over time as we continue to develop our documentation writing style based on user questions and feedback.	5 years ago
Martin Atkins	30204ecded	command/cliconfig: Allow development overrides for providers ... For normal provider installation we want to associate each provider with a selected version number and find a suitable package for that version that conforms to the official hashes for that release. Those requirements are very onerous for a provider developer currently testing a not-yet-released build, though. To allow for that case this new CLI configuration feature allows overriding specific providers to refer to give local filesystem directories. Any provider overridden in this way is not subject to the usual restrictions about selected versions or checksum conformance, and activating an override won't cause any changes to the selections recorded in the lock file because it's intended to be a temporary setting for one developer only. This is, in a sense, a spiritual successor of an old capability we had to override specific plugins in the CLI configuration file. There were some vestiges of that left in the main package and CLI config package but nothing has actually been honoring them for several versions now and so this commit removes them to avoid confusion with the new mechanism.	5 years ago
James Bardin	1ecd86d08e	Merge pull request #26155 from hashicorp/unused-env-var ... Remove unused env var TF_SKIP_PROVIDER_VERIFY	5 years ago
Martin Atkins	e70ab09bf1	command: new cache directory .terraform/providers for providers ... Terraform v0.10 introduced .terraform/plugins as a cache directory for automatically-installed plugins, Terraform v0.13 later reorganized the directory structure inside but retained its purpose as a cache. The local cache used to also serve as a record of specifically which packages were selected in a particular working directory, with the intent that a second run of "terraform init" would always select the same packages again. That meant that in some sense it behaved a bit like a local filesystem mirror directory, even though that wasn't its intended purpose. Due to some unfortunate miscommunications, somewhere a long the line we published some documentation that _recommended_ using the cache directory as if it were a filesystem mirror directory when working with Terraform Cloud. That was really only working as an accident of implementation details, and Terraform v0.14 is now going to break that because the source of record for the currently-selected provider versions is now the public-facing dependency lock file rather than the contents of an existing local cache directory on disk. After some consideration of how to move forward here, this commit implements a compromise that tries to avoid silently doing anything surprising while still giving useful guidance to folks who were previously using the unsupported strategy. Specifically: - The local cache directory will now be .terraform/providers rather than .terraform/plugins, because .terraform/plugins is effectively "poisoned" by the incorrect usage that we can't reliably distinguish from prior version correct usage. - The .terraform/plugins directory is now the "legacy cache directory". It is intentionally _not_ now a filesystem mirror directory, because that would risk incorrectly interpreting providers automatically installed by Terraform v0.13 as if they were a local mirror, and thus upgrades and checksum fetches from the origin registry would be blocked. - Because of the previous two points, someone who _was_ trying to use the legacy cache directory as a filesystem mirror would see installation fail for any providers they manually added to the legacy directory. To avoid leaving that user stumped as to what went wrong, there's a heuristic for the case where a non-official provider fails installation and yet we can see it in the legacy cache directory. If that heuristic matches then we'll produce a warning message hinting to move the provider under the terraform.d/plugins directory, which is a _correct_ location for "bundled" provider plugins that belong only to a single configuration (as opposed to being installed globally on a system). This does unfortunately mean that anyone who was following the incorrectly-documented pattern will now encounter an error (and the aforementioned warning hint) after upgrading to Terraform v0.14. This seems like the safest compromise because Terraform can't automatically infer the intent of files it finds in .terraform/plugins in order to decide automatically how best to handle them. The internals of the .terraform directory are always considered implementation detail for a particular Terraform version and so switching to a new directory for the _actual_ cache directory fits within our usual set of guarantees, though it's definitely non-ideal in isolation but okay when taken in the broader context of this problem, where the alternative would be silent misbehavior when upgrading.	6 years ago
Martin Atkins	2611e08430	command/init: Mention using the lock file for provider selection ... This probably isn't the best UI we could do here, but it's a placeholder for now just to avoid making it seem like we're ignoring the lock file and checking for new versions anyway.	6 years ago
Martin Atkins	b3f5c7f1e6	command/init: Read, respect, and update provider dependency locks ... This changes the approach used by the provider installer to remember between runs which selections it has previously made, using the lock file format implemented in internal/depsfile. This means that version constraints in the configuration are considered only for providers we've not seen before or when -upgrade mode is active.	6 years ago
Kristin Laemmert	d2e999ba1f	remove unused code (#26503) ... * remove unused code I've removed the provider-specific code under registry, and unused nil backend, and replaced a call to helper from backend/oss (the other callers of that func are provisioners scheduled to be deprecated). I also removed the Dockerfile, as our build process uses a different file. Finally I removed the examples directory, which had outdated examples and links. There are better, actively maintained examples available. * command: remove various unused bits * test wasn't running * backend: remove unused err	6 years ago
Pam Selle	1817c8ac3c	Merge pull request #26412 from OwenTuz/issue-26411-fix-failing-e2e-test-provider-not-found ... Fix bad string match that was causing TestInitProviderNotFound to fail	6 years ago
Martin Atkins	59b116f7bf	command/init: Remove support for legacy provider addresses ... We no longer need to support 0.12-and-earlier-style provider addresses because users should've upgraded their existing configurations and states on Terraform 0.13 already. For now this is only checked in the "init" command, because various test shims are still relying on the idea of legacy providers the core layer. However, rejecting these during init is sufficient grounds to avoid supporting legacy provider addresses in the new dependency lock file format, and thus sets the stage for a more severe removal of legacy provider support in a later commit.	6 years ago
Owen Tuz	5453147381	Fix bad string match that was causing TestInitProviderNotFound to fail on Macs ... The tmp directory path is longer on Macs than other systems and was wrapping across lines when printed, breaking the string match in the test. Fix suggested by @apparentlymart is to add two spaces before the leading '-' when printing to prevent the diagnostic renderer wrapping the line.	6 years ago
Martin Atkins	f53264d378	command/init: Better error message for provider unsupported platform ... As we add and remove support for different target platforms over time, there will be transition periods where the available platforms for each provider will be different than the available platforms for Terraform CLI itself. In recognition of that possibility, here we add a more specialized error message for that situation which tries to explain the problem a little more clearly than the generic error message that came before it. In an ideal world we'd be able to detect situations where a newer or older version has support in a similar vein to what we do with provider protocol incompatibilities, but we don't currently have logic to fetch the data necessary to implement that, so this is instead a presentation-only change aimed at allowing some possible near-future changes to the supported target platforms.	6 years ago
Martin Atkins	0b734a2803	command: Make provider installation interruptible ... In earlier commits we started to make the installation codepath context-aware so that it could be canceled in the event of a SIGINT, but we didn't complete wiring that through the API of the getproviders package. Here we make the getproviders.Source interface methods, along with some other functions that can make network requests, take a context.Context argument and act appropriately if that context is cancelled. The main providercache.Installer.EnsureProviderVersions method now also has some context-awareness so that it can abort its work early if its context reports any sort of error. That avoids waiting for the process to wind through all of the remaining iterations of the various loops, logging each request failure separately, and instead returns just a single aggregate "canceled" error. We can then set things up in the "terraform init" and "terraform providers mirror" commands so that the context will be cancelled if we get an interrupt signal, allowing provider installation to abort early while still atomically completing any local-side effects that may have started.	6 years ago
Paul Tyng	f3ff843ffd	Remove unused env var TF_SKIP_PROVIDER_VERIFY	6 years ago
Alisdair McDiarmid	9f824c53a5	command: Better in-house provider install errors ... When init attempts to install a legacy provider required by state and fails, but another provider with the same type is successfully installed, this almost definitely means that the user is migrating an in-house provider. The solution here is to use the `terraform state replace-provider` subcommand. This commit makes that next step clearer, by detecting this specific case, and displaying a list of commands to fix the existing state provider references.	6 years ago
Alisdair McDiarmid	fc7e467d19	command: Add redirect support to 0.13upgrade ... If a provider changes namespace in the registry, we can detect this when running the 0.13upgrade command. As long as there is a version matching the user's constraints, we now use the provider's new source address. Otherwise, warn the user that the provider has moved and a version upgrade is necessary to move to it.	6 years ago
Alisdair McDiarmid	f028b0a2bf	command: Fix backend config schema validation ... When applying a backend config override file, we must not check for the presence of all required fields, as the override can be a partial configuration. It is only valid to check for required fields after all overrides have been merged, which init already does.	6 years ago
Alisdair McDiarmid	c5d9935c0e	Merge pull request #25960 from hashicorp/alisdair/backend-config-override-fix ... command: Fix backend config override validation	6 years ago
Alisdair McDiarmid	677aabc767	command: Fix backend config override validation ... When loading a backend config override file, init was doing two things wrong: - First, if the file failed to parse, we accidentally didn't return, which caused a panic due to the parsed body being nil; - Secondly, we were overzealous with the validation of the file, allowing only attributes. While most backend configs are attributes only, the enhanced remote backend body also contains a `workspaces` block, which we need to support here. This commit fixes the first bug with an early return and adds test cases for missing file and intentionally-blank filename (to clear the config). We also add a schema validation for the backend block, based on the backend schema itself. This requires constructing an HCL body schema so that we can call `Content` and check for diagnostic errors. The result is more useful errors when an invalid backend config override file is used, while also supporting the enhanced remote backend config fully. Does not include tests specific to the remote backend, because the mocking involved to allow the backend to fully initialize is too involved to be worth it.	6 years ago
Alisdair McDiarmid	b239570abb	command: Always validate workspace name ... The workspace name can be overridden by setting a TF_WORKSPACE environment variable. If this is done, we should still validate the resulting workspace name; otherwise, we could end up with an invalid and unselectable workspace. This change updates the Meta.Workspace function to return an error, and handles that error wherever necessary.	6 years ago
Alisdair McDiarmid	53e587e1a6	Merge pull request #25504 from hashicorp/alisdair/post-install-cache-validate ... Add post-install provider cache validation and error reporting	6 years ago
Martin Atkins	7909dd318d	command/init: Specialized error message for host that isn't registry ... If the user specifies a host that isn't a provider registry in a provider source address then we'll print out some specialized error messages for different variants of that situation. In particular, this includes a special case for when the error is on the hostname "github.com", in anticipation of folks incorrectly attempting to use GitHub repository URLs (or Go-style module paths that happen to be on GitHub) to specify providers, so we can give a more specific hint about that. This is just a different presentation of an existing error case that we are already covering in the installer tests, so there are no new tests here. We could in principle have a test covering the exact text of these error messages, but we don't have much precedent for command package tests covering that level of cosmetic detail.	6 years ago
Martin Atkins	80ab867e57	command/init: Remove special 0.12upgrade heuristic ... For Terraform v0.12 we introduced a special loading mode where we would use the 0.11-syntax-compatible "earlyconfig" package as a heuristic to identify situations where it was likely that the user was trying to use 0.11-only syntax that the upgrade tool might help with. However, as the language has moved on that is no longer a suitable heuristic in Terraform 0.13 and later: other new additions to the language can cause the main loader to disagree with earlyconfig, which would lead us to give poor advice about how to respond. Instead, we'll now return the same generic "there are errors" message in all syntax error cases. We have an extra message for errors in this case (as compared to other commands) because "terraform init" is usually the first command a new user interacts with and so this message gives some extra explanation about what "terraform init" will do with the configuration once it's valid. This also includes a reset control character in the output of the message as part of our ongoing mission to stop Terraform printing out whole paragraphs of colored text, which can often be hard to read for various reasons.	6 years ago
Alisdair McDiarmid	87d1fb4006	command/init: Display provider validation errors ... After installing providers, we validate the presence of an executable file, and generate a selected versions lockfile. If this process fails, notify the user. One possible cause for this is an invalid provider package with a missing or misnamed executable file.	6 years ago
Kristin Laemmert	df244b87c2	command/init: return an error with invalid -backend-config files (#25411) ... * command/init: return an error with invalid -backend-config files The -backend-config flag expects a set of key-value pairs or a file containing key-value pairs. If the file instead contains a full backend configuration block, it was silently ignored. This commit adds a check for blocks in the file and returns an error if they are encountered. Fixes #24845 * emphasize backend configuration file in docs	6 years ago
Kristin Laemmert	47e657c611	internal/getproviders: decode and return any registry warnings (#25337) ... * internal/getproviders: decode and return any registry warnings The public registry may include a list of warnings in the "versions" response for any given provider. This PR adds support for warnings from the registry and an installer event to return those warnings to the user.	6 years ago
Alisdair McDiarmid	820ed48813	command: Fix init flags silent exit bug ... When using `-flag=value` with Powershell, unquoted values are broken into separate arguments. This means that the following command: terraform init -backend-config=./backend.conf is interpreted by Terraform as: terraform init -backend-config= ./backend.conf This results in an empty backend-config setting (which is semantically valid!) followed by a custom configuration path (pointing at a file). Due to a bug where we could exit without printing diagnostics, this would result in a silent failure that was very difficult to diagnose.	6 years ago
Alisdair McDiarmid	9263b28e99	command/init: Improve diags for legacy providers ... When initializing a configuration which refers to re-namespaced legacy providers, we attempt to detect this and display a diagnostic message. Previously this message would direct the user to run the 0.13upgrade command, but without specifying in which directories. This commit detects which modules are using the providers in question, and for local modules displays a list of upgrade commands which specify the source directories of these modules. For remote modules, we display a separate list noting that they need to be upgraded elsewhere, providing both the local module call name and the module source address.	6 years ago
Alisdair McDiarmid	ca40107066	command/init: Better diagnostics for provider 404s ... Fetching a default namespace provider from the public registry can result in 404 Not Found error. This might be caused by a previously- default provider moving to a new namespace, which means that the configuration needs to be upgraded to use an explicit provider source. This commit adds a more detailed diagnostic for this situation, suggesting that the intended provider might be in a new namespace. The recommended course of action is to run the 0.13upgrade command to generate the correct required_providers configuration.	6 years ago
Alisdair McDiarmid	ef28671b34	Merge pull request #24932 from hashicorp/signing-language ... Modify language for reporting signing state	6 years ago
Paul Tyng	22ef5cc99c	Modify language for reporting signing state ... Be more explicit about the signing status of fetched plugins and provide documentation about the different signing options.	6 years ago
Alisdair McDiarmid	62d826e066	command/init: Use full config for provider reqs ... Relying on the early config for provider requirements was necessary in Terraform 0.12, to allow the 0.12upgrade command to run after init installs providers. However in 0.13, the same restrictions do not apply, and the detection of provider requirements has changed. As a result, the early config loader gives incorrect provider requirements in some circumstances, such as those in the new test in this commit. Therefore we are changing the init command to use the requirements found by the full configuration loader. This also means that we can remove the internal initwd CheckCoreVersionRequirements function.	6 years ago
Kristin Laemmert	eead4c49fe	command/init: add e2e tests for provider not found messages	6 years ago