135 Commits (073fd0d183a6fa68965bdfcedbf459e2fbc78f8d)

Author	SHA1	Message	Date
Kristin Laemmert	df244b87c2	command/init: return an error with invalid -backend-config files (#25411) ... * command/init: return an error with invalid -backend-config files The -backend-config flag expects a set of key-value pairs or a file containing key-value pairs. If the file instead contains a full backend configuration block, it was silently ignored. This commit adds a check for blocks in the file and returns an error if they are encountered. Fixes #24845 * emphasize backend configuration file in docs	6 years ago
Kristin Laemmert	47e657c611	internal/getproviders: decode and return any registry warnings (#25337) ... * internal/getproviders: decode and return any registry warnings The public registry may include a list of warnings in the "versions" response for any given provider. This PR adds support for warnings from the registry and an installer event to return those warnings to the user.	6 years ago
Alisdair McDiarmid	820ed48813	command: Fix init flags silent exit bug ... When using `-flag=value` with Powershell, unquoted values are broken into separate arguments. This means that the following command: terraform init -backend-config=./backend.conf is interpreted by Terraform as: terraform init -backend-config= ./backend.conf This results in an empty backend-config setting (which is semantically valid!) followed by a custom configuration path (pointing at a file). Due to a bug where we could exit without printing diagnostics, this would result in a silent failure that was very difficult to diagnose.	6 years ago
Alisdair McDiarmid	9263b28e99	command/init: Improve diags for legacy providers ... When initializing a configuration which refers to re-namespaced legacy providers, we attempt to detect this and display a diagnostic message. Previously this message would direct the user to run the 0.13upgrade command, but without specifying in which directories. This commit detects which modules are using the providers in question, and for local modules displays a list of upgrade commands which specify the source directories of these modules. For remote modules, we display a separate list noting that they need to be upgraded elsewhere, providing both the local module call name and the module source address.	6 years ago
Alisdair McDiarmid	ca40107066	command/init: Better diagnostics for provider 404s ... Fetching a default namespace provider from the public registry can result in 404 Not Found error. This might be caused by a previously- default provider moving to a new namespace, which means that the configuration needs to be upgraded to use an explicit provider source. This commit adds a more detailed diagnostic for this situation, suggesting that the intended provider might be in a new namespace. The recommended course of action is to run the 0.13upgrade command to generate the correct required_providers configuration.	6 years ago
Alisdair McDiarmid	ef28671b34	Merge pull request #24932 from hashicorp/signing-language ... Modify language for reporting signing state	6 years ago
Paul Tyng	22ef5cc99c	Modify language for reporting signing state ... Be more explicit about the signing status of fetched plugins and provide documentation about the different signing options.	6 years ago
Alisdair McDiarmid	62d826e066	command/init: Use full config for provider reqs ... Relying on the early config for provider requirements was necessary in Terraform 0.12, to allow the 0.12upgrade command to run after init installs providers. However in 0.13, the same restrictions do not apply, and the detection of provider requirements has changed. As a result, the early config loader gives incorrect provider requirements in some circumstances, such as those in the new test in this commit. Therefore we are changing the init command to use the requirements found by the full configuration loader. This also means that we can remove the internal initwd CheckCoreVersionRequirements function.	6 years ago
Kristin Laemmert	eead4c49fe	command/init: add e2e tests for provider not found messages	6 years ago
Kristin Laemmert	8d28d73de3	getproviders: add a registry-specific error and modify output when a ... provider is not found. Previously a user would see the following error even if terraform was only searching the local filesystem: "provider registry registry.terraform.io does not have a provider named ...." This PR adds a registry-specific error type and modifies the MultiSource installer to check for registry errors. It will return the registry-specific error message if there is one, but if not the error message will list all locations searched.	6 years ago
Kristin Laemmert	a33a613703	command/init: add debug messages when -plugin-dir is set	6 years ago
Kristin Laemmert	862dc36884	command/init: include config loader diagnostics in output (#24921) ... Previously the diagnostics from the config loaders (earlyconfig and regular) were only appended to the overall diags if an error was found. This adds all diagnostics from the regular config loader so that any generated warnings will be displayed, even if there are no errors. I did not add the `earlyconfig` warnings since they will be displayed if there is an error and are likely to be duplicated by the config loader.	6 years ago
Kristin Laemmert	60321b41e8	getproviders: move protocol compatibility functions into registry client (#24846) ... * internal/registry source: return error if requested provider version protocols are not supported * getproviders: move responsibility for protocol compatibility checks into the registry client The original implementation had the providercache checking the provider metadata for protocol compatibility, but this is only relevant for the registry source so it made more sense to move the logic into getproviders. This also addresses an issue where we were pulling the metadata for every provider version until we found one that was supported. I've extended the registry client to unmarshal the protocols in `ProviderVersions` so we can filter through that list, instead of pulling each version's metadata.	6 years ago
Kristin Laemmert	0b76100da0	init: return proper config errors (#24865) ... Fixed a bug where we were returning earlyConfDiags instead of confDiags.	6 years ago
Kristin Laemmert	ce03f1255f	internal/providercache: fix error message for protocol mismatch (#24818) ... There was a bug in the installer trying to pass a nil error.	6 years ago
Alisdair McDiarmid	a5b3d497cc	internal: Verify provider signatures on install ... Providers installed from the registry are accompanied by a list of checksums (the "SHA256SUMS" file), which is cryptographically signed to allow package authentication. The process of verifying this has multiple steps: - First we must verify that the SHA256 hash of the package archive matches the expected hash. This could be done for local installations too, in the future. - Next we ensure that the expected hash returned as part of the registry API response matches an entry in the checksum list. - Finally we verify the cryptographic signature of the checksum list, using the public keys provided by the registry. Each of these steps is implemented as a separate PackageAuthentication type. The local archive installation mechanism uses only the archive checksum authenticator, and the HTTP installation uses all three in the order given. The package authentication system now also returns a result value, which is used by command/init to display the result of the authentication process. There are three tiers of signature, each of which is presented differently to the user: - Signatures from the embedded HashiCorp public key indicate that the provider is officially supported by HashiCorp; - If the signing key is not from HashiCorp, it may have an associated trust signature, which indicates that the provider is from one of HashiCorp's trusted partners; - Otherwise, if the signature is valid, this is a community provider.	6 years ago
Martin Atkins	297a3a5db9	command/init: Restore the unconstrained provider warnings ... When a provider dependency is implicit rather than explicit, or otherwise when version constraints are lacking, we produce a warning recommending the addition of explicit version constraints in the configuration. This restores the warning functionality from previous Terraform versions, adapting it slightly to account for the new provider FQN syntax and to recommend using a required_providers block rather than version constraints in "provider" blocks, because the latter is no longer recommended in the documentation.	6 years ago
Alisdair McDiarmid	b233aa39e9	addrs: Simplify presentation of provider FQNs ... The provider fully-qualified name string used in configuration is very long, and since most providers are hosted in the public registry, most of that length is redundant. This commit adds and uses a `ForDisplay` method, which simplifies the presentation of provider FQNs. If the hostname is the default hostname, we now display only the namespace and type. This is only used in UI, but should still be unambiguous, as it matches the FQN string parsing behaviour.	6 years ago
Martin Atkins	958ea4f7d1	internal/providercache: Handle built-in providers ... Built-in providers are special providers that are distributed as part of Terraform CLI itself, rather than being installed separately. They always live in the terraform.io/builtin/... namespace so it's easier to see that they are special, and currently there is only one built-in provider named "terraform". Previous commits established the addressing scheme for built-in providers. This commit makes the installer aware of them to the extent that it knows not to try to install them the usual way and it's able to report an error if the user requests a built-in provider that doesn't exist or tries to impose a particular version constraint for a built-in provider. For the moment the tests for this are the ones in the "command" package because that's where the existing testing infrastructure for this functionality lives. A later commit should add some more focused unit tests here in the internal/providercache package, too.	6 years ago
Martin Atkins	c4fb22863c	command/init: Support -plugin-dir again ... This is a slightly different approach than we used to take for this option: rather than disabling the installer and causing all future commands to look elsewhere for plugins, we'll now leave the installer enabled by constrain it to only look at the given directories. This is overall simpler because it doesn't require any special tracking of the plugin directories for subsequent commands. Instead, the selections file generated by the installer will record the versions it selected from the specified directories, and we'll link them in to the local cache just as we would normally so that other commands don't need to do anything special to select the right plugins in either case.	6 years ago
Martin Atkins	549aede792	Remove terraform.ResourceProvider, use providercache.Installer instead ... Back when we first introduced provider versioning in Terraform 0.10, we did the provider version resolution in terraform.NewContext because we weren't sure yet how exactly our versioning model was going to play out (whether different versions could be selected per provider configuration, for example) and because we were building around the limitations of our existing filesystem-based plugin discovery model. However, the new installer codepath is new able to do all of the selections up front during installation, so we don't need such a heavy inversion of control abstraction to get this done: the command package can select the exact provider versions and pass their factories directly to terraform.NewContext as a simple static map. The result of this commit is that CLI commands other than "init" are now able to consume the local cache directory and selections produced by the installation process in "terraform init", passing all of the selected providers down to the terraform.NewContext function for use in implementing the main operations. This commit is just enough to get the providers passing into the terraform.Context. There's still plenty more to do here, including to repair all of the tests this change has additionally broken.	6 years ago
Martin Atkins	f113a7c22d	command/init: Collect provider dependencies using our new helpers ... This produces a value shaped the way the provider installer expects without the need for further flattening and preprocessing.	6 years ago
Martin Atkins	e6df3905c9	command/init: Generate progress output during provider installation ... This restores some sequential event log output similar to what we had in the previous implementation of plugin installation.	6 years ago
Martin Atkins	94e1ac2d07	command: Minimal integration of new provider installer in "init" ... There's still a lot of work to do here around both the UX and the follow-up steps that need to happen after installation completes, but this is enough to faciliate some initial end-to-end testing of the new-style install process.	6 years ago
Alisdair McDiarmid	67203dade8	command: Simplify Meta.process helper method ... After some refactoring, this helper method had an unused argument (vars) and an always-nil error return value. This commit cleans this up.	6 years ago
Kristin Laemmert	4f141ae365	rename provider-specific functions (#24417) ... missingPlugins was hard-coded to work only with provider plugins, so I renamed it to clarify the usage. Also renamed a test provider from greater_than to greater-than as the underscore is an invalid provider name character and this will become a hard error in the near future.	6 years ago
Kristin Laemmert	5f313a65ad	command: remove 0.12upgrade (#24403) ... * command: remove 0.12upgrade and related `configupgrade` library * leave deprecation warning for 0.12upgrade to point users to v0.12	6 years ago
Martin Atkins	f899f5aa42	command/init: new error message for signature verification failure ... Our initial Terraform 0.13.0 release will continue to support only the hard-coded official HashiCorp signing key, with support for other keys to follow in a later release once the trust infrastructure is in place to support that. This change is intended to (marginally) improve the UX for a possible future situation where a HashiCorp-distributed provider makes a released signed with a new key and a prior version of Terraform ends up trying to install it due to incorrect version constraints. With this new text we hope to give the user a better prompt for onward troubleshooting, but in a sort of hedging way because we have not yet finalized the details of how new keys might roll out in practice. Hopefully a user seeing this message would consult the release notes for Terraform itself and for the provider in question and find some as-yet-undetermined information about how to proceed. If the decentralized trust model design comes together before the v0.13.0 release then we may make further amendments here to prepare for that, but that work should not block the v0.13.0 release if other work concludes first.	6 years ago
Kristin Laemmert	4d8fde3d6f	command: use backend config from state when backend=false is used. (#23802) ... * command: use backend config from state when backend=false is used. When a user runs `terraform init --backend=false`, terraform should inspect the state for a previously-configured backend, and use that backend, ignoring any backend config in the current configuration. If no backend is configured or there is no state, return a local backend. Fixes #16593	6 years ago
Kristin Laemmert	f8a23da480	explicitly create legacy-style provider	6 years ago
Kristin Laemmert	efafadbe5e	command: rename choosePlugins to chooseProviders to clarify scope of function	6 years ago
Kristin Laemmert	9891d0354a	providers: use addrs.Provider as map keys for provider.Factory (#23548) ... * terraform/context: use new addrs.Provider as map key in provider factories * added NewLegacyProviderType and LegacyString funcs to make it explicit that these are temporary placeholders This PR introduces a new concept, provider fully-qualified name (FQN), encapsulated by the `addrs.Provider` struct.	6 years ago
Kristin Laemmert	6728e521c1	addrs: rename Provider Name to more accurate Provider Type (#23449)	6 years ago
Martin Atkins	39e609d5fd	vendor: switch to HCL 2.0 in the HCL repository ... Previously we were using the experimental HCL 2 repository, but now we'll shift over to the v2 import path within the main HCL repository as part of actually releasing HCL 2.0 as stable. This is a mechanical search/replace to the new import paths. It also switches to the v2.0.0 release of HCL, which includes some new code that Terraform didn't previously have but should not change any behavior that matters for Terraform's purposes. For the moment the experimental HCL2 repository is still an indirect dependency via terraform-config-inspect, so it remains in our go.sum and vendor directories for the moment. Because terraform-config-inspect uses a much smaller subset of the HCL2 functionality, this does still manage to prune the vendor directory a little. A subsequent release of terraform-config-inspect should allow us to completely remove that old repository in a future commit.	7 years ago
Kristin Laemmert	120bb0a66c	plugin/discovery: use new addrs.ProviderType in place of a provider typeName string (#22724) ... This is a relatively small change meant to lay the foundation for future enhancements to providers' address.	7 years ago
Kristin Laemmert	a16e1fc0a1	command/init: omit a warning if -backend-config is used with no backend (#22164) ... * command/init: omit a warning if -backend-config is used with no backend block Terraform would silently accept - and swallow - `-backend-config` on the CLI when there was no `backend` block. Since it is mostly expected to override existing backend configuration, terraform should omit a warning if there is no backend configuration to override. If the user intended to override the default (local) backend configuration, they can first add a `backend` block to the `terraform` block to silence the warning (or just ignore it): ```hcl terraform { backend "local" {} } ```	7 years ago
Alex Pilon	0450f487fa	move IsEmptyDir to configs package	7 years ago
James Bardin	06dfc4abd8	allow setting -backend-config='' to unset override ... There is currently no way to unset -backend-config during init, since not setting that option assumes the user will use the saved config. Allow setting `-backend-config=""` to specify no overrides.	7 years ago
Pam Selle	346e341ff2	Only display status link if public registry used ... Updates to throw a specific error if using the defined public registry vs. another registry.	7 years ago
Pam Selle	ff7245f27c	Add status link to make message more helpful	7 years ago
Pam Selle	d72456d188	Add friendly error for when registry unresponsive ... If the registry is unresponsive, you will now get an error specific to this, rather than a misleading "provider unavailable" type error. Also adds debug logging for when errors like this may occur	7 years ago
findkim	161fe47b34	plugin/discovery: improve providery discovery verification errors	7 years ago
Justin Campbell	24e13d8ec1	plugin/discovery: Return tfdiags from Get ... Allows us to surface warnings to the user using the tfdiags interfaces.	7 years ago
James Bardin	c814f2da37	Change backend.ValidateConfig to PrepareConfig ... This mirrors the change made for providers, so that default values can be inserted into the config by the backend implementation. This is only the interface and method name changes, it does not yet add any default values.	7 years ago
findkim	7816e61614	Bump installer protocol version to 5 and separate client and server protocol references	7 years ago
Kim Ngo	41a2376915	Merge pull request #19981 from findkim/colorize-protocol-mismatch-msg ... Condense protocol mismatch UI error message	7 years ago
findkim	6e0de3e3f5	Nest incompatible provider protocol error to include dynamic custom msg	7 years ago
Martin Atkins	86c02d5c35	command: "terraform init" can partially initialize for 0.12upgrade ... There are a few constructs from 0.11 and prior that cause 0.12 parsing to fail altogether, which previously created a chicken/egg problem because we need to install the providers in order to run "terraform 0.12upgrade" and thus fix the problem. This changes "terraform init" to use the new "early configuration" loader for module and provider installation. This is built on the more permissive parser in the terraform-config-inspect package, and so it allows us to read out the top-level blocks from the configuration while accepting legacy HCL syntax. In the long run this will let us do version compatibility detection before attempting a "real" config load, giving us better error messages for any future syntax additions, but in the short term the key thing is that it allows us to install the dependencies even if the configuration isn't fully valid. Because backend init still requires full configuration, this introduces a new mode of terraform init where it detects heuristically if it seems like we need to do a configuration upgrade and does a partial init if so, before finally directing the user to run "terraform 0.12upgrade" before running any other commands. The heuristic here is based on two assumptions: - If the "early" loader finds no errors but the normal loader does, the configuration is likely to be valid for Terraform 0.11 but not 0.12. - If there's already a version constraint in the configuration that excludes Terraform versions prior to v0.12 then the configuration is probably _already_ upgraded and so it's just a normal syntax error, even if the early loader didn't detect it. Once the upgrade process is removed in 0.13.0 (users will be required to go stepwise 0.11 -> 0.12 -> 0.13 to upgrade after that), some of this can be simplified to remove that special mode, but the idea of doing the dependency version checks against the liberal parser will remain valuable to increase our chances of reporting version-based incompatibilities rather than syntax errors as we add new features in future.	7 years ago
Sander van Harmelen	ef9054562e	commands: make sure the correct flagset is used ... A lot of commands used `c.Meta.flagSet()` to create the initial flagset for the command, while quite a few of them didn’t actually use or support the flags that are then added. So I updated a few commands to use `flag.NewFlagSet()` instead to only add the flags that are actually needed/supported. Additionally this prevents a few commands from using locking while they actually don’t need locking (as locking is enabled as a default in `c.Meta.flagSet()`.	7 years ago
Martin Atkins	544c2932ce	command: Fix TestInit_checkRequiredVersion ... In prior refactoring we lost the required core version check from "terraform init", which we restore here. Additionally, this test used to have an incorrect name that suggested it was testing something in the "getProvider" codepath, but version checking happens regardless of what other options are selected.	7 years ago