1474 Commits (c2f1033e865bc689a9b08321a7bf5b83457e3ce4)

Author	SHA1	Message	Date
Kristin Laemmert	eead4c49fe	command/init: add e2e tests for provider not found messages	6 years ago
Kristin Laemmert	8d28d73de3	getproviders: add a registry-specific error and modify output when a ... provider is not found. Previously a user would see the following error even if terraform was only searching the local filesystem: "provider registry registry.terraform.io does not have a provider named ...." This PR adds a registry-specific error type and modifies the MultiSource installer to check for registry errors. It will return the registry-specific error message if there is one, but if not the error message will list all locations searched.	6 years ago
Kristin Laemmert	a33a613703	command/init: add debug messages when -plugin-dir is set	6 years ago
Kristin Laemmert	0d620018fe	provider cache: log errors and validate dir exists (#24993) ... * providercache: add logging for errors from getproviders.SearchLocalDirectory providercache.fillMetaCache() was silently swallowing errors when searching the cache directory. This commit logs the error without changing the behavior otherwise. * command/cliconfig: validate plugin cache dir exists The plugin cache directory must exist for terraform to use it, so we will add a check at the begining.	6 years ago
James Bardin	756e889697	remove unused DependsOn usage in state	6 years ago
Kristin Laemmert	862dc36884	command/init: include config loader diagnostics in output (#24921) ... Previously the diagnostics from the config loaders (earlyconfig and regular) were only appended to the overall diags if an error was found. This adds all diagnostics from the regular config loader so that any generated warnings will be displayed, even if there are no errors. I did not add the `earlyconfig` warnings since they will be displayed if there is an error and are likely to be duplicated by the config loader.	6 years ago
Kristin Laemmert	60321b41e8	getproviders: move protocol compatibility functions into registry client (#24846) ... * internal/registry source: return error if requested provider version protocols are not supported * getproviders: move responsibility for protocol compatibility checks into the registry client The original implementation had the providercache checking the provider metadata for protocol compatibility, but this is only relevant for the registry source so it made more sense to move the logic into getproviders. This also addresses an issue where we were pulling the metadata for every provider version until we found one that was supported. I've extended the registry client to unmarshal the protocols in `ProviderVersions` so we can filter through that list, instead of pulling each version's metadata.	6 years ago
Alisdair McDiarmid	82ebbf9a23	command: Add prompt & confirmation to 0.13upgrade	6 years ago
Alisdair McDiarmid	62b0cbed12	internal: Fix LookupLegacyProvider ... When looking up the namespace for a legacy provider source, we need to use the /v1/providers/-/{name}/versions endpoint. For non-HashiCorp providers, the /v1/providers/-/{name} endpoint returns a 404. This commit updates the LegacyProviderDefaultNamespace method and the mock registry servers accordingly.	6 years ago
Alisdair McDiarmid	1fdcbc4825	command: Fix 0.13upgrade bug with multiple blocks ... If a configuration had multiple blocks in the versions.tf file, it would be added to the `rewritePaths` list multiple times. We would then remove it from this slice, but only once, and so the output file would later be rewritten to remove the required providers block. This commit uses a set instead of a list to prevent this case, and adds a regression test.	6 years ago
Alisdair McDiarmid	e2be704d81	command: Extract reused hclwrite helper code	6 years ago
Alisdair McDiarmid	a740b739e0	command: Change 0.13upgrade default to versions.tf ... Instead of using providers.tf as the default output file for the upgrader, we now default to versions.tf. This means that if the configuration has no `required_providers` blocks at all, or has multiple, the provider version requirements will be stored in the versions.tf file. We now also update the versions.tf file to set a `required_version` attribute in the first `terraform` block, with value ">= 0.13". This is similar to the behaviour of the 0.12upgrade command, and signals that the configuration should not be used with older versions of Terraform.	6 years ago
Alisdair McDiarmid	01a3376ead	command: Check required_version before upgrading ... If a configuration has a version constraint which prevents use with Terraform 0.13, the upgrade command should exit before making any changes.	6 years ago
Alisdair McDiarmid	ae98bd12a7	command: Rework 0.13upgrade sub-command ... This commit implements most of the intended functionality of the upgrade command for rewriting configurations. For a given module, it makes a list of all providers in use. Then it attempts to detect the source address for providers without an explicit source. Once this step is complete, the tool rewrites the relevant configuration files. This results in a single "required_providers" block for the module, with a source for each provider. Any providers for which the source cannot be detected (for example, unofficial providers) will need a source to be defined by the user. The tool writes an explanatory comment to the configuration to help with this.	6 years ago
Mike Morris	9568de6b90	command: Fix OAuth2 PKCE arguments ... Providers like Okta and AWS Cognito expect that the PKCE challenge uses base64 URL Encoding without any padding (base64.RawURLEncoding) Additionally, Okta strictly adheres to section 4.2 of RFC 7636 and requires that the unencoded key for the PKCE data is at least 43 characters in length.	6 years ago
Kristin Laemmert	0b76100da0	init: return proper config errors (#24865) ... Fixed a bug where we were returning earlyConfDiags instead of confDiags.	6 years ago
James Bardin	9debd341bc	Merge pull request #24807 from hashicorp/jbardin/remove-each-mode ... Remove EachMode from state	6 years ago
Kristin Laemmert	ce03f1255f	internal/providercache: fix error message for protocol mismatch (#24818) ... There was a bug in the installer trying to pass a nil error.	6 years ago
James Bardin	2bfaddcf57	fix state mv to work without EachMode ... The only situation where `state mv` needs to understand the each mode is when with resource addresses that may reference a single instance, or a group of for_each or count instances. In this case we can differentiate the two by checking the existence of the NoKey instance key.	6 years ago
James Bardin	15a95031e5	remove a few traces of states.EachMode	6 years ago
Pam Selle	87bce5f9dd	Support reading module outputs in terraform console (#24808) ... * Include eval in output walk This allows outputs to be evaluated in the evalwalk, impacting terraform console. Outputs are still not evaluated for terraform console in the root module, so this has no impact on writing to state (as child module outputs are not written to state). Also adds test coverage to the console command, including for evaluating locals (another use of the evalwalk)	6 years ago
Alisdair McDiarmid	9266e944fa	Merge pull request #24752 from hashicorp/alisdair/terraform-state-replace-provider-docs ... website: Add docs for state replace-provider	6 years ago
Martin Atkins	1ce3c60693	command/cliconfig: Explicit provider installation method configuration ... This set of commits allows explicit configuration of provider installation methods in the CLI config, overriding the implicit method selections.	6 years ago
Martin Atkins	622abf707d	command/cliconfig: Remove redundant struct types ... These were being used in an earlier iteration of the provider installation configuration but it was all collapsed down into a single ProviderInstallationMethod type later, making these redundant.	6 years ago
Martin Atkins	6b2050f42a	main: Properly handle provider installation method exclusions ... Previously we were incorrectly using the Include configuration for both the include and exclude list, making the include portion totally ineffective.	6 years ago
Martin Atkins	3167067029	command/e2etest: provider installation with explicit install methods ... This exercises the ability to customize the installation methods used by the provider plugin installer, in this case forcing the use of a custom local directory with a result essentially the same as what happens when you pass -plugin-dir to "terraform init".	6 years ago
Martin Atkins	c7fe6b9160	command/cliconfig: handle provider_installation block in JSON syntax ... The CLI config can be written in both native HCL and HCL JSON syntaxes, so the provider_installation block must be expressible using JSON too. Our previous checks to approximate HCL 2-level strictness were too strict for HCL JSON where things are more ambiguous even in HCL 2, so this includes some additional relaxations if we detect that we're decoding an AST produced from a JSON file. This is still subject to the quirky ways HCL 1 handles JSON though, so the JSON value must be structured in a way that doesn't trigger HCL's heuristics that try to guess what is a block and what is an attribute. (This is the issue that HCL 2 fixes by always decoding using a schema; there's more context on this in: https://log.martinatkins.me/2019/04/25/hcl-json/ )	6 years ago
Martin Atkins	f5012c12da	command/cliconfig: Installation methods, not installation sources ... Unfortunately in the user model the noun "source" is already used for the argument in the required_providers block to specify which provider to use, so it's confusing to use the same noun to also refer to the method used to obtain that provider. In the hope of mitigating that confusion, here we use the noun "method", as in "installation method", to talk about the decision between getting a provider directly from its origin registry or getting it from some mirror. This is distinct from the provider's "source", which is the location where a provider _originates_ (prior to mirroring). This noun is also not super awesome, but better than overloading an existing term in the same feature.	6 years ago
Martin Atkins	e872ec4461	command/cliconfig: Remove remnant extraArg checks in provider_installation ... In the first pass of implementing this it was strict about what arguments are allowed inside source blocks, but that was counter to our usual design principles for CLI config where we tend to ignore unrecognized things to allow for some limited kinds of future expansion without breaking compatibility with older versions of Terraform that will be sharing the same CLI configuration files with newer versions. However, I'd removed the tracking of that prior to the initial commit. I missed some leftover parts when doing that removal, so this cleans up the rest of it.	6 years ago
Martin Atkins	8b75d1498f	command/cliconfig: Use existing HTTP mirror source rather than new stub ... An earlier commit added a redundant stub for a new network mirror source that was already previously stubbed as HTTPMirrorSource. This commit removes the unnecessary extra stub and changes the CLI config handling to use it instead. Along the way this also switches to using a full base URL rather than just a hostname for the mirror, because using the usual "Terraform-native service discovery" protocol here doesn't isn't as useful as in the places we normally use it (the mirror mechanism is already serving as an indirection over the registry protocol) and using a direct base URL will make it easier to deploy an HTTP mirror under a path prefix on an existing static file server.	6 years ago
Martin Atkins	b8856c677c	cliconfig: Ignore config dir if TF_CLI_CONFIG_FILE envvar is set ... When we originally introduced this environment variable it was intended to solve for the use-case where a particular invocation of Terraform needs a different CLI configuration than usual, such as if Terraform is being run as part of an automated test suite or other sort of automated situation with different needs than normal use. However, we accidentally had it only override the original singleton CLI config file, while leaving the CLI configuration directory still enabled. Now we'll take the CLI configuration out of the equation too, so that only the single specified configuration file and any other environment-sourced settings will be included.	6 years ago
Alisdair McDiarmid	52d64127f7	website: Add docs for state replace-provider	6 years ago
Kristin Laemmert	21b9da5a02	internal/providercache: verify that the provider protocol version is compatible (#24737) ... * internal/providercache: verify that the provider protocol version is compatible The public registry includes a list of supported provider protocol versions for each provider version. This change adds verification of support and adds a specific error message pointing users to the closest matching version.	6 years ago
Martin Atkins	2c535d829d	command/cliconfig: Decode provider_installation blocks ... This new CLI config block type allows explicitly specifying where Terraform should look to find provider plugins for installation. This is not used anywhere as of this commit, but in a future commit we'll change package main to treat the presence of a block of this type as a request to disable the default set of provider sources and use these explicitly- specified ones instead.	6 years ago
Kristin Laemmert	8108face36	terraform: return `initialization required` error when provider schemas not found (#24715) ... A side effect of the various changes to the provider installer included losing the initialization required error message which would occur if a user removed or modified the .terraform directory. Previously, plugin factories were created after the configuration was loaded, in terraform.NewContext. Terraform would compare the required providers (from config and state) to the available providers and return the aforementioned error if a provider was missing. Provider factories are now loaded at the beginning of any terraform command, before terraform even loads the configuration, and therefore before terraform has a list of required providers. This commit replaces the current error when a providers' schema cannot be found in the provider factories with the init error, and adds a command test (to plan tests, for no real reason other than that's what I thought of first).	6 years ago
Alisdair McDiarmid	e32e7e2c4b	Merge pull request #24617 from hashicorp/alisdair/provider-installer-signature-verification ... internal: Verify provider signatures on install	6 years ago
Martin Atkins	92d6a30bb4	main: skip direct provider installation for providers available locally ... This more closely replicates the 0.12-and-earlier behavior, where having at least one version of a provider installed locally would totally disable any attempt to look for newer versions remotely. This is just for the implicit default behavior. Assumption is that later we'll have an explicit configuration mechanism that will allow the user to specify exactly where to look for what, and thus avoid tricky heuristics like this.	6 years ago
Alisdair McDiarmid	a5b3d497cc	internal: Verify provider signatures on install ... Providers installed from the registry are accompanied by a list of checksums (the "SHA256SUMS" file), which is cryptographically signed to allow package authentication. The process of verifying this has multiple steps: - First we must verify that the SHA256 hash of the package archive matches the expected hash. This could be done for local installations too, in the future. - Next we ensure that the expected hash returned as part of the registry API response matches an entry in the checksum list. - Finally we verify the cryptographic signature of the checksum list, using the public keys provided by the registry. Each of these steps is implemented as a separate PackageAuthentication type. The local archive installation mechanism uses only the archive checksum authenticator, and the HTTP installation uses all three in the order given. The package authentication system now also returns a result value, which is used by command/init to display the result of the authentication process. There are three tiers of signature, each of which is presented differently to the user: - Signatures from the embedded HashiCorp public key indicate that the provider is officially supported by HashiCorp; - If the signing key is not from HashiCorp, it may have an associated trust signature, which indicates that the provider is from one of HashiCorp's trusted partners; - Otherwise, if the signature is valid, this is a community provider.	6 years ago
Kristin Laemmert	f09ae6f862	provider source tests: added test suite to exercise hyphenated providers (#24685)	6 years ago
Kristin Laemmert	269d511481	command/providers: refactor with new provider types and functions ... The providers command has been refactored to use the modern provider types and ProviderRequirements() functions. This resulted in a breaking change to the output: it no longer outputs the providers by module and no longer prints `(inherited)` or `(from state)` to show why a provider is included. We decided that at this time it was best to stick with the existing functions and make this change, but if we get feedback from the community we will revisit. Additional tests to exercise providers in modules and providers from state have been included.	6 years ago
Kristin Laemmert	0a5fb40fdf	configs: include provider configs in ProviderRequirements() ... This PR adds iteration through any provider configuration blocks in the config in addProviderRequirements(). A stale comment (of mine!) would leave one expecting the module.ProviderRequirements to include any requirements from provider configs. The comment was inaccurate and has been updated.	6 years ago
James Bardin	b84e6f7f95	remove race from closed-over err variable	6 years ago
Martin Atkins	297a3a5db9	command/init: Restore the unconstrained provider warnings ... When a provider dependency is implicit rather than explicit, or otherwise when version constraints are lacking, we produce a warning recommending the addition of explicit version constraints in the configuration. This restores the warning functionality from previous Terraform versions, adapting it slightly to account for the new provider FQN syntax and to recommend using a required_providers block rather than version constraints in "provider" blocks, because the latter is no longer recommended in the documentation.	6 years ago
Martin Atkins	a6f63c4891	command/e2etest: update "init" tests for abbreviated provider addresses ... We're now longer showing the default registry hostname as part of addresses coming from that registry.	6 years ago
Kristin Laemmert	5852a5c33d	Mildwonkey/last tests (#24553) ... * show text fixture update * temporarily disable providers tests	6 years ago
Alisdair McDiarmid	0f5a38b384	internal: Fix init provider lockfile test ... The fake installable package meta used a ZIP archive which gave different checksums between macOS and Linux targets. This commit removes the target from the contents of this archive, and updates the golden hash value in the test to match. This test should now pass on both platforms.	6 years ago
Alisdair McDiarmid	b233aa39e9	addrs: Simplify presentation of provider FQNs ... The provider fully-qualified name string used in configuration is very long, and since most providers are hosted in the public registry, most of that length is redundant. This commit adds and uses a `ForDisplay` method, which simplifies the presentation of provider FQNs. If the hostname is the default hostname, we now display only the namespace and type. This is only used in UI, but should still be unambiguous, as it matches the FQN string parsing behaviour.	6 years ago
Martin Atkins	8c928e8358	main: Consult local directories as potential mirrors of providers ... This restores some of the local search directories we used to include when searching for provider plugins in Terraform 0.12 and earlier. The directory structures we are expecting in these are different than before, so existing directory contents will not be compatible without restructuring, but we need to retain support for these local directories so that users can continue to sideload third-party provider plugins until the explicit, first-class provider mirrors configuration (in CLI config) is implemented, at which point users will be able to override these to whatever directories they want. This also includes some new search directories that are specific to the operating system where Terraform is running, following the documented layout conventions of that platform. In particular, this follows the XDG Base Directory specification on Unix systems, which has been a somewhat-common request to better support "sideloading" of packages via standard Linux distribution package managers and other similar mechanisms. While it isn't strictly necessary to add that now, it seems ideal to do all of the changes to our search directory layout at once so that our documentation about this can cleanly distinguish "0.12 and earlier" vs. "0.13 and later", rather than having to document a complex sequence of smaller changes. Because this behavior is a result of the integration of package main with package command, this behavior is verified using an e2etest rather than a unit test. That test, TestInitProvidersVendored, is also fixed here to create a suitable directory structure for the platform where the test is being run. This fixes TestInitProvidersVendored.	6 years ago
Martin Atkins	fcb8c53454	command/jsonconfig: Use correct provider address to access schema ... There was a remaining TODO in this package to find the true provider FQN when looking up the schema for a resource type. We now have that data available in the Provider field of configs.Resource, so we can now complete that change. The tests for this functionality actually live in the parent "command" package as part of the tests for the "terraform show" command, so this fix is verified by all of the TestShow... tests now passing except one, and that remaining one is failing for some other reason which we'll address in a later commit.	6 years ago
Martin Atkins	69cae48a11	command/import: fix TestImport_initializationErrorShouldUnlock ... This was checking for a specific output error message which has changed due to our new provider installer/selection approach.	6 years ago