27 Commits (fc8a76a2a3a2ce0d318cdb2e6f3f76ebbc6b28ac)

Author	SHA1	Message	Date
James Bardin	bc1a841d65	Merge pull request #26665 from hashicorp/jbardin/logging ... Restore "crash.log" behavior and remove prefixedio	6 years ago
James Bardin	ef4fee5acb	warnings are now on stderr	6 years ago
Kristin Laemmert	9d623290f4	command/e2etest: do not leave bad directories behind ... If you run the e2etests locally and use a configured plugin_cache_dir, the test will leave a bad directory behind in your cache dir that causes later `init`s to fail. To circumvent this, pass an explicity-empty CLI config file. This is a nicety for local developers and not necessarily required, but it happens to me often enough that I'd like to fix it. It's probably not a *bad* idea to pass an explicit cli config to all e2etests, honestly, but this is the only one that causes active problems so I limited this PR to that one test. Here's the error which occurs on subsequent `init` if this test is run on a machine that uses a plugin cache dir: 2020/10/13 10:41:05 [TRACE] providercache.fillMetaCache: error while scanning directory /Users/mildwonkey/.terraform.d/plugin-cache: failed to read metadata about /Users/mildwonkey/.terraform.d/plugin-cache/example.com/awesomecorp/happycloud/1.2.0/darwin_amd64: stat /Users/mildwonkey/.terraform.d/plugin-cache/example.com/awesomecorp/happycloud/1.2.0/darwin_amd64: no such file or directory	6 years ago
Martin Atkins	e70ab09bf1	command: new cache directory .terraform/providers for providers ... Terraform v0.10 introduced .terraform/plugins as a cache directory for automatically-installed plugins, Terraform v0.13 later reorganized the directory structure inside but retained its purpose as a cache. The local cache used to also serve as a record of specifically which packages were selected in a particular working directory, with the intent that a second run of "terraform init" would always select the same packages again. That meant that in some sense it behaved a bit like a local filesystem mirror directory, even though that wasn't its intended purpose. Due to some unfortunate miscommunications, somewhere a long the line we published some documentation that _recommended_ using the cache directory as if it were a filesystem mirror directory when working with Terraform Cloud. That was really only working as an accident of implementation details, and Terraform v0.14 is now going to break that because the source of record for the currently-selected provider versions is now the public-facing dependency lock file rather than the contents of an existing local cache directory on disk. After some consideration of how to move forward here, this commit implements a compromise that tries to avoid silently doing anything surprising while still giving useful guidance to folks who were previously using the unsupported strategy. Specifically: - The local cache directory will now be .terraform/providers rather than .terraform/plugins, because .terraform/plugins is effectively "poisoned" by the incorrect usage that we can't reliably distinguish from prior version correct usage. - The .terraform/plugins directory is now the "legacy cache directory". It is intentionally _not_ now a filesystem mirror directory, because that would risk incorrectly interpreting providers automatically installed by Terraform v0.13 as if they were a local mirror, and thus upgrades and checksum fetches from the origin registry would be blocked. - Because of the previous two points, someone who _was_ trying to use the legacy cache directory as a filesystem mirror would see installation fail for any providers they manually added to the legacy directory. To avoid leaving that user stumped as to what went wrong, there's a heuristic for the case where a non-official provider fails installation and yet we can see it in the legacy cache directory. If that heuristic matches then we'll produce a warning message hinting to move the provider under the terraform.d/plugins directory, which is a _correct_ location for "bundled" provider plugins that belong only to a single configuration (as opposed to being installed globally on a system). This does unfortunately mean that anyone who was following the incorrectly-documented pattern will now encounter an error (and the aforementioned warning hint) after upgrading to Terraform v0.14. This seems like the safest compromise because Terraform can't automatically infer the intent of files it finds in .terraform/plugins in order to decide automatically how best to handle them. The internals of the .terraform directory are always considered implementation detail for a particular Terraform version and so switching to a new directory for the _actual_ cache directory fits within our usual set of guarantees, though it's definitely non-ideal in isolation but okay when taken in the broader context of this problem, where the alternative would be silent misbehavior when upgrading.	6 years ago
Martin Atkins	b3f5c7f1e6	command/init: Read, respect, and update provider dependency locks ... This changes the approach used by the provider installer to remember between runs which selections it has previously made, using the lock file format implemented in internal/depsfile. This means that version constraints in the configuration are considered only for providers we've not seen before or when -upgrade mode is active.	6 years ago
Pam Selle	1817c8ac3c	Merge pull request #26412 from OwenTuz/issue-26411-fix-failing-e2e-test-provider-not-found ... Fix bad string match that was causing TestInitProviderNotFound to fail	6 years ago
Martin Atkins	59b116f7bf	command/init: Remove support for legacy provider addresses ... We no longer need to support 0.12-and-earlier-style provider addresses because users should've upgraded their existing configurations and states on Terraform 0.13 already. For now this is only checked in the "init" command, because various test shims are still relying on the idea of legacy providers the core layer. However, rejecting these during init is sufficient grounds to avoid supporting legacy provider addresses in the new dependency lock file format, and thus sets the stage for a more severe removal of legacy provider support in a later commit.	6 years ago
Owen Tuz	5453147381	Fix bad string match that was causing TestInitProviderNotFound to fail on Macs ... The tmp directory path is longer on Macs than other systems and was wrapping across lines when printed, breaking the string match in the test. Fix suggested by @apparentlymart is to add two spaces before the leading '-' when printing to prevent the diagnostic renderer wrapping the line.	6 years ago
Kristin Laemmert	47e657c611	internal/getproviders: decode and return any registry warnings (#25337) ... * internal/getproviders: decode and return any registry warnings The public registry may include a list of warnings in the "versions" response for any given provider. This PR adds support for warnings from the registry and an installer event to return those warnings to the user.	6 years ago
Kristin Laemmert	020084f6d0	update e2etests for windows compatibility	6 years ago
Alisdair McDiarmid	ca40107066	command/init: Better diagnostics for provider 404s ... Fetching a default namespace provider from the public registry can result in 404 Not Found error. This might be caused by a previously- default provider moving to a new namespace, which means that the configuration needs to be upgraded to use an explicit provider source. This commit adds a more detailed diagnostic for this situation, suggesting that the intended provider might be in a new namespace. The recommended course of action is to run the 0.13upgrade command to generate the correct required_providers configuration.	6 years ago
Kristin Laemmert	eead4c49fe	command/init: add e2e tests for provider not found messages	6 years ago
Martin Atkins	3167067029	command/e2etest: provider installation with explicit install methods ... This exercises the ability to customize the installation methods used by the provider plugin installer, in this case forcing the use of a custom local directory with a result essentially the same as what happens when you pass -plugin-dir to "terraform init".	6 years ago
Martin Atkins	92d6a30bb4	main: skip direct provider installation for providers available locally ... This more closely replicates the 0.12-and-earlier behavior, where having at least one version of a provider installed locally would totally disable any attempt to look for newer versions remotely. This is just for the implicit default behavior. Assumption is that later we'll have an explicit configuration mechanism that will allow the user to specify exactly where to look for what, and thus avoid tricky heuristics like this.	6 years ago
Martin Atkins	a6f63c4891	command/e2etest: update "init" tests for abbreviated provider addresses ... We're now longer showing the default registry hostname as part of addresses coming from that registry.	6 years ago
Martin Atkins	8c928e8358	main: Consult local directories as potential mirrors of providers ... This restores some of the local search directories we used to include when searching for provider plugins in Terraform 0.12 and earlier. The directory structures we are expecting in these are different than before, so existing directory contents will not be compatible without restructuring, but we need to retain support for these local directories so that users can continue to sideload third-party provider plugins until the explicit, first-class provider mirrors configuration (in CLI config) is implemented, at which point users will be able to override these to whatever directories they want. This also includes some new search directories that are specific to the operating system where Terraform is running, following the documented layout conventions of that platform. In particular, this follows the XDG Base Directory specification on Unix systems, which has been a somewhat-common request to better support "sideloading" of packages via standard Linux distribution package managers and other similar mechanisms. While it isn't strictly necessary to add that now, it seems ideal to do all of the changes to our search directory layout at once so that our documentation about this can cleanly distinguish "0.12 and earlier" vs. "0.13 and later", rather than having to document a complex sequence of smaller changes. Because this behavior is a result of the integration of package main with package command, this behavior is verified using an e2etest rather than a unit test. That test, TestInitProvidersVendored, is also fixed here to create a suitable directory structure for the platform where the test is being run. This fixes TestInitProvidersVendored.	6 years ago
Martin Atkins	de6c9ccec1	command/init: Move "vendored provider" test to e2etests ... In the new design the ProviderSource is decided by package main, not by the "command" package, and so making sure the vendor directory is included is the responsibility of that package instead. Therefore we can no longer test this at the "command" package level, but we'll retain a test for it in e2etests to record that it isn't currently working, so that we have a prompt to fix it before releasing.	6 years ago
Martin Atkins	f35ebe2d65	internal/providercache: Fix incorrect logic in Installer.SetGlobalCacheDir ... Due to some incomplete rework of this function in an earlier commit, the safety check for using the same directory as both the target and the cache was inverted and was raising an error _unless_ they matched, rather than _if_ they matched. This change is verified by the e2etest TestInitProviders_pluginCache, which is also updated to use the new-style cache directory layout as part of this commit.	6 years ago
Martin Atkins	14d456372a	command/e2etest: Update expected output for new plugin installer ... These tests make assertions against specific user-oriented output from the "terraform init" command, but we've intentionally changed some of these messages as part of introducing support for the decentralized provider namespace.	6 years ago
Martin Atkins	cf43663e85	command/e2etest: Fix TestInitProviders ... The canonical location of the "template" provider is now in the hashicorp namespace rather than the terraform-providers namespace, so the output has changed to reflect that.	7 years ago
Radek Simko	5b9f2fafc8	Standardise directory name for test data	7 years ago
Martin Atkins	eed605ac05	[WIP] Re-enable the end-to-end tests (#20044) ... * internal/initwd: Allow deprecated relative module paths In Terraform 0.11 we deprecated this form but didn't have any explicit warning for it. Now we'll still accept it but generate a warning. In a future major release we will drop this form altogether, since it is ambiguous with registry module source addresses. This codepath is covered by the command/e2etest suite. * e2e: Skip copying .exists file, if present We use this only in the "empty" test fixture in order to let git know that the directory exists. We need to skip copying it so that we can test "terraform init -from-module=...", which expects to find an empty directory. * command/e2etests: Re-enable and fix up the e2etest "acctests" We disabled all of the tests that accessed remote services like the Terraform Registry while they were being updated to support the new protocols we now expect. With those services now in place, we can re-enable these tests. Some details of exactly what output we print, etc, have intentionally changed since these tests were last updated. * e2e: refactor for modern states and plans * command/e2etest: re-enable e2etests and update for tf 0.12 compatibility plugin/discovery: mkdirAll instead of mkdir when creating cache dir	7 years ago
Martin Atkins	d4ee58ce59	Re-integrate the "terraform" provider into the main binary ... As part of the 0.10 core/provider split we moved this provider, along with all the others, out into its own repository. In retrospect, the "terraform" provider doesn't really make sense to be separated since it's just a thin wrapper around some core code anyway, and so re-integrating it into core avoids the confusion that results when Terraform Core and the terraform provider have inconsistent versions of the backend code and dependencies. There is no good reason to use a different version of the backend code in the provider than in core, so this new "internal provider" mechanism is stricter than the old one: it's not possible to use an external build of this provider at all, and version constraints for it are rejected as a result. This provider is also run in-process rather than in a child process, since again it's just a very thin wrapper around code that's already running in Terraform core anyway, and so the process barrier between the two does not create enough advantage to warrant the additional complexity.	9 years ago
James Bardin	53c8c1e208	e2e test for `init -from-module` ... Pull down the hashicorp/vault/aws module into the current directory with init.	9 years ago
Martin Atkins	30e6f4e066	command/e2etest: test installation of plugins from cache ... This is a tough one to unit tests because the behavior is tangled up in the code that hits releases.hashicorp.com, so we'll add this e2etest as some extra insurance that this works end-to-end.	9 years ago
Radek Simko	9e7e4ff4fb	e2e: Decouple logic for running e2e tests	9 years ago
Martin Atkins	52df81ee49	command/e2etest: test that we can install provider plugins ... We already have good tests for the business logic around provider installation, but the existing tests all stub out the main repository server. This test completes that coverage by verifying that the installer is able to run against the real repository and install an official release of the template provider.	9 years ago