15 Commits (a574e95dcde2d3bbb399b025f515ccd062abeeca)

Author	SHA1	Message	Date
Martin Atkins	d0a35c60a7	providercache: Ignore lock-mismatching global cache entries ... When we originally introduced the trust-on-first-use checksum locking mechanism in v0.14, we had to make some tricky decisions about how it should interact with the pre-existing optional read-through global cache of provider packages: The global cache essentially conflicts with the checksum locking because if the needed provider is already in the cache then Terraform skips installing the provider from upstream and therefore misses the opportunity to capture the signed checksums published by the provider developer. We can't use the signed checksums to verify a cache entry because the origin registry protocol is still using the legacy ziphash scheme and that is only usable for the original zipped provider packages and not for the unpacked-layout cache directory. Therefore we decided to prioritize the existing cache directory behavior at the expense of the lock file behavior, making Terraform produce an incomplete lock file in that case. Now that we've had some real-world experience with the lock file mechanism, we can see that the chosen compromise was not ideal because it causes "terraform init" to behave significantly differently in its lock file update behavior depending on whether or not a particular provider is already cached. By robbing Terraform of its opportunity to fetch the official checksums, Terraform must generate a lock file that is inherently non-portable, which is problematic for any team which works with the same Terraform configuration on multiple different platforms. This change addresses that problem by essentially flipping the decision so that we'll prioritize the lock file behavior over the provider cache behavior. Now a global cache entry is eligible for use if and only if the lock file already contains a checksum that matches the cache entry. This means that the first time a particular configuration sees a new provider it will always be fetched from the configured installation source (typically the origin registry) and record the checksums from that source. On subsequent installs of the same provider version already locked, Terraform will then consider the cache entry to be eligible and skip re-downloading the same package. This intentionally makes the global cache mechanism subordinate to the lock file mechanism: the lock file must be populated in order for the global cache to be effective. For those who have many separate configurations which all refer to the same provider version, they will need to re-download the provider once for each configuration in order to gather the information needed to populate the lock file, whereas before they would have only downloaded it for the _first_ configuration using that provider. This should therefore remove the most significant cause of folks ending up with incomplete lock files that don't work for colleagues using other platforms, and the expense of bypassing the cache for the first use of each new package with each new configuration. This tradeoff seems reasonable because otherwise such users would inevitably need to run "terraform providers lock" separately anyway, and that command _always_ bypasses the cache. Although this change does decrease the hit rate of the cache, if we subtract the never-cached downloads caused by "terraform providers lock" then this is a net benefit overall, and does the right thing by default without the need to run a separate command.	4 years ago
Martin Atkins	783a07d9e8	build: Use Go 1.19 ... Go 1.19's "fmt" has some awareness of the new doc comment formatting conventions and adjusts the presentation of the source comments to make it clearer how godoc would interpret them. Therefore this commit includes various updates made by "go fmt" to acheve that. In line with our usual convention that we make stylistic/grammar/spelling tweaks typically only when we're "in the area" changing something else anyway, I also took this opportunity to review most of the comments that this updated to see if there were any other opportunities to improve them.	4 years ago
James Bardin	553b8c6de5	expand module subdir globs	4 years ago
James Bardin	fd742cd544	add e2e test with provider schema capabilities ... enable destroy planning for the simple providers used in the e2e tests	4 years ago
Martin Atkins	096f0dc0da	build: Write the detected version number into the generated executable ... This also sets an additional variable if it detects that this is an alpha or development build, which currently does nothing but might eventually turn on the ability to use experimental features, if we make that something available only in prereleases.	4 years ago
Eng Zer Jun	fedd315275	test: use `T.TempDir` to create temporary test directory (#30803) ... This commit replaces `ioutil.TempDir` with `t.TempDir` in tests. The directory created by `t.TempDir` is automatically removed when the test and all its subtests complete. Prior to this commit, temporary directory created using `ioutil.TempDir` needs to be removed manually by calling `os.RemoveAll`, which is omitted in some tests. The error handling boilerplate e.g. defer func() { if err := os.RemoveAll(dir); err != nil { t.Fatal(err) } } is also tedious, but `t.TempDir` handles this for us nicely. Reference: https://pkg.go.dev/testing#T.TempDir Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>	4 years ago
Martin Atkins	b802db75d7	build: Build and run e2etest as part of the release build pipeline ... This uses the decoupled build and run strategy to run the e2etests so that we can arrange to run the tests against the real release packages produced elsewhere in this workflow, rather than ones generated just in time by the test harness. The modifications to make-archive.sh here make it more consistent with its originally-intended purpose of producing a harness for testing "real" release executables. Our earlier compromise of making it include its own terraform executable came from a desire to use that script as part of manual cross-platform testing when we weren't yet set up to support automation of those tests as we're doing here. That does mean, however, that the terraform-e2etest package content must be combined with content from a terraform release package in order to produce a valid contest for running the tests. We use a single job to cross-compile the test harness for all of the supported platforms, because that build is relatively fast and so not worth the overhead of matrix build, but then use a matrix build to actually run the tests so that we can run them in a worker matching the target platform. We currently have access only to amd64 (x64) runners in GitHub Actions and so for the moment this process is limited only to the subset of our supported platforms which use that architecture.	4 years ago
Alisdair McDiarmid	d42d83572b	cli: Fix backend init failure with deleted cache ... When an explicit backend is configured with a configuration which has not yet been initialized, running `terraform init` performs a state migration to fetch the remotely stored state in order to operate on it. Like the previous bug introduced by the recent provider diagnostics change, this code path was not correctly configured to enable init mode for the backend, which resulted in a fatal error during init when the cache dir is deleted. Setting the `Init` backend option allows this code path to continue without error when first initializing the backend for state migration. The new e2e test fails without this change.	5 years ago
Alisdair McDiarmid	1190b95fe2	command/e2etest: Ensure init fixes missing cache ... We test that a deleted provider cache results in an error when running terraform plan, but previously did not test that running init (as instructed) would resolve the issue. This (failing) e2e test adds that step.	5 years ago
Martin Atkins	01b22f4b76	command/e2etest: TestProviderTampering ... We have various mechanisms that aim to ensure that the installed provider plugins are consistent with the lock file and that the lock file is consistent with the provider requirements, and we do have existing unit tests for them, but all of those cases mock our fake out at least part of the process and in the past that's caused us to miss usability regressions, where we still catch the error but do so at the wrong layer and thus generate error message lacking useful additional context. Here we'll add some new end-to-end tests to supplement the existing unit tests, making sure things work as expected when we assemble the system together as we would in a release. These tests cover a number of different ways in which the plugin selections can grow inconsistent. These new tests all run only when we're in a context where we're allowed to access the network, because they exercise the real plugin installer codepath. We could technically build this to use a local filesystem mirror or other such override to avoid that, but the point here is to make sure we see the expected behavior in the main case, and so it's worth the small additional cost of downloading the null provider from the real registry.	5 years ago
Kristin Laemmert	4928e1dd01	terraform: use ProtocolVersion from unmanaged providers' reattachConfig to chose the correct PluginClient (#28190) ... * add/use ProtocolVersion with unmanaged providers reattach config	5 years ago
Martin Atkins	d35bc05312	Update various non-code references to renamed Go packages ... Previous commits have moved all of our Go packages under "internal", so this is a retroactive update of non-Go references to those same packages.	5 years ago
Martin Atkins	b40a4fb741	Move plugin/ and plugin6/ to internal/plugin{,6}/ ... This is part of a general effort to move all of Terraform's non-library package surface under internal in order to reinforce that these are for internal use within Terraform only. If you were previously importing packages under this prefix into an external codebase, you could pin to an earlier release tag as an interim solution until you've make a plan to achieve the same functionality some other way.	5 years ago
Martin Atkins	034e944070	Move plans/ to internal/plans/ ... This is part of a general effort to move all of Terraform's non-library package surface under internal in order to reinforce that these are for internal use within Terraform only. If you were previously importing packages under this prefix into an external codebase, you could pin to an earlier release tag as an interim solution until you've make a plan to achieve the same functionality some other way.	5 years ago
Martin Atkins	ffe056bacb	Move command/ to internal/command/ ... This is part of a general effort to move all of Terraform's non-library package surface under internal in order to reinforce that these are for internal use within Terraform only. If you were previously importing packages under this prefix into an external codebase, you could pin to an earlier release tag as an interim solution until you've make a plan to achieve the same functionality some other way.	5 years ago