terraform

History

Martin Atkins d0a35c60a7 providercache: Ignore lock-mismatching global cache entries When we originally introduced the trust-on-first-use checksum locking mechanism in v0.14, we had to make some tricky decisions about how it should interact with the pre-existing optional read-through global cache of provider packages: The global cache essentially conflicts with the checksum locking because if the needed provider is already in the cache then Terraform skips installing the provider from upstream and therefore misses the opportunity to capture the signed checksums published by the provider developer. We can't use the signed checksums to verify a cache entry because the origin registry protocol is still using the legacy ziphash scheme and that is only usable for the original zipped provider packages and not for the unpacked-layout cache directory. Therefore we decided to prioritize the existing cache directory behavior at the expense of the lock file behavior, making Terraform produce an incomplete lock file in that case. Now that we've had some real-world experience with the lock file mechanism, we can see that the chosen compromise was not ideal because it causes "terraform init" to behave significantly differently in its lock file update behavior depending on whether or not a particular provider is already cached. By robbing Terraform of its opportunity to fetch the official checksums, Terraform must generate a lock file that is inherently non-portable, which is problematic for any team which works with the same Terraform configuration on multiple different platforms. This change addresses that problem by essentially flipping the decision so that we'll prioritize the lock file behavior over the provider cache behavior. Now a global cache entry is eligible for use if and only if the lock file already contains a checksum that matches the cache entry. This means that the first time a particular configuration sees a new provider it will always be fetched from the configured installation source (typically the origin registry) and record the checksums from that source. On subsequent installs of the same provider version already locked, Terraform will then consider the cache entry to be eligible and skip re-downloading the same package. This intentionally makes the global cache mechanism subordinate to the lock file mechanism: the lock file must be populated in order for the global cache to be effective. For those who have many separate configurations which all refer to the same provider version, they will need to re-download the provider once for each configuration in order to gather the information needed to populate the lock file, whereas before they would have only downloaded it for the _first_ configuration using that provider. This should therefore remove the most significant cause of folks ending up with incomplete lock files that don't work for colleagues using other platforms, and the expense of bypassing the cache for the first use of each new package with each new configuration. This tradeoff seems reasonable because otherwise such users would inevitably need to run "terraform providers lock" separately anyway, and that command _always_ bypasses the cache. Although this change does decrease the hit rate of the cache, if we subtract the never-cached downloads caused by "terraform providers lock" then this is a net benefit overall, and does the right thing by default without the need to run a separate command.		3 years ago
..
addrs	addrs: ModuleSourceRemote.String correctly handles query string in URL	4 years ago
backend	S3 Backend : Bucket key should not contain trailing slash	3 years ago
builtin	add simple error indicating backend removal	4 years ago
checks	core: Propagate check results accurately from plan to apply	4 years ago
cloud	Add tests for cloud backend taskStage	3 years ago
command	providercache: Ignore lock-mismatching global cache entries	3 years ago
communicator	…
configs	Do not apply type defaults to null values	4 years ago
copy	…
dag	Clarify some comments in internal/dag	3 years ago
depsfile	Ignore existing package hashes for `providers lock` command (#31389 )	4 years ago
didyoumean	…
e2e	…
earlyconfig	tfdiags: Expose the "extra information" concept from HCL	4 years ago
experiments	build: Use Go 1.19	4 years ago
getmodules	add XTerraformGetLimit to prevent redirect loops	4 years ago
getproviders	build: Use Go 1.19	4 years ago
grpcwrap	s/Capabilities/ServerCapabilities/	4 years ago
helper/slowmessage	…
httpclient	backend/azurerm: removing ADAL support	4 years ago
initwd	expand module subdir globs	4 years ago
instances	…
ipaddr	build: Use Go 1.19	4 years ago
lang	lang/funcs: "timecmp" function	4 years ago
legacy	build: Use Go 1.19	4 years ago
logging	…
modsdir	…
moduledeps	…
moduletest	fixup broken test fixtures	4 years ago
plans	update UIMode comment	3 years ago
plugin	s/Capabilities/ServerCapabilities/	4 years ago
plugin6	s/Capabilities/ServerCapabilities/	4 years ago
provider-simple	s/Capabilities/ServerCapabilities/	4 years ago
provider-simple-v6	s/Capabilities/ServerCapabilities/	4 years ago
provider-terraform/main	…
providercache	providercache: Ignore lock-mismatching global cache entries	3 years ago
providers	s/Capabilities/ServerCapabilities/	4 years ago
provisioner-local-exec/main	…
provisioners	…
refactoring	allow cross-package move statements (#31556 )	4 years ago
registry	revert "run goimports"	4 years ago
repl	Unify all sensitive value plan output as "(sensitive value)"	3 years ago
replacefile	…
states	normalize empty CheckResults fields in stateV4	3 years ago
terminal	…
terraform	use UIMode instead of 0 changes to detect refresh	3 years ago
tfdiags	build: Use Go 1.19	4 years ago
tfplugin5	docs/plugin-protocol: Add notes about missing configuration in ReadResource and UpgradeResourceState request messages (#31998 )	3 years ago
tfplugin6	docs/plugin-protocol: Add notes about missing configuration in ReadResource and UpgradeResourceState request messages (#31998 )	3 years ago