20 Commits (94de72c63834acea811c27a585c19eae29374a4b)

Author	SHA1	Message	Date
Eng Zer Jun	fedd315275	test: use `T.TempDir` to create temporary test directory (#30803) ... This commit replaces `ioutil.TempDir` with `t.TempDir` in tests. The directory created by `t.TempDir` is automatically removed when the test and all its subtests complete. Prior to this commit, temporary directory created using `ioutil.TempDir` needs to be removed manually by calling `os.RemoveAll`, which is omitted in some tests. The error handling boilerplate e.g. defer func() { if err := os.RemoveAll(dir); err != nil { t.Fatal(err) } } is also tedious, but `t.TempDir` handles this for us nicely. Reference: https://pkg.go.dev/testing#T.TempDir Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>	4 years ago
Martin Atkins	23395a1022	providercache: Discard lock entries for unused providers ... Previously we would only ever add new lock entries or update existing ones. However, it's possible that over time a module may _cease_ using a particular provider, at which point we ought to remove it from the lock file so that operations won't fail when seeing that the provider cache directory is inconsistent with the lock file. Now the provider installer (EnsureProviderVersions) will remove any lock file entries that relate to providers not included in the given requirements, which therefore makes the resulting lock file properly match the set of packages the installer wrote into the cache. This does potentially mean that someone could inadvertently defeat the lock by removing a provider dependency, running "terraform init", then undoing that removal, and finally running "terraform init" again. However, that seems relatively unlikely compared to the likelihood of removing a provider and keeping it removed, and in the event it _did_ happen the changes to the lock entry for that provider would be visible in the diff of the provider lock file as usual, and so could be noticed in code review just as for any other change to dependencies.	4 years ago
Martin Atkins	df578afd7e	backend/local: Check dependency lock consistency before any operations ... In historical versions of Terraform the responsibility to check this was inside the terraform.NewContext function, along with various other assorted concerns that made that function particularly complicated. More recently, we reduced the responsibility of the "terraform" package only to instantiating particular named plugins, assuming that its caller is responsible for selecting appropriate versions of any providers that _are_ external. However, until this commit we were just assuming that "terraform init" had correctly selected appropriate plugins and recorded them in the lock file, and so nothing was dealing with the problem of ensuring that there haven't been any changes to the lock file or config since the most recent "terraform init" which would cause us to need to re-evaluate those decisions. Part of the game here is to slightly extend the role of the dependency locks object to also carry information about a subset of provider addresses whose lock entries we're intentionally disregarding as part of the various little edge-case features we have for overridding providers: dev_overrides, "unmanaged providers", and the testing overrides in our own unit tests. This is an in-memory-only annotation, never included in the serialized plan files on disk. I had originally intended to create a new package to encapsulate all of this plugin-selection logic, including both the version constraint checking here and also the handling of the provider factory functions, but as an interim step I've just made version constraint consistency checks the responsibility of the backend/local package, which means that we'll always catch problems as part of preparing for local operations, while not imposing these additional checks on commands that _don't_ run local operations, such as "terraform apply" when in remote operations mode.	5 years ago
Martin Atkins	3f85591998	depsfile: SaveLocksToBytes function ... In a future commit we'll use this to include dependency lock information in full fidelity inside a saved plan file.	5 years ago
Martin Atkins	b9a93a0fe7	Move addrs/ to internal/addrs/ ... This is part of a general effort to move all of Terraform's non-library package surface under internal in order to reinforce that these are for internal use within Terraform only. If you were previously importing packages under this prefix into an external codebase, you could pin to an earlier release tag as an interim solution until you've make a plan to achieve the same functionality some other way.	5 years ago
Martin Atkins	05caff2ca3	Move tfdiags/ to internal/tfdiags/ ... This is part of a general effort to move all of Terraform's non-library package surface under internal in order to reinforce that these are for internal use within Terraform only. If you were previously importing packages under this prefix into an external codebase, you could pin to an earlier release tag as an interim solution until you've make a plan to achieve the same functionality some other way.	5 years ago
Masayuki Morita	31a5aa1878	command/init: Add a new flag `-lockfile=readonly` (#27630) ... Fixes #27506 Add a new flag `-lockfile=readonly` to `terraform init`. It would be useful to allow us to suppress dependency lockfile changes explicitly. The type of the `-lockfile` flag is string rather than bool, leaving room for future extensions to other behavior variants. The readonly mode suppresses lockfile changes, but should verify checksums against the information already recorded. It should conflict with the `-upgrade` flag. Note: In the original use-case described in #27506, I would like to suppress adding zh hashes, but a test code here suppresses adding h1 hashes because it's easy for testing. Co-authored-by: Alisdair McDiarmid <alisdair@users.noreply.github.com>	5 years ago
Anthony Sottile	8cd72e51cb	fix file mode on lock file (#27205) ... Signed-off-by: Anthony Sottile <asottile@umich.edu>	5 years ago
Martin Atkins	21d6fb5a37	depsfile: Don't panic when lock file is unreadable ... Previously we were expecting that the *hcl.File would always be non-nil, even in error cases. That isn't always true, so now we'll be more robust about it and explicitly return an empty locks object in that case, along with the error diagnostics. In particular this avoids a panic in a strange situation where the user created a directory where the lock file would normally go. There's no meaning to such a directory, so it would always be a mistake and so now we'll return an error message about it, rather than panicking as before. The error message for the situation where the lock file is a directory is currently not very specific, but since it's HCL responsible for generating that message we can't really fix that at this layer. Perhaps in future we can change HCL to have a specialized error message for that particular error situation, but for the sake of this commit the goal is only to stop the panic and return a normal error message.	5 years ago
Martin Atkins	fc5a41b5e5	depsfile: Allow loading locks from a byte array in memory ... This won't be a typical usage pattern for normal code, but will be useful for tests that need to work with locks as input so that they don't need to write out a temporary file on disk just to read it back in immediately.	6 years ago
Alisdair McDiarmid	9576a5b2d8	internal: Fix lockfile constraint output for 1.2.* ... If a configuration requires a partial provider version (with some parts unspecified), Terraform considers this as a constrained-to-zero version. For example, a version constraint of 1.2 will result in an attempt to install version 1.2.0, even if 1.2.1 is available. When writing the dependency locks file, we previously would write 1.2.*, as this is the in-memory representation of 1.2. This would then cause an error on re-reading the locks file, as this is not a valid constraint format. Instead, we now explicitly convert the constraint to its zero-filled representation before writing the locks file. This ensures that it correctly round-trips. Because this change is made in getproviders.VersionConstraintsString, it also affects the output of the providers sub-command.	6 years ago
Martin Atkins	0009768c7f	internal/depsfile: Update the dependency lock file atomically ... In this case, "atomic" means that there will be no situation where the file contains only part of the newContent data, and therefore other software monitoring the file for changes (using a mechanism like inotify) won't encounter a truncated file. It does _not_ mean that there can't be existing filehandles open against the old version of the file. On Windows systems the write will fail in that case, but on Unix systems the write will typically succeed but leave the existing filehandles still pointing at the old version of the file. They'll need to reopen the file in order to see the new content.	6 years ago
Martin Atkins	e1cf0ac801	internal/depsfile: Control how the "hashes" value is formatted ... Previously we were just letting hclwrite do its default formatting behavior here. The current behavior there isn't ideal anyway -- it puts big data structures all on one line -- but even ignoring that our goal for this file format is to keep things in a highly-normalized shape so that diffs against the file are clear and easy to read. With that in mind, here we directly control how we write that value into the file, which means that later changes to hclwrite's list/set presentation won't affect it, regardless of what form they take.	6 years ago
Martin Atkins	b3f5c7f1e6	command/init: Read, respect, and update provider dependency locks ... This changes the approach used by the provider installer to remember between runs which selections it has previously made, using the lock file format implemented in internal/depsfile. This means that version constraints in the configuration are considered only for providers we've not seen before or when -upgrade mode is active.	6 years ago
Martin Atkins	4a1b081afb	depsfile: Locks.Equal and Locks.Empty methods ... These are helper functions to give the installation UI some hints about whether the lock file has changed so that it can in turn give the user advice about it. The UI-layer callers of these will follow in a later commit.	6 years ago
Martin Atkins	6694cfaa0e	getproviders: Add a real type Hash for package hashes ... The logic for what constitutes a valid hash and how different hash schemes are represented was starting to get sprawled over many different files and packages. Consistently with other cases where we've used named types to gather the definition of a particular string into a single place and have the Go compiler help us use it properly, this introduces both getproviders.Hash representing a hash value and getproviders.HashScheme representing the idea of a particular hash scheme. Most of this changeset is updating existing uses of primitive strings to uses of getproviders.Hash. The new type definitions are in internal/getproviders/hash.go.	6 years ago
Martin Atkins	264a3cf031	depsfile: Flatten the "hashes" locks to a single set of strings ... Although origin registries return specific [filename, hash] pairs, our various different installation methods can't produce a structured mapping from platform to hash without breaking changes. Therefore, as a compromise, we'll continue to do platform-specific checks against upstream data in the cases where that's possible (installation from origin registry or network mirror) but we'll treat the lock file as just a flat set of equally-valid hashes, at least one of which must match after we've completed whatever checks we've made against the upstream-provided checksums/signatures. This includes only the minimal internal/getproviders updates required to make this compile. A subsequent commit will update that package to actually support the idea of verifying against multiple hashes.	6 years ago
Martin Atkins	773dd56b42	internal/depsfile: Introduce the concept of "non-lockable" providers ... It doesn't make sense for a built-in provider to appear in a lock file because built-in providers have no version independent of the version of Terraform they are compiled into. We also exclude legacy providers here, because they were supported only as a transitional aid to enable the Terraform 0.13 upgrade process and are not intended for explicit selection. The provider installer will, once it's updated to understand dependency locking, use this concept to decide which subset of its selections to record in the dependency lock file for reference for future installation requests.	6 years ago
Martin Atkins	98e2e69abb	internal/depsfile: SaveLocksToFile implementation ... This is an initial implementation of writing locks back to a file on disk. This initial implementation is incomplete because it does not write the changes to the new file atomically. We'll revisit that in a later commit as we return to polish these codepaths, once we've proven out this package's design by integrating it with Terraform's provider installer.	6 years ago
Martin Atkins	92723661d0	internal/depsfile: Loading locks from HCL files on disk ... This is the initial implementation of the parser/decoder portion of the new dependency lock file handler. It's currently dead code because the caller isn't written yet. We'll continue to build out this functionality here until we have the basic level of both load and save functionality before introducing this into the provider installer codepath.	6 years ago