398 Commits (eba067c8bc11c53fc6ec94c5ed587401ac168bf4)

Author	SHA1	Message	Date
Martin Atkins	d3b8a55781	go.mod: go get google.golang.org/protobuf@v1.33.0 ... Many of the changes between these two versions seem to be in parts of the library that Terraform doesn't use, but there are a few sensitive parts: - Slight changes to details of how the library implements the JSON encoding of protobuf. Terraform CLI/Core don't use this, but some of the backends might use it indirectly when talking to their underlying APIs. - Some details of the "protoreflect" package for working with protobuf descriptors dynamically (rather than using generated stubs) have changed, but it seems pretty marginal and unlikely to cause problems.	2 years ago
Martin Atkins	84df5562f2	moduleaddrs: Terraform now owns module source address normalization ... We've been gradually chipping away at how much we use go-getter for source packages, because it's generally been a bit of a nightmare and sharing it with other codebases means that any time someone wants to change something we end up needing to find some way to prevent it breaking Terraform's compatibility promises. Here we make one further step: Terraform owns the "detectors" idea that deals with source address normalization, and now always produces fully-qualified addresses for go-getter to chew on only for the getting and decompressing steps. Retaining go-getter for the actual getting part is helpful because we can then benefit from security fixes upstream, but Terraform owning the first layer of parsing means that we can fix in place the definition of what "module source address" syntax means, and thus we can avoid having everything in this codebase indirectly depend on go-getter just because it wants to parse module source addresses. Now only the module installer actually depends indirectly on go-getter, which finally disconnects go-getter's subtree from all of the remote state backend dependency graphs.	2 years ago
Martin Atkins	78ae6af906	legacy/terraform: Minimize the contents of this legacy package ... At one point we were relying on this for a few different things, but now it's here exclusively to support legacy/helper/schema, the legacy SDK snapshot that is still being used by a few of the remote state backends. The SDK only really needs a tiny portion of the functionality from this package, and by deleting as much of the unused stuff as possible we can reduce the indirect dependencies of this package and thus, in turn, the indirect dependencies of the remote state backends. This also selectively deletes a few parts of the legacy SDK that are focused on providers and provisioners, because we know that we have this here only to support the backends that haven't yet been weaned off it, but we'll save a more adventurous purge of _that_ package for another day.	2 years ago
Martin Atkins	621befc3a2	configtesting: Separate package for the SynthBody helper ... This "SynthBody" helper doesn't really have anything to do with package configs except that it's sometimes useful for creating fake instances of the types in there for testing. Moving it into its own package means that package backend no longer needs to depend on package configs, since it doesn't actually have any need for parsing module configurations. The old configs.SynthBody function is still available as a forwarding alias, because most of the existing callers of it are from packages that already need to import package configs for other reasons, and so this reduces the need to churn loads of existing tests.	2 years ago
Martin Atkins	56006416ba	go.mod: go get golang.org/x/tools@v0.19.0 ... We use this just for development-time tools like "stringer", so this upgrade should not have any runtime impact.	2 years ago
Martin Atkins	c541757f4a	go.mod: go get golang.org/x/mod@v0.16.0 ... Aside from development tool dependencies we use this only for the directory hashing logic used to compute go module hashes, which we also use for Terraform provider hashes. There are no changes to the dirhash package in this update, so this should have no runtime impact.	2 years ago
Martin Atkins	b059985227	go.mod: go get golang.org/x/sync@v0.6.0 ... This update only changes a documentation comment, so cannot affect runtime behavior.	2 years ago
Martin Atkins	caa7a757bc	go.mod: go get golang.org/x/oauth2@v0.18.0 ... This only upgrades a Google-specific subpackage, and so the only potential impact is to the GCS backend.	2 years ago
Martin Atkins	9c7ece8ad4	go.mod: go get golang.org/x/net@v0.22.0 ... This update is largely focused on QUIC implementation changes, which do not directly affect Terraform. There is one fix for the HTTP2 implementation to terminate the stream if a sender recieves an out-of-bounds window update, which makes the implementation better match the specification. This seems unlikely to cause any practical problems for Terraform.	2 years ago
Martin Atkins	fad9c2a4cd	go.mod: go get golang.org/x/crypto@v0.21.0 ... This update is mostly internal changes that don't affect external behavior. The only update potentially significant to Terraform is the new exclusion of a particular root certificate that is used only for some specific names under the .tr CCTLD, but the certificate database doesn't specify that constraint in a way that the library can understand, so it's excluded to avoid making the trust for that certificate broader than intended.	2 years ago
Martin Atkins	a1d2169c85	go.mod: go get golang.org/x/term@v0.18.0 ... This update contains no real changes. It just updates transitive dependencies to newer versions that we've already selected anyway.	2 years ago
Martin Atkins	b3f6a6230d	go.mod: go get golang.org/x/sys@v0.18.0 ... This small update does not directly affect any Terraform functionality.	2 years ago
Martin Atkins	61a5c1f57b	go.mod: go get github.com/hashicorp/terraform-registry-address@v0.2.3 ... This is really just a bunch of changes to the upstream go.mod for updates to its own dependencies, but Terraform already has new enough versions of those dependencies for other reasons and so this changes nothing.	2 years ago
Martin Atkins	8603ee0ab5	go.mod: go get github.com/hashicorp/go-slug@v0.14.0 ... This has just some changes to the "sourcebundle" building API, which doesn't affect Terraform CLI or Terraform Core because they currently only consume source bundles, not build them.	2 years ago
Martin Atkins	914618b061	gcs: Adopt the "ReadPathOrContents" helper function ... Package backend previously had a ReadPathOrContents helper that takes a string that could represent either a filename or some literal content, and then reads the given file if it's a filename or just returns the literal content otherwise. We originally used variations of this helper to shim a number of different situations that had originally accepted filenames but later changed to accept literal content, but over time we've eliminated all of those and so the only remaining caller is the gcs backend which uses it to accept GCP credentials either as literal content or as a file containing the credentials. Therefore here we move that helper function to be just an unexported part of the gcs backend package. This is part of an ongoing effort to minimize what each remote state backend depends on (either directly or indirectly) so that we can make gradual progress towards these remote state backends being self-contained enough to move into other codebases. In the short term though, it just puts this helper function closer to its caller so it's easier to maintain them both together if needed.	2 years ago
Martin Atkins	f058de612c	statemgr: Incorporate IntermediateStateConditionalPersister ... This interface was originally in the local backend's package because that backend is the only one that _truly_ runs Terraform operations and thus needs to negotiate how to store the intermediate state snapshots that result from them. However, putting the interface and its supporting types/functions there means that anything that wants to implement this interface needs to import the local backend package, and thus indirectly depends on the entire modules runtime, etc. We already had package stagemgr as a home for the shared stuff used by most state storage managers, and so we'll just move these symbols over there and then we can shed some internal dependencies on package local. This also severs the last dependency link between the remote state backends and the modules runtime, allowing us to shed a bunch of indirect dependencies from those remote state backends.	2 years ago
Martin Atkins	cf3bbb890f	State managers use schemarepo.Schemas instead of terraform.Schemas ... schemarepo.Schemas used to be terraform.Schemas, but its presence in the terraform package was the only reason for a state manager to need to import package terraform, so we've moved it out into its own package in the parent commit. This means that none of the state-related packages need to import package terraform any longer, and so once we've untangled some other dependencies this will curtail the number of packages in this codebase that the remote state backends transitively depend on, as part of an ongoing effort to eventually separate them from this codebase entirely.	2 years ago
Martin Atkins	88cb4aeea3	backendrun: Separate the types/etc for backends that support operations ... We previously had all of the types and helpers for all kinds of backends together in package backend. That kept things relatively simple, but it also meant that the majority of backends that only deal with remote state storage ended up still indirectly depending on the entire Terraform modules runtime, configuration loader, etc, etc, which brings into scope a bunch of external dependencies that the remote state backends don't really need. Since backends that support operations are a rare exception, we'll move the types and helpers for those into a separate package "backendrun", and then the main package backend can have a much more modest set of types and, more importantly, a modest set of dependencies on other packages in this codebase. This is part of an ongoing effort to reduce the exposure of Terraform Core and CLI code to the remote backends and vice-versa, so that in the long run we can more often treat them as separate for dependency maintenance purposes.	2 years ago
Martin Atkins	5f2eab6944	providerreqs: Provider requirement/version types in their own package ... As we previously found with "package addrs", any package that contains types used as identifiers or addresses tends to get imported from all over, which means that if we mix real logic with those types in the same package then a lot more packages end up depending indirectly on our external dependencies. In this case, the getproviders package depends on various things we need only during provider installation, such as the library we use for GPG signature verification. Lots of different packages in Terraform need to talk about provider requirements, but only the CLI layer actually needs to do provider installation, and so this commit splits the types we use to talk about requirements into a new package "providerreqs", which package getproviders then depends on. To keep the size of this diff relatively small to start I've left forwarding aliases in package getproviders, which means that not all of the callers needed to be updated all at once. This commit does update all of the packages that the remote state backends depend on though, because that then allows us to reduce the set of indirect dependencies for each of those backends. The nested module go.mod files show that this has disconnected the indirect dependencies on github.com/ProtonMail/go-crypto and on github.com/cloudflare/circl , both of which Terraform CLI uses only for provider installation.	2 years ago
Martin Atkins	310331bdc6	backend/consul: go.mod updates from a merge conflict ... Two changes landed concurrently and caused these dependencies to get out of sync.	2 years ago
Martin Atkins	5811be26f8	go.mod: Separate modules at code ownership boundaries ... Historically it's been hard for us to keep on top of dependency upgrades as a matter of course because it's unclear which code owners might be affected by or responsible for an upgrade of each module. After trying a few different techniques to try to mitigate this, the most promising one seems to be to tell the Go toolchain that each of these components is a separate Go module, but then to stitch them all back together again using replace directives so that we can mostly ignore these module boundaries in everyday development. The one case where we _do_ need to pay attention to these boundaries is also the case where the separation is useful: upgrading a dependency of any one of these modules might potentially force upgrading it for another module too, and so the Go toolchain will help us notice that interaction and we can immediately which code owners might need to be involved in reviewing and testing that particular upgrade. To make that process less onerous, this adds a new makefile target "make syncdeps" which runs "go mod tidy" in all of the modules at once and thus forces the toolchain to align their dependencies to the extent required for them all to be linked together successfully, and to generate go.mod and/or go.sum diffs that will match our configured code ownership paths to trigger the appropriate code review requests. Hopefully before too long we'll be able to move all the backends out into provider plugins and delete the "legacy" module entirely and then this oddness won't be needed anymore, but as long as we're all trying to play together in this same sandbox this is a small amount of extra ceremony in return for hopefully reducing the level of anxiety involved in keeping the system's dependencies up to date more consistently.	2 years ago
Martin Atkins	6bd1c66cbc	backendbase: Basic support for default values ... Some existing backends make very heavy use of environment variable-based default values and fallback values, which are available in the legacy SDK but not in Terraform's own schema model. To ease the migration away from the legacy SDK for those backends, backendbase.Base now supports a lightweight mechanism for declaring such defaults, and applies them automatically as part of PrepareConfig. This gets us pretty close to the assumptions that the SDK allowed callers to make, but with considerably less code. It's intentionally considerably less flexible than the legacy SDK was, because in practice most backends use only a tiny subset of the SDK functionality anyway.	2 years ago
Martin Atkins	b16cb49fb6	backendbase: "SDK-like" helpers ... A lot of the remote state backends' current Configure functions unfortunately rely quite heavily on the "do the best you can" API of the legacy SDK, which makes questionable shortcuts like treating null as equivalent to the Go zero value of a type, but in return makes it convenient to assign into other data structures that happen to make similar assumptions. And of course that means that the backends' other data structures were built to make those similar assumptions! This new set of utilities therefore aims to provide a bare-minimum subset of "SDK-like" functionality to ease the initial migration away from the legacy SDK. It isn't exactly equivalent an in particular focuses only on the small subset of legacy SDK functionality that most existing backends rely on, but this should at least make the initial adaptations a little more mechanical so that we can continue to defer the full modernization of these backends for some undetermined future time without having to continue retaining the huge legacy SDK snapshot in this codebase.	2 years ago
Martin Atkins	a5d63a23ab	backend/consul: Use backendbase instead of helper/schema ... The legacy helper/schema package is really just a frozen copy of Terraform's old plugin SDK, which is a complicated heap of code that nobody at HashiCorp really understands anymore. This backend will now use the lightweight "backendbase" helper package instead, which is at a slightly lower level of abstraction and therefore requires more explicit logic during config decoding, but these extra calls to helper functions are still considerably less code than the whole helper/schema package and its various external dependencies that Terraform imports purely to keep it working.	2 years ago
Martin Atkins	eb87e86ddf	backend/http: Use backendbase instead of helper/schema ... The legacy helper/schema package is really just a frozen copy of Terraform's old plugin SDK, which is a complicated heap of code that nobody at HashiCorp really understands anymore. This backend will now use the lightweight "backendbase" helper package instead, which is at a slightly lower level of abstraction and therefore requires more explicit logic during config decoding, but these extra calls to helper functions are still considerably less code than the whole helper/schema package and its various external dependencies that Terraform imports purely to keep it working.	2 years ago
Martin Atkins	2222c23d4b	backendbase: BoolValue and MustBoolValue ... With legacy helper/schema it's common to just type-assert whatever ResourceData.Get returns and trust that it'll always be valid. We can't achieve something quite that lazy with cty, but we can at least factor out the work of turning a value we already know is a bool into a bool value. We also have a "must" variant for this one, because it's relatively easy for a caller to ensure that it controls all of the inputs enough to ensure that it should never fail.	2 years ago
Martin Atkins	1eaa18d60a	backendbase: IntValue ... With legacy helper/schema it's common to just type-assert whatever ResourceData.Get returns and trust that it'll always be valid. We can't achieve something quite that lazy with cty, but we can at least factor out the work of turning a value we already know is a number into an int64 value.	2 years ago
Martin Atkins	70537aab08	backendbase: ErrorAsDiagnostics ... The legacy helper/schema based Backend implementation expected the backend-specific code to return error and then wrapped it in diagnostics itself before returning. Backends using backendbase are supposed to return diagnostics directly themselves, but having to manually replace every error case with a higher-quality diagnostic message would get in the way of EOLing the legacy helper/schema implementation, and so this helper function is a pragmatic compromise to allow us to keep those updates purely mechanical for now and then improve diagnostics as a separate effort later.	2 years ago
Martin Atkins	46951b5b35	backendbase: Attribute lookup helpers ... Most of the remote state backends are currently written in terms of schema.ResourceData, lookup up attributes by path and possibly substituting default values when the attributes aren't explicitly set. These helpers aim to substitute for those common applications so that we can convert the existing backend logic more directly, without significantly increasing its complexity by suddenly moving the default value handling inline in their Configure functions.	2 years ago
Martin Atkins	d4c58fac4c	backend/inmem: No more legacy/helper/schema ... This now uses "backendbase", which is a lighter replacement for our local copy of Terraform's legacy plugin SDK.	2 years ago
Martin Atkins	df4cefbea9	backend: new "backendbase" helper package ... This is intended as a lighter-weight replacement for the Backend type in the legacy/helper/schema package, which indirectly pulls in our local copy of the entire legacy Terraform plugin SDK. This depends on the slightly-lower-level configschema package, which means that it's not quite as feature-rich as what it's replacing, but backends have relatively modest needs and so we can compensate with just some extra logic in their Configure functions, which will be considerably simpler than the huge bunch of legacy code we're currently depending on to get the same effects. In particular, configschema has no concept of "default values" for arguments, and so we'll need to deal with those in the Configure methods using imperative code, instead of declaratively. Nothing is using this yet, but in future commits we'll gradually convert all of the remote state backends to use this instead of the legacy system, and then remove the legacy system.	2 years ago
Martin Atkins	c0943fdd2a	backend/azure: Bring randString test helper into this package ... We've been keeping around the whole legacy "acctest" package just for this one function, used only by the Azure provider. Instead we'll inline the function into this package, so that we can delete the acctest package in a future commit.	2 years ago
Martin Atkins	4295cebb05	dependencies: No more golang.org/x/exp ... This module was a temporary home for what are now the standard library "slices" and "maps" packages. We're building Terraform with a sufficiently new version of Go that we can use the standard library equivalents, so we no longer need this dependency. It lives on as an indirect dependency for now because hashicorp/aws-sdk-go-base is also using it, but hopefully it'll vanish altogether in future if that library is updated to also use stdlib instead.	2 years ago
Martin Atkins	1358245617	dependencies: Remove all non-legacy uses of hashicorp/go-multierror ... This functionality is now provided by the "errors" package in the standard library, so the HashiCorp-specific library is obsolete. This also removed our one direct use of github.com/pkg/errors, which we were using in a way that was totally redundant with the stdlib package errors. However, it lives on as an indirect dependency through the Consul backend's dependency on the Consul SDK, which uses that library.	2 years ago
thirdkeyword	8284cde302	remove repetitive words ... Signed-off-by: thirdkeyword <fliterdashen@gmail.com>	2 years ago
Jared Baker	54866790ca	backend/s3: remove use_legacy_workflow argument ... The `use_legacy_workflow` argument has been removed to encourage consistency with the AWS SDKs. The backend will now search for credentials in the same order as the default provider chain in the AWS SDKs and AWS CLI.	2 years ago
Daniel Schmidt	8564e934aa	Merge pull request #34699 from hashicorp/terraform-test-use-vars-within-blocks ... test: allow using global variables in suite-level variable definitions	2 years ago
Daniel Schmidt	5a72e696ce	check for presence and not for warnings	2 years ago
Daniel Schmidt	a4651fac4c	append diags unconditionally	2 years ago
Daniel Schmidt	57f6ce81b5	rename suite to file and make var internal	2 years ago
Daniel Schmidt	6a4ef8e2b3	fix comment ... Co-authored-by: Liam Cervante <liam.cervante@hashicorp.com>	2 years ago
Daniel Schmidt	631293060b	only extract global and suite level variables once	2 years ago
Daniel Schmidt	e57368045c	fix implementation of variable checking	2 years ago
Daniel Schmidt	df3ee21d82	extract global and suite-level variables before configuring providers	2 years ago
Daniel Schmidt	697e0d1d9e	refactor variable extraction	2 years ago
Liam Cervante	c46515c5da	stacks: add support for external providers to validate context (#34701)	2 years ago
Daniel Schmidt	7ae7a203d8	test: allow using global variables in suite-level variable definitions ... Closes #34534	2 years ago
Liam Cervante	f0b61090a2	Move the testing MockProvider into a dedicated shared package (#34681) ... * move the testing MockProvider into a dedicated shared package * address comments * fix new changes	2 years ago
Liam Cervante	58289db4a9	terraform test: use platform independent path functions (#34666)	2 years ago
Martin Atkins	884e1fb2a4	terraform: Plans can be "complete" and "applyable" ... These ideas are both already implied by some logic elsewhere in the system, but until now we didn't have the decision logic centralized in a single place that could therefore evolve over time without necessarily always updating every caller together. We'll now have the modules runtime produce its own boolean ruling about each characteristic, which callers can rely on for the mechanical decision-making of whether to offer the user an "approve" prompt, and whether to remind the user after apply that it was an incomplete plan that will probably therefore need at least one more plan/apply round to converge. The "Applyable" flag directly replaces the previous method Plan.CanApply, with equivalent logic. Making this a field instead of a method means that we can freeze it as part of a saved plan, rather than recalculating it when we reload the plan, and we can export the field value in our export formats like JSON while ensuring it'll always be consistent with what Terraform is using internally. Callers can (and should) still use other context in the plan to return more tailored messages for specific situations they already know about that might be useful to users, but with these flags as a baseline callers can now just fall back to a generic presentation when encountering a situation they don't yet understand, rather than making the wrong decision and causing something strange to happen. That is: a lack of awareness of a new rule will now cause just a generic message in the UI, rather than incorrect behavior. This commit mostly just deals with populating the flags, and then all of the direct consequences of that on our various tests. Further changes to actually make use of these flags elsewhere in the system will follow in later commits, both in this repository and in other repositories.	2 years ago