414 Commits (148b19b0d7e36b6f2b8a169d2cb7b6069ab5d00a)

Author	SHA1	Message	Date
Martin Atkins	660ff86c4c	backendbase: Preserve unset arguments as null when not set by env ... Previously we tried a simplification where any attribute with an "SDK-like default" was guaranteed to never be null, because that seemed like a plausible mimic of the legacy SDK's general aversion to nulls. However, the SDK does still respect the absense of an argument as different to its zero value in one narrow case: when the argument isn't set in the configuration, also isn't set by any environment variables, and has no static fallback default. In that case, the SDK's built-in validation rules such as "ConflictsWith" _do_ treat absent as different from zero-value. To allow backends to continue making such distinctions themselves where that's useful, SDKLikeDefaults now leaves an attribute set to null if it wasn't present in the configuration and none of the environment variables or fallback value cause it to end up being a non-empty string. The backend can then choose to bypass the SDKLikeData API and check the cty.Value directly if it wants to distinguish set from zero even though SDKLikeData is designed to avoid the need to do that. This also updates the gcs backend to make use of the new facility, since it wants to raise an error if two arguments are both set even if they are both set to the empty string. Restoring that original error case is the main motivation for this change.	2 years ago
Martin Atkins	c77898c90d	logging: Remove import github.com/coreos/pkg/capnslog ... We were importing this to resolve an init-time conflict with this library when it was indirectly loaded by the etcd libraries. We removed the etcd backends a while back and so we no longer use any of the etcd modules in Terraform, and so this tricky import was our only remaining reference to github.com/coreos/pkg/capnslog. Dropping this eliminates two unnecessary dependencies.	2 years ago
James Bardin	b79589d2d5	udpate submodule deps	2 years ago
Jamie Finnigan	e9a33de0d2	go get github.com/hashicorp/go-getter && go mod tidy (#35041) ... * go get github.com/hashicorp/go-getter && go mod tidy * make syncdeps	2 years ago
Nick Fagerlund	40b6f21b3d	Merge pull request #35050 from hashicorp/rebrand-hcp-terraform ... Rebrand Terraform Cloud to HCP Terraform	2 years ago
Sebastian Rivera	015f795ff0	Rebrand Terraform Cloud to HCP Terraform	2 years ago
Liam Cervante	448775102c	terraform test: Push evaluation of variables to as late as possible (#35014) ... * terraform test: Push evaluation of variables to as late as possible * Update internal/moduletest/hcl/variable_cache.go Co-authored-by: kmoe <5575356+kmoe@users.noreply.github.com> * address comments --------- Co-authored-by: kmoe <5575356+kmoe@users.noreply.github.com>	2 years ago
Martin Atkins	5868f99413	backend/gcs: Stop using legacy helper/schema ... As part of our efforts to remove the large snapshot of the legacy SDK that we've been using, this replaces the uses of that package with our new lightweight "backendbase" package, which is a collection of helpers that can substitute for most of the parts of the legacy SDK that backends tend to use. For this backend for now we'll use the "SDK like" helpers which aim to retain some of the legacy-SDK-specific assumptions that aren't technically true anymore, such as the idea that null and empty string are equivalent. Hopefully one day we'll be able to wean this backend off of that more completely, but this is a pragmatic way to get away from the legacy SDK without a large rewrite. This also drew attention to the fact that this backend was previously relying on the weird context.Context value that the SDK was passing in to the configure method, even though that context isn't connected up to anything particularly useful. In the long run we should change the backend APIs to pass context.Context to each method that might make network requests, but for now we're using context.TODO so that we can more easily find these cases and update them later once there's a real context to use.	2 years ago
Liam Cervante	6963c54bec	terraform test: don't remove sensitive marks from inputs (#35021)	2 years ago
Martin Atkins	5e7d0f0854	go.mod: Update to latest github.com/zclconf/go-cty-debug ... This extends the "CmpOptions" to include a comparer for cty.Path values, so that we don't have to hand-write that in each case where we're comparing data structures containing those.	2 years ago
Martin Atkins	397ea6d05c	moduletest: Capture optional "execution meta" for test runs ... For now this is only to have a place to save the execution duration, but wrapped in a separate struct so that we can distinguish between cases when it is set and when it isn't, because not all test runners will necessarily populate this and even those that do might not populate it in situations where, for example, a test run gets skipped due to an upstream failure. As of this commit, nothing makes use of this new information. A future commit will use it as part of the experimental JUnit XML output from "terraform test", since the JUnit XML format de-facto _requires_ an execution duration for every test case.	2 years ago
Nick Fagerlund	0a4c6c3cb8	Merge pull request #34914 from hashicorp/nf/mar24-enable-deferred-actions ... Allow enabling deferred actions in PlanOpts	2 years ago
Martin Atkins	40255906f0	go.mod: Update OpenTelemetry dependencies to v1.24.0 ... We previously had a strange assortment of different versions, but it's typically best to have all of these fixed to the same version because otherwise there can be quirky interactions where e.g. an interface type has changed without updating its implementors, etc. This intentionally does not yet upgrade the contrib packages or exporters because they imply other dependency upgrades that we want to perform separately first, so we can avoid changing too much at once.	2 years ago
Nick Fagerlund	1781c8e8e9	Experimental `-allow-deferral` flag for enabling deferred actions on CLI ... Now that deferred actions is a top-level per-plan option, here's a lil command-line option for turning it on! - It's called `-allow-deferral`. - It is ONLY legal to use if this is an experimental build, i.e. compiled with `go build -ldflags "-w -s -X 'main.experimentsAllowed=yes'"` or the like. Implementation notes: The design constraint here was, "please avoid having to change the build infrastructure or alter the function of every args parsing function". So, since `extendedFlagSet()` doesn't have access to the var that says we're in an experimental build, I had to move the validation for that particular flag to later in the process, when we're building the operation request.	2 years ago
Jared Baker	c140e2eef1	Merge branch 'main' into patch-1	2 years ago
Martin Atkins	d3b8a55781	go.mod: go get google.golang.org/protobuf@v1.33.0 ... Many of the changes between these two versions seem to be in parts of the library that Terraform doesn't use, but there are a few sensitive parts: - Slight changes to details of how the library implements the JSON encoding of protobuf. Terraform CLI/Core don't use this, but some of the backends might use it indirectly when talking to their underlying APIs. - Some details of the "protoreflect" package for working with protobuf descriptors dynamically (rather than using generated stubs) have changed, but it seems pretty marginal and unlikely to cause problems.	2 years ago
Martin Atkins	84df5562f2	moduleaddrs: Terraform now owns module source address normalization ... We've been gradually chipping away at how much we use go-getter for source packages, because it's generally been a bit of a nightmare and sharing it with other codebases means that any time someone wants to change something we end up needing to find some way to prevent it breaking Terraform's compatibility promises. Here we make one further step: Terraform owns the "detectors" idea that deals with source address normalization, and now always produces fully-qualified addresses for go-getter to chew on only for the getting and decompressing steps. Retaining go-getter for the actual getting part is helpful because we can then benefit from security fixes upstream, but Terraform owning the first layer of parsing means that we can fix in place the definition of what "module source address" syntax means, and thus we can avoid having everything in this codebase indirectly depend on go-getter just because it wants to parse module source addresses. Now only the module installer actually depends indirectly on go-getter, which finally disconnects go-getter's subtree from all of the remote state backend dependency graphs.	2 years ago
Martin Atkins	78ae6af906	legacy/terraform: Minimize the contents of this legacy package ... At one point we were relying on this for a few different things, but now it's here exclusively to support legacy/helper/schema, the legacy SDK snapshot that is still being used by a few of the remote state backends. The SDK only really needs a tiny portion of the functionality from this package, and by deleting as much of the unused stuff as possible we can reduce the indirect dependencies of this package and thus, in turn, the indirect dependencies of the remote state backends. This also selectively deletes a few parts of the legacy SDK that are focused on providers and provisioners, because we know that we have this here only to support the backends that haven't yet been weaned off it, but we'll save a more adventurous purge of _that_ package for another day.	2 years ago
Martin Atkins	621befc3a2	configtesting: Separate package for the SynthBody helper ... This "SynthBody" helper doesn't really have anything to do with package configs except that it's sometimes useful for creating fake instances of the types in there for testing. Moving it into its own package means that package backend no longer needs to depend on package configs, since it doesn't actually have any need for parsing module configurations. The old configs.SynthBody function is still available as a forwarding alias, because most of the existing callers of it are from packages that already need to import package configs for other reasons, and so this reduces the need to churn loads of existing tests.	2 years ago
Martin Atkins	56006416ba	go.mod: go get golang.org/x/tools@v0.19.0 ... We use this just for development-time tools like "stringer", so this upgrade should not have any runtime impact.	2 years ago
Martin Atkins	c541757f4a	go.mod: go get golang.org/x/mod@v0.16.0 ... Aside from development tool dependencies we use this only for the directory hashing logic used to compute go module hashes, which we also use for Terraform provider hashes. There are no changes to the dirhash package in this update, so this should have no runtime impact.	2 years ago
Martin Atkins	b059985227	go.mod: go get golang.org/x/sync@v0.6.0 ... This update only changes a documentation comment, so cannot affect runtime behavior.	2 years ago
Martin Atkins	caa7a757bc	go.mod: go get golang.org/x/oauth2@v0.18.0 ... This only upgrades a Google-specific subpackage, and so the only potential impact is to the GCS backend.	2 years ago
Martin Atkins	9c7ece8ad4	go.mod: go get golang.org/x/net@v0.22.0 ... This update is largely focused on QUIC implementation changes, which do not directly affect Terraform. There is one fix for the HTTP2 implementation to terminate the stream if a sender recieves an out-of-bounds window update, which makes the implementation better match the specification. This seems unlikely to cause any practical problems for Terraform.	2 years ago
Martin Atkins	fad9c2a4cd	go.mod: go get golang.org/x/crypto@v0.21.0 ... This update is mostly internal changes that don't affect external behavior. The only update potentially significant to Terraform is the new exclusion of a particular root certificate that is used only for some specific names under the .tr CCTLD, but the certificate database doesn't specify that constraint in a way that the library can understand, so it's excluded to avoid making the trust for that certificate broader than intended.	2 years ago
Martin Atkins	a1d2169c85	go.mod: go get golang.org/x/term@v0.18.0 ... This update contains no real changes. It just updates transitive dependencies to newer versions that we've already selected anyway.	2 years ago
Martin Atkins	b3f6a6230d	go.mod: go get golang.org/x/sys@v0.18.0 ... This small update does not directly affect any Terraform functionality.	2 years ago
Martin Atkins	61a5c1f57b	go.mod: go get github.com/hashicorp/terraform-registry-address@v0.2.3 ... This is really just a bunch of changes to the upstream go.mod for updates to its own dependencies, but Terraform already has new enough versions of those dependencies for other reasons and so this changes nothing.	2 years ago
Martin Atkins	8603ee0ab5	go.mod: go get github.com/hashicorp/go-slug@v0.14.0 ... This has just some changes to the "sourcebundle" building API, which doesn't affect Terraform CLI or Terraform Core because they currently only consume source bundles, not build them.	2 years ago
Martin Atkins	914618b061	gcs: Adopt the "ReadPathOrContents" helper function ... Package backend previously had a ReadPathOrContents helper that takes a string that could represent either a filename or some literal content, and then reads the given file if it's a filename or just returns the literal content otherwise. We originally used variations of this helper to shim a number of different situations that had originally accepted filenames but later changed to accept literal content, but over time we've eliminated all of those and so the only remaining caller is the gcs backend which uses it to accept GCP credentials either as literal content or as a file containing the credentials. Therefore here we move that helper function to be just an unexported part of the gcs backend package. This is part of an ongoing effort to minimize what each remote state backend depends on (either directly or indirectly) so that we can make gradual progress towards these remote state backends being self-contained enough to move into other codebases. In the short term though, it just puts this helper function closer to its caller so it's easier to maintain them both together if needed.	2 years ago
Martin Atkins	f058de612c	statemgr: Incorporate IntermediateStateConditionalPersister ... This interface was originally in the local backend's package because that backend is the only one that _truly_ runs Terraform operations and thus needs to negotiate how to store the intermediate state snapshots that result from them. However, putting the interface and its supporting types/functions there means that anything that wants to implement this interface needs to import the local backend package, and thus indirectly depends on the entire modules runtime, etc. We already had package stagemgr as a home for the shared stuff used by most state storage managers, and so we'll just move these symbols over there and then we can shed some internal dependencies on package local. This also severs the last dependency link between the remote state backends and the modules runtime, allowing us to shed a bunch of indirect dependencies from those remote state backends.	2 years ago
Martin Atkins	cf3bbb890f	State managers use schemarepo.Schemas instead of terraform.Schemas ... schemarepo.Schemas used to be terraform.Schemas, but its presence in the terraform package was the only reason for a state manager to need to import package terraform, so we've moved it out into its own package in the parent commit. This means that none of the state-related packages need to import package terraform any longer, and so once we've untangled some other dependencies this will curtail the number of packages in this codebase that the remote state backends transitively depend on, as part of an ongoing effort to eventually separate them from this codebase entirely.	2 years ago
Martin Atkins	88cb4aeea3	backendrun: Separate the types/etc for backends that support operations ... We previously had all of the types and helpers for all kinds of backends together in package backend. That kept things relatively simple, but it also meant that the majority of backends that only deal with remote state storage ended up still indirectly depending on the entire Terraform modules runtime, configuration loader, etc, etc, which brings into scope a bunch of external dependencies that the remote state backends don't really need. Since backends that support operations are a rare exception, we'll move the types and helpers for those into a separate package "backendrun", and then the main package backend can have a much more modest set of types and, more importantly, a modest set of dependencies on other packages in this codebase. This is part of an ongoing effort to reduce the exposure of Terraform Core and CLI code to the remote backends and vice-versa, so that in the long run we can more often treat them as separate for dependency maintenance purposes.	2 years ago
Martin Atkins	5f2eab6944	providerreqs: Provider requirement/version types in their own package ... As we previously found with "package addrs", any package that contains types used as identifiers or addresses tends to get imported from all over, which means that if we mix real logic with those types in the same package then a lot more packages end up depending indirectly on our external dependencies. In this case, the getproviders package depends on various things we need only during provider installation, such as the library we use for GPG signature verification. Lots of different packages in Terraform need to talk about provider requirements, but only the CLI layer actually needs to do provider installation, and so this commit splits the types we use to talk about requirements into a new package "providerreqs", which package getproviders then depends on. To keep the size of this diff relatively small to start I've left forwarding aliases in package getproviders, which means that not all of the callers needed to be updated all at once. This commit does update all of the packages that the remote state backends depend on though, because that then allows us to reduce the set of indirect dependencies for each of those backends. The nested module go.mod files show that this has disconnected the indirect dependencies on github.com/ProtonMail/go-crypto and on github.com/cloudflare/circl , both of which Terraform CLI uses only for provider installation.	2 years ago
Martin Atkins	310331bdc6	backend/consul: go.mod updates from a merge conflict ... Two changes landed concurrently and caused these dependencies to get out of sync.	2 years ago
Martin Atkins	5811be26f8	go.mod: Separate modules at code ownership boundaries ... Historically it's been hard for us to keep on top of dependency upgrades as a matter of course because it's unclear which code owners might be affected by or responsible for an upgrade of each module. After trying a few different techniques to try to mitigate this, the most promising one seems to be to tell the Go toolchain that each of these components is a separate Go module, but then to stitch them all back together again using replace directives so that we can mostly ignore these module boundaries in everyday development. The one case where we _do_ need to pay attention to these boundaries is also the case where the separation is useful: upgrading a dependency of any one of these modules might potentially force upgrading it for another module too, and so the Go toolchain will help us notice that interaction and we can immediately which code owners might need to be involved in reviewing and testing that particular upgrade. To make that process less onerous, this adds a new makefile target "make syncdeps" which runs "go mod tidy" in all of the modules at once and thus forces the toolchain to align their dependencies to the extent required for them all to be linked together successfully, and to generate go.mod and/or go.sum diffs that will match our configured code ownership paths to trigger the appropriate code review requests. Hopefully before too long we'll be able to move all the backends out into provider plugins and delete the "legacy" module entirely and then this oddness won't be needed anymore, but as long as we're all trying to play together in this same sandbox this is a small amount of extra ceremony in return for hopefully reducing the level of anxiety involved in keeping the system's dependencies up to date more consistently.	2 years ago
Martin Atkins	6bd1c66cbc	backendbase: Basic support for default values ... Some existing backends make very heavy use of environment variable-based default values and fallback values, which are available in the legacy SDK but not in Terraform's own schema model. To ease the migration away from the legacy SDK for those backends, backendbase.Base now supports a lightweight mechanism for declaring such defaults, and applies them automatically as part of PrepareConfig. This gets us pretty close to the assumptions that the SDK allowed callers to make, but with considerably less code. It's intentionally considerably less flexible than the legacy SDK was, because in practice most backends use only a tiny subset of the SDK functionality anyway.	2 years ago
Martin Atkins	b16cb49fb6	backendbase: "SDK-like" helpers ... A lot of the remote state backends' current Configure functions unfortunately rely quite heavily on the "do the best you can" API of the legacy SDK, which makes questionable shortcuts like treating null as equivalent to the Go zero value of a type, but in return makes it convenient to assign into other data structures that happen to make similar assumptions. And of course that means that the backends' other data structures were built to make those similar assumptions! This new set of utilities therefore aims to provide a bare-minimum subset of "SDK-like" functionality to ease the initial migration away from the legacy SDK. It isn't exactly equivalent an in particular focuses only on the small subset of legacy SDK functionality that most existing backends rely on, but this should at least make the initial adaptations a little more mechanical so that we can continue to defer the full modernization of these backends for some undetermined future time without having to continue retaining the huge legacy SDK snapshot in this codebase.	2 years ago
Martin Atkins	a5d63a23ab	backend/consul: Use backendbase instead of helper/schema ... The legacy helper/schema package is really just a frozen copy of Terraform's old plugin SDK, which is a complicated heap of code that nobody at HashiCorp really understands anymore. This backend will now use the lightweight "backendbase" helper package instead, which is at a slightly lower level of abstraction and therefore requires more explicit logic during config decoding, but these extra calls to helper functions are still considerably less code than the whole helper/schema package and its various external dependencies that Terraform imports purely to keep it working.	2 years ago
Martin Atkins	eb87e86ddf	backend/http: Use backendbase instead of helper/schema ... The legacy helper/schema package is really just a frozen copy of Terraform's old plugin SDK, which is a complicated heap of code that nobody at HashiCorp really understands anymore. This backend will now use the lightweight "backendbase" helper package instead, which is at a slightly lower level of abstraction and therefore requires more explicit logic during config decoding, but these extra calls to helper functions are still considerably less code than the whole helper/schema package and its various external dependencies that Terraform imports purely to keep it working.	2 years ago
Martin Atkins	2222c23d4b	backendbase: BoolValue and MustBoolValue ... With legacy helper/schema it's common to just type-assert whatever ResourceData.Get returns and trust that it'll always be valid. We can't achieve something quite that lazy with cty, but we can at least factor out the work of turning a value we already know is a bool into a bool value. We also have a "must" variant for this one, because it's relatively easy for a caller to ensure that it controls all of the inputs enough to ensure that it should never fail.	2 years ago
Martin Atkins	1eaa18d60a	backendbase: IntValue ... With legacy helper/schema it's common to just type-assert whatever ResourceData.Get returns and trust that it'll always be valid. We can't achieve something quite that lazy with cty, but we can at least factor out the work of turning a value we already know is a number into an int64 value.	2 years ago
Martin Atkins	70537aab08	backendbase: ErrorAsDiagnostics ... The legacy helper/schema based Backend implementation expected the backend-specific code to return error and then wrapped it in diagnostics itself before returning. Backends using backendbase are supposed to return diagnostics directly themselves, but having to manually replace every error case with a higher-quality diagnostic message would get in the way of EOLing the legacy helper/schema implementation, and so this helper function is a pragmatic compromise to allow us to keep those updates purely mechanical for now and then improve diagnostics as a separate effort later.	2 years ago
Martin Atkins	46951b5b35	backendbase: Attribute lookup helpers ... Most of the remote state backends are currently written in terms of schema.ResourceData, lookup up attributes by path and possibly substituting default values when the attributes aren't explicitly set. These helpers aim to substitute for those common applications so that we can convert the existing backend logic more directly, without significantly increasing its complexity by suddenly moving the default value handling inline in their Configure functions.	2 years ago
Martin Atkins	d4c58fac4c	backend/inmem: No more legacy/helper/schema ... This now uses "backendbase", which is a lighter replacement for our local copy of Terraform's legacy plugin SDK.	2 years ago
Martin Atkins	df4cefbea9	backend: new "backendbase" helper package ... This is intended as a lighter-weight replacement for the Backend type in the legacy/helper/schema package, which indirectly pulls in our local copy of the entire legacy Terraform plugin SDK. This depends on the slightly-lower-level configschema package, which means that it's not quite as feature-rich as what it's replacing, but backends have relatively modest needs and so we can compensate with just some extra logic in their Configure functions, which will be considerably simpler than the huge bunch of legacy code we're currently depending on to get the same effects. In particular, configschema has no concept of "default values" for arguments, and so we'll need to deal with those in the Configure methods using imperative code, instead of declaratively. Nothing is using this yet, but in future commits we'll gradually convert all of the remote state backends to use this instead of the legacy system, and then remove the legacy system.	2 years ago
Martin Atkins	c0943fdd2a	backend/azure: Bring randString test helper into this package ... We've been keeping around the whole legacy "acctest" package just for this one function, used only by the Azure provider. Instead we'll inline the function into this package, so that we can delete the acctest package in a future commit.	2 years ago
Martin Atkins	4295cebb05	dependencies: No more golang.org/x/exp ... This module was a temporary home for what are now the standard library "slices" and "maps" packages. We're building Terraform with a sufficiently new version of Go that we can use the standard library equivalents, so we no longer need this dependency. It lives on as an indirect dependency for now because hashicorp/aws-sdk-go-base is also using it, but hopefully it'll vanish altogether in future if that library is updated to also use stdlib instead.	2 years ago
Martin Atkins	1358245617	dependencies: Remove all non-legacy uses of hashicorp/go-multierror ... This functionality is now provided by the "errors" package in the standard library, so the HashiCorp-specific library is obsolete. This also removed our one direct use of github.com/pkg/errors, which we were using in a way that was totally redundant with the stdlib package errors. However, it lives on as an indirect dependency through the Consul backend's dependency on the Consul SDK, which uses that library.	2 years ago
thirdkeyword	8284cde302	remove repetitive words ... Signed-off-by: thirdkeyword <fliterdashen@gmail.com>	2 years ago