7 Commits (4835deddaef450198bba00a50311f2ada6f79b96)

Author	SHA1	Message	Date
James Bardin	47f9850f02	add XTerraformGetLimit to prevent redirect loops	4 years ago
Martin Atkins	74761b2f8b	getmodules: Use go-getter v1.5.10 and return to upstream GitGetter ... There was an unintended regression in go-getter v1.5.9's GitGetter which caused us to temporarily fork that particular getter into Terraform to expedite a fix. However, upstream v1.5.10 now includes a functionally-equivalent fix and so we can heal that fork by upgrading. We'd also neglected to update the Module Sources docs when upgrading to go-getter v1.5.9 originally and so we were missing documentation about the new "depth" argument to enable shadow cloning, which I've added retroactively here along with documenting its restriction of only supporting named refs. This new go-getter release also introduces a new credentials-passing method for the Google Cloud Storage getter, and so we must incorporate that into the Terraform-level documentation about module sources.	4 years ago
Martin Atkins	c4d46e7c6b	getmodules: Re-allow git:: source with ref=COMMIT_ID ... Earlier versions of this code allowed "ref" to take any value that would be accepted by "git checkout" as a valid target of a symbolic ref. We inadvertently accepted a breaking change to upstream go-getter that broke that as part of introducing a shallow clone optimization, because shallow clone requires selecting a single branch. To restore the previous capabilities while retaining the "depth" argument, here we accept a compromise where "ref" has the stronger requirement of being a valid named ref in the remote repository if and only if "depth" is set to a value greater than zero. If depth isn't set or is less than one, we will do the old behavior of just cloning all of the refs in the remote repository in full and then switching to refer to the selected branch, tag, or naked commit ID as a separate step. This includes a heuristic to generate an additional error message hint if we get an error from "git clone" and it looks like the user might've been trying to use "depth" and "ref=COMMIT" together. We can't recognize that error accurately because it's only reported as human-oriented git command output, but this heuristic should hopefully minimize situations where we show it inappropriately. For now this is a change in the Terraform repository directly, so that we can expedite the fix to an already-reported regression. After this is released I tend to also submit a similar set of changes to upstream go-getter, at which point we can revert Terraform to using the upstream getter.GitGetter instead of our own local fork.	4 years ago
Martin Atkins	b0ff17ef2a	getmodules: Inline our own fork of getter.GitGetter ... This is a pragmatic temporary solution to allow us to more quickly resolve an upstream regression in go-getter locally within Terraform, so that the work to upstream it for other callers can happen asynchronously and with less time pressure. This commit doesn't yet include any changes to address the bug, and instead aims to be functionally equivalent to getter.GitGetter. A subsequent commit will then address the regression, so that the diff of that commit will be easier to apply later to the upstream to get the same effect there.	4 years ago
kmoe	40ec62c139	command: make module installation interruptible ... Earlier work to make "terraform init" interruptible made the getproviders package context-aware in order to allow provider installation to be cancelled. Here we make a similar change for module installation, which is now also cancellable with SIGINT. This involves plumbing context through initwd and getmodules. Functions which can make network requests now include a context parameter whose cancellation cancels those requests. Since the module installation code is shared, "terraform get" is now also interruptible during module installation.	4 years ago
Martin Atkins	1a8da65314	Refactoring of module source addresses and module installation ... It's been a long while since we gave close attention to the codepaths for module source address parsing and external module package installation. Due to their age, these codepaths often diverged from our modern practices such as representing address types in the addrs package, and encapsulating package installation details only in a particular location. In particular, this refactor makes source address parsing a separate step from module installation, which therefore makes the result of that parsing available to other Terraform subsystems which work with the configuration representation objects. This also presented the opportunity to better encapsulate our use of go-getter into a new package "getmodules" (echoing "getproviders"), which is intended to be the only part of Terraform that directly interacts with go-getter. This is largely just a refactor of the existing functionality into a new code organization, but there is one notable change in behavior here: the source address parsing now happens during configuration loading rather than module installation, which may cause errors about invalid addresses to be returned in different situations than before. That counts as backward compatible because we only promise to remain compatible with configurations that are _valid_, which means that they can be initialized, planned, and applied without any errors. This doesn't introduce any new error cases, and instead just makes a pre-existing error case be detected earlier. Our module registry client is still using its own special module address type from registry/regsrc for now, with a small shim from the new addrs.ModuleSourceRegistry type. Hopefully in a later commit we'll also rework the registry client to work with the new address type, but this commit is already big enough as it is.	5 years ago
Martin Atkins	f25935649a	getmodules: Beginnings of a new package about Terraform module packages ... This new package aims to encapsulate all of our interactions with go-getter to fetch remote module packages, to ensure that the rest of Terraform will only use the small subset of go-getter functionality that our modern module installer uses. In older versions of Terraform, go-getter was the entire implementation of module installation, but along the way we found that several aspects of its design are poor fit for Terraform's needs, and so now we're using it as just an implementation detail of Terraform's handling of remote module packages only, hiding it behind this wrapper API which exposes only the services that our module installer needs. This new package isn't actually used yet, but in a later commit we will change all of the other callers to go-getter to only work indirectly through this package, so that this will be the only package that actually imports the go-getter packages.	5 years ago