305 Commits (69ebfd6f037f8d51709770cc36d266bb86cf950d)

Author	SHA1	Message	Date
Martin Atkins	6b8e103d6a	configs: Include "moved" blocks when merging multiple files into a module ... An earlier commit added logic to decode "moved" blocks and do static validation of them. Here we now include that result also in modules produced from those files, which we can then use in Terraform Core to actually implement the moves. This also places the feature behind an active experiment keyword called config_driven_move. For now activating this doesn't actually achieve anything except let you include moved blocks that Terraform will summarily ignore, but we'll expand the scope of this in later commits to eventually reach the point where it's really usable.	5 years ago
Martin Atkins	d92b5e5f5e	configs: valid-modules test ignores experimental features warning ... A common source of churn when we're running experiments is that a module that would otherwise be valid ends up generating a warning merely because the experiment is active. That means we end up needing to shuffle the test files around if the feature ultimately graduates to stable. To reduce that churn in simple cases, we'll make an exception to disregard the "Experiment is active" warning for any experiment that a module has intentionally opted into, because those warnings are always expected and not a cause for concern. It's still possible to test those warnings explicitly using the testdata/warning-files directory, if needed.	5 years ago
Martin Atkins	708003b035	configs: For Moved blocks, use addrs.MoveEndpoint instead of addrs.Target ... Although addrs.Target can in principle capture the information we need to represent move endpoints, it's semantically confusing because addrs.Targetable uses addrs.Abs... types which are typically for absolute addresses, but we were using them for relative addresses here. We now have specialized address types for representing moves and probably other things which have similar requirements later on. These types largely communicate the same information in the end, but aim to do so in a way that's explicit about which addresses are relative and which are absolute, to make it less likely that we'd inadvertently misuse these addresses.	5 years ago
Martin Atkins	4cbe6cabfc	addrs: AbsMoveable, ConfigMoveable, and MoveableEndpoint ... These three types represent the three different address representations we need to represent different stages of analysis for "moved" blocks in the configuration. The goal here is to encapsulate all of the static address wrangling inside these types so that users of these types elsewhere would have to work pretty hard to use them incorrectly. In particular, the MovableEndpoint type intentionally fully encapsulates the weird relative addresses we use in configuration so that code elsewhere in Terraform can never end up holding an address of a type that suggests absolute when it's actually relative. That situation only occurs in the internals of MoveableEndpoint where we use not-really-absolute AbsMoveable address types to represent the not-yet-resolved relative addresses. This only takes care of the static address wrangling. There's lots of other rules for what makes a "moved" block valid which will need to be checked elsewhere because they require more context than just the content of the address itself.	5 years ago
Martin Atkins	3212f6f367	addrs: AbsModuleCall type ... Our documentation for ModuleCall originally asserted that we didn't need AbsModuleCall because ModuleInstance captured the same information, but when we added count and for_each for modules we introduced ModuleCallInstance to represent a reference to an instance of a local module call, and now _that_ is the type whose absolute equivalent is ModuleInstance. We previously had no absolute representation of the call itself, without any particular instance. That's what AbsModuleCall now represents, allowing us to be explicit about when we're talking about the module block vs. instances it declares, which is the same distinction represented by AbsResource vs. AbsResourceInstance. Just like with AbsResource and AbsResourceInstance though, there is syntactic ambiguity between a no-key call instance and a whole module call, and so some codepaths might accept both to start and then use other context to dynamically choose a particular interpretation, in which case this distinction becomes meaningful in representing the result of that decision.	5 years ago
Martin Atkins	ab350289ab	addrs: Rename AbsModuleCallOutput to ModuleCallInstanceOutput ... The previous name didn't fit with the naming scheme for addrs types: The "Abs" prefix typically means that it's an addrs.ModuleInstance combined with whatever type name appears after "Abs", but this is instead a ModuleCallOutput combined with an InstanceKey, albeit structured the other way around for convenience, and so the expected name for this would be the suffix "Instance". We don't have an "Abs" type corresponding with this one because it would represent no additional information than AbsOutputValue.	5 years ago
Kristin Laemmert	35c19d7c9f	command/jsonstate: remove redundant remarking of resource instance (#29049) ... * command/jsonstate: remove redundant remarking of resource instance ResourceInstanceObjectSrc.Decode already handles marking values with any marks stored in ri.Current.AttrSensitivePaths, so re-applying those marks is not necessary. We've gotten reports of panics coming from this line of code, though I have yet to reproduce the panic in a test. * Implement test to reproduce panic on #29042 Co-authored-by: David Alger <davidmalger@gmail.com>	5 years ago
Martin Atkins	70bc432f85	command/views/json: Never generate invalid diagnostic snippet offsets ... Because our snippet generator is trying to select whole lines to include in the snippet, it has some edge cases for odd situations where the relevant source range starts or ends directly at a newline, which were previously causing this logic to return out-of-bounds offsets into the code snippet string. Although arguably it'd be better for the original diagnostics to report more reasonable source ranges, it's better for us to report a slightly-inaccurate snippet than to crash altogether, and so we'll extend our existing range checks to check both bounds of the string and thus avoid downstreams having to deal with out-of-bounds indices. For completeness here I also added some similar logic to the human-oriented diagnostic formatter, which consumes the result of the JSON diagnostic builder. That's not really needed with the additional checks in the JSON diagnostic builder, but it's nice to reinforce that this code can't panic (in this way, at least) even if its input isn't valid.	5 years ago
James Bardin	c687ebeaf1	Merge pull request #29039 from hashicorp/jbardin/sensitive ... New marks.Sensitive type, and audit of sensitive marks usage	5 years ago
James Bardin	80ef795cbf	add marks.Raw	5 years ago
James Bardin	55ebb2708c	remove IsMarked and ContainsMarked calls ... Make sure sensitivity checks are looking for specific marks rather than any marks at all.	5 years ago
James Bardin	cdf7469efd	marks.Has and marks.Contains	5 years ago
James Bardin	d9dfd451ea	update to use typed sensitive marks	5 years ago
James Bardin	2c493e38c7	marks package ... marks.Sensitive	5 years ago
Kristin Laemmert	096010600d	terraform: use hcl.MergeBodies instead of configs.MergeBodies for pro… (#29000) ... * terraform: use hcl.MergeBodies instead of configs.MergeBodies for provider configuration Previously, Terraform would return an error if the user supplied provider configuration via interactive input iff the configuration provided on the command line was missing any required attributes - even if those attributes were already set in config. That error came from configs.MergeBody, which was designed for overriding valid configuration. It expects that the first ("base") body has all required attributes. However in the case of interactive input for provider configuration, it is perfectly valid if either or both bodies are missing required attributes, as long as the final body has all required attributes. hcl.MergeBodies works very similarly to configs.MergeBodies, with a key difference being that it only checks that all required attributes are present after the two bodies are merged. I've updated the existing test to use interactive input vars and a schema with all required attributes. This test failed before switching from configs.MergeBodies to hcl.MergeBodies. * add a command package test that shows that we can still have providers with dynamic configuration + required + interactive input merging This test failed when buildProviderConfig still used configs.MergeBodies instead of hcl.MergeBodies	5 years ago
Kristin Laemmert	3acb5e2841	configs: add decodeMovedBlock behind a locked gate. (#28973) ... This PR adds decoding for the upcoming "moved" blocks in configuration. This code is gated behind an experiment called EverythingIsAPlan, but the experiment is not registered as an active experiment, so it will never run (there is a test in place which will fail if the experiment is ever registered). This also adds a new function to the Targetable interface, AddrType, to simplifying comparing two addrs.Targetable. There is some validation missing still: this does not (yet) descend into resources to see if the actual resource types are the same (I've put this off in part because we will eventually need the provider schema to verify aliased resources, so I suspect this validation will have to happen later on).	5 years ago
Alisdair McDiarmid	5611da8eb5	Merge pull request #28975 from hashicorp/alisdair/jsonplan-drift ... json-output: Omit unchanged resource_drift entries	5 years ago
James Bardin	6a495c8d42	fixupBody missing MissingItemRange ... When blocktoattr.fixupBody returned its content, the value for `MissingItemRange` was omitted, losing the diagnostic Subject.	5 years ago
Alisdair McDiarmid	3326ab7dae	json-output: Omit unchanged resource_drift entries ... Previously, if any resources were found to have drifted, the JSON plan output would include a drift entry for every resource in state. This commit aligns the JSON plan output with the CLI UI, and only includes those resources where the old value does not equal the new value---i.e. drift has been detected. Also fixes a bug where the "address" field was missing from the drift output, and adds some test coverage.	5 years ago
Kristin Laemmert	583859e510	commands: `terraform add` (#28874) ... * command: new command, terraform add, generates resource templates terraform add ADDRESS generates a resource configuration template with all required (and optionally optional) attributes set to null. This can optionally also pre-populate nonsesitive attributes with values from an existing resource of the same type in state (sensitive vals will be populated with null and a comment indicating sensitivity) * website: terraform add documentation	5 years ago
Bryan Eastes	714632206c	Quoting filesystem path in scp command argument (#28626) ... * Quoting filesystem path in scp command argument * Adding proper shell quoting for scp commands * Running go fmt * Using a library for quoting shell commands * Don't export quoteShell function	5 years ago
James Bardin	2ecdf44918	Merge pull request #28941 from hashicorp/jbardin/objchange-unknown-blocks ... handle unexpected changes to unknown block	5 years ago
Kristin Laemmert	329585d07d	jsonconfig: properly unwind and enumerate references (#28884) ... The "references" included in the expression representation now properly unwrap for each traversal step, to match what was documented.	5 years ago
Kristin Laemmert	ac03d35997	jsonplan and jsonstate: include sensitive_values in state representations (#28889) ... * jsonplan and jsonstate: include sensitive_values in state representations A sensitive_values field has been added to the resource in state and planned values which is a map of all sensitive attributes with the values set to true. It wasn't entirely clear to me if the values in state would suffice, or if we also need to consult the schema - I believe that this is sufficient for state files written since v0.15, and if that's incorrect or insufficient, I'll add in the provider schema check as well. I also updated the documentation, and, since we've considered this before, bumped the FormatVersions for both jsonstate and jsonplan.	5 years ago
James Bardin	b7f8ef4dc6	handle unexpected changes to unknown block ... An unknown block represents a dynamic configuration block with an unknown for_each value. We were not catching the case where a provider modified this value unexpectedly, which would crash with block of type NestingList blocks where the config value has no length for comparison.	5 years ago
James Bardin	09c33fa449	account for noop deposed instances in json plan ... When rendering a json plan, we need to account for deposed instances that have become a noop rather than a destroy.	5 years ago
Nick Fagerlund	ab70879b9b	Remote backend: Stop setting message when creating runs ... Historically, we've used TFC's default run messages as a sort of dumping ground for metadata about the run. We've recently decided to mostly stop doing that, in favor of: - Only specifying the run's source in the default message. - Letting TFC itself handle the default messages. Today, the remote backend explicitly sets a run message, overriding any default that TFC might set. This commit removes that explicit message so we can allow TFC to sort it out. This shouldn't have any bad effect on TFE out in the wild, because it's known how to set a default message for remote backend runs since late 2018.	5 years ago
Alisdair McDiarmid	24ace6ae7d	Merge pull request #28864 from hashicorp/alisdair/fix-remote-backend-multi-workspace-state-migration ... Fix remote backend multi workspace state migration	5 years ago
Kristin Laemmert	5a48530f47	tools: remove terraform-bundle. (#28876) ... * tools: remove terraform-bundle. terraform-bundle is no longer supported in the main branch of terraform. Users can build terraform-bundle from terraform tagged v0.15 and older. * add a README pointing users to the v0.15 branch	5 years ago
Martin Atkins	51b0aee36c	addrs: ModuleRegistryPackage for representing module registry packages ... Previously we had a separation between ModuleSourceRemote and ModulePackage as a way to represent within the type system that there's an important difference between a module source address and a package address, because module packages often contain multiple modules and so a ModuleSourceRemote combines a ModulePackage with a subdirectory to represent one specific module. This commit applies that same strategy to ModuleSourceRegistry, creating a new type ModuleRegistryPackage to represent the different sort of package that we use for registry modules. Again, the main goal here is to try to reflect the conceptual modelling more directly in the type system so that we can more easily verify that uses of these different address types are correct. To make use of that, I've also lightly reworked initwd's module installer to use addrs.ModuleRegistryPackage directly, instead of a string representation thereof. This was in response to some earlier commits where I found myself accidentally mixing up package addresses and source addresses in the installRegistryModule method; with this new organization those bugs would've been caught at compile time, rather than only at unit and integration testing time. While in the area anyway, I also took this opportunity to fix some historical confusing names of fields in initwd.ModuleInstaller, to be clearer that they are only for registry packages and not for all module source address types.	5 years ago
Martin Atkins	7b2a0284e0	initwd: Fix registry acceptance tests for upstream registry changes ... We have some tests in this package that install real modules from the real registry at registry.terraform.io. Those tests were written at an earlier time when the registry's behavior was to return the URL of a .tar.gz archive generated automatically by GitHub, which included an extra level of subdirectory that would then be reflected in the paths to the local copies of these modules. GitHub started rate limiting those tar archives in a way that Terraform's module installer couldn't authenticate to, and so the registry switched to returning direct git repository URLs instead, which don't have that extra subdirectory and so the local paths on disk now end up being a little different, because the actual module directories are at a different subdirectory of the package.	5 years ago
Martin Atkins	17b766c3ea	configs: EntersNewPackage methods for descendant modules ... Now that we (in the previous commit) refactored how we deal with module sources to do the parsing at config loading time rather than at module installation time, we can expose a method to centralize the determination for whether a particular module call (and its resulting Config object) enters a new external package. We don't use this for anything yet, but in later commits we will use this for some cross-module features that are available only for modules belonging to the same package, because we assume that modules grouped together in a package can change together and thus it's okay to permit a little more coupling of internal details in that case, which would not be appropriate between modules that are versioned separately.	5 years ago
Martin Atkins	1a8da65314	Refactoring of module source addresses and module installation ... It's been a long while since we gave close attention to the codepaths for module source address parsing and external module package installation. Due to their age, these codepaths often diverged from our modern practices such as representing address types in the addrs package, and encapsulating package installation details only in a particular location. In particular, this refactor makes source address parsing a separate step from module installation, which therefore makes the result of that parsing available to other Terraform subsystems which work with the configuration representation objects. This also presented the opportunity to better encapsulate our use of go-getter into a new package "getmodules" (echoing "getproviders"), which is intended to be the only part of Terraform that directly interacts with go-getter. This is largely just a refactor of the existing functionality into a new code organization, but there is one notable change in behavior here: the source address parsing now happens during configuration loading rather than module installation, which may cause errors about invalid addresses to be returned in different situations than before. That counts as backward compatible because we only promise to remain compatible with configurations that are _valid_, which means that they can be initialized, planned, and applied without any errors. This doesn't introduce any new error cases, and instead just makes a pre-existing error case be detected earlier. Our module registry client is still using its own special module address type from registry/regsrc for now, with a small shim from the new addrs.ModuleSourceRegistry type. Hopefully in a later commit we'll also rework the registry client to work with the new address type, but this commit is already big enough as it is.	5 years ago
Martin Atkins	2e7db64968	addrs: Representation of module source addresses ... We've previously had the syntax and representation of module source addresses pretty sprawled around the codebase and intermingled with other systems such as the module installer. I've created a factored-out implementation here with the intention of enabling some later refactoring to centralize the address parsing as part of configuration decoding, and thus in turn allow the parsing result to be seen by other parts of Terraform that interact with configuration objects, so that they can more robustly handle differences between local and remote modules, and can present module addresses consistently in the UI.	5 years ago
Martin Atkins	f25935649a	getmodules: Beginnings of a new package about Terraform module packages ... This new package aims to encapsulate all of our interactions with go-getter to fetch remote module packages, to ensure that the rest of Terraform will only use the small subset of go-getter functionality that our modern module installer uses. In older versions of Terraform, go-getter was the entire implementation of module installation, but along the way we found that several aspects of its design are poor fit for Terraform's needs, and so now we're using it as just an implementation detail of Terraform's handling of remote module packages only, hiding it behind this wrapper API which exposes only the services that our module installer needs. This new package isn't actually used yet, but in a later commit we will change all of the other callers to go-getter to only work indirectly through this package, so that this will be the only package that actually imports the go-getter packages.	5 years ago
Martin Atkins	e5f52e56f8	addrs: Rename DefaultRegistryHost to DefaultProviderRegistryHost ... As the comment notes, this hostname is the default for provide source addresses. We'll shortly be adding some address types to represent module source addresses too, and so we'll also have DefaultModuleRegistryHost for that situation. (They'll actually both contain the the same hostname, but that's a coincidence rather than a requirement.)	5 years ago
Alisdair McDiarmid	3f0c6a2217	cli: Add -ignore-remote-version flag for init ... When performing state migration to a remote backend target, Terraform may fail due to mismatched remote and local Terraform versions. Here we add the `-ignore-remote-version` flag to allow users to ignore this version check when necessary.	5 years ago
Alisdair McDiarmid	6692336541	cli: Fix state migration version check ... When migrating multiple local workspaces to a remote backend target using the `prefix` argument, we need to perform the version check against all existing workspaces returned by the `Workspaces` method. Failing to do so will result in a version check error.	5 years ago
Martin Atkins	4e74a7a4f1	initwd: Error message for local paths escaping module packages ... Our module installer has a somewhat-informal idea of a "module package", which is some external thing we can go fetch in order to add one or more modules to the current configuration. Our documentation doesn't talk much about it because most users seem to have found the distinction between external and local modules pretty intuitive without us throwing a lot of funny terminology at them, but there are some situations where the distinction between a module and a module package are material to the end-user. One such situation is when using an absolute rather than relative filesystem path: we treat that as an external package in order to make the resulting working directory theoretically "portable" (although users can do various other things to defeat that), and so Terraform will copy the directory into .terraform/modules in the same way as it would download and extract a remote archive package or clone a git repository. A consequence of this, though, is that any relative paths called from inside a module loaded from an absolute path will fail if they try to traverse upward into the parent directory, because at runtime we're actually running from a copy of the directory that's been taking out of its original context. A similar sort of situation can occur in a truly remote module package if the author accidentally writes a "../" source path that traverses up out of the package root, and so this commit introduces a special error message for both situations that tries to be a bit clearer about there being a package boundary and use that to explain why installation failed. We would ideally have made escaping local references like that illegal in the first place, but sadly we did not and so when we rebuilt the module installer for Terraform v0.12 we ended up keeping the previous behavior of just trying it and letting it succeed if there happened to somehow be a matching directory at the given path, in order to remain compatible with situations that had worked by coincidence rather than intention. For that same reason, I've implemented this as a replacement error message we will return only if local module installation was going to fail anyway, and thus it only modifies the error message for some existing error situations rather than introducing new error situations. This also includes some light updates to the documentation to say a little more about how Terraform treats absolute paths, though aiming not to get too much into the weeds about module packages since it's something that most users can get away with never knowing.	5 years ago
CJ Horton	2f980f1dfe	Merge pull request #28820 from hashicorp/radditude/refresh-error-message ... backend/remote: point refresh users towards -refresh-only	5 years ago
CJ Horton	314d322b59	backend/remote: encourage use of -refresh-only	5 years ago
Alisdair McDiarmid	a3b1764fd7	backend/local: Fix lock when applying stale plan ... When returning from the context method, a deferred function call checked for error diagnostics in the `diags` variable, and unlocked the state if any exist. This means that we need to be extra careful to mutate that variable when returning errors, rather than returning a different set of diags in the same position. Previously this would result in an invalid plan file causing a lock to become stuck.	5 years ago
Alisdair McDiarmid	52591ba5e9	Merge pull request #28806 from hashicorp/alisdair/remove-json-drift-summary ... command/views: Remove unused drift summary message	5 years ago
James Bardin	99d0266585	return error for invalid resource import ... Most legacy provider resources do not implement any import functionality other than returning an empty object with the given ID, relying on core to later read that resource and obtain the complete state. Because of this, we need to check the response from ReadResource for a null value, and use that as an indication the import id was invalid.	5 years ago
Alisdair McDiarmid	953738c128	command/views: Remove unused drift summary message ... This was dead code, and there is no clear way to retrieve this information, as we currently only derive the drift information as part of the rendering process.	5 years ago
James Bardin	3bf498422c	typo	5 years ago
James Bardin	45b5c289a0	no drift with only deposed changes ... Changes to deposed instances should not triggers any drift to be rendered, as they will have nothing to display.	5 years ago
James Bardin	57aa7c6025	skip drift rendering for deposed resources ... Deposed instances have no current state and are only scheduled for deletion, so there is no reason to try and render drift on these instances.	5 years ago
James Bardin	aae642fb07	fix schemas and add deposed test ... The schemas for provider and the resources didn't match, so the changes were not going to be rendered at all. Add a test which contains a deposed resource.	5 years ago
James Bardin	95e788f13b	deposed instances should not be counted as orphans ... Do not add orphan nodes for resources that only contain deposed instances.	5 years ago