45 Commits (6440c839474f052d8997bb5cec19c2743e5108e6)

Author	SHA1	Message	Date
Pam Selle	e0e6f4fd13	Merge pull request #26443 from hashicorp/pselle/sensitive-var-module-merge ... Implement module merge for sensitive variable config	5 years ago
Pam Selle	f2fe0ceb0a	Implement module merge for sensitive config ... Implements merging behavior for when sensitive is set on a variable and adds testing accordingly	5 years ago
Pam Selle	ada6f45871	Add test file for bad sensitive value ... Adds a test file for an incorrectly typed (non-boolean) sensitive value to ensure it errors	5 years ago
Alisdair McDiarmid	13ed5af5c8	configs: Deprecate nested redundant interpolations ... Previous deprecations only included direct assignment of template-only expressions to arguments. That is, this was not deprecated: locals { foo = ["${var.foo}"] } This commit uses hclsyntax.VisitAll to detect and show deprecations for all template-only expressions, no matter how deep they are in a given expression.	6 years ago
Alisdair McDiarmid	18f9ea53b9	command: Providers schema shows required_providers ... The providers schema command is using the Config.ProviderTypes method, which had not been kept up to date with the changes to provider requirements detection made in Config.ProviderRequirements. This resulted in any currently-unused providers being omitted from the output. This commit changes the ProviderTypes method to use the same underlying logic as ProviderRequirements, which ensures that `required_providers` blocks are taken into account. Includes an integration test case to verify that this fixes the provider schemas command bug.	6 years ago
Alisdair McDiarmid	803c95e552	Merge pull request #26105 from hashicorp/alisdair/more-interpolation-only-expression-deprecations ... configs: More interpolation-only expr deprecations	6 years ago
Pam Selle	81a6cde3cf	Merge pull request #26203 from hashicorp/pselle/deprecate-vendor-provisioners ... Deprecate vendor (3rd party) provisioners	6 years ago
Pam Selle	02c1bddfe1	Create experiment for sensitive attribute	6 years ago
Pam Selle	7fef1db20d	Add sensitive variable configs test coverage	6 years ago
Kristin Laemmert	923e157b5c	configs: deprecate version argument inside provider configuration blocks (#26135) ... The version argument is deprecated in Terraform v0.14 in favor of required_providers and will be removed in a future version of terraform (expected to be v0.15). The provider configuration documentation already discourages use of 'version' inside provider configuration blocks, so it only needed an extra note that it is actively deprecated.	6 years ago
Alisdair McDiarmid	e693c14e5a	configs: More interpolation-only expr deprecations ... Extend the deprecation for interpolation-only expressions to include module calls, data sources, outputs, and locals.	6 years ago
Pam Selle	edc670d079	Add deprecation warning for vendor provisioners ... Adds a warning for chef, habitat, puppet, and salt-masterless provisioners, and a corresponding test file to test for the warning	6 years ago
Kristin Laemmert	df4627fecb	configs: include "providers" when overriding modules (#25496)	6 years ago
Kristin Laemmert	9cb8456f3d	configs: prevent panic with invalid type name (#25562) ... An invalid type name in a resource (or data source) could cause a panic when determining the implied provider for the resource. This commit adds verification that the type name is valid. It does not add a diagnostic, since the invalid type name would have already been caught by the parser. Fixes #25560	6 years ago
Martin Atkins	0a46ded3e7	addrs: Disallow provider source addresses starting with terraform- ... The main motivation here is to produce a helpful error if a user incorrectly uses the terraform-provider- prefix (which we see on provider VCS repositories and plugin executables) as part of the source address. However, this also more broadly blocks "terraform-" as a prefix in anticipation of whatever instinct causes the phenomenon where e.g. Python's PyPI has thousands of packages whose names start with "python-", even though everything on PyPI is for Python by definition. This is definitely not _necessary_, but it's better to be restrictive at first and weaken later as needed.	6 years ago
Alisdair McDiarmid	5e3d2dbdfa	configs: Fail early for invalid resource provider ... If a resource's "provider" reference is invalid and cannot be parsed, we should not store the reference as part of a `ProviderConfigRef`. Doing so creates an invalid data structure, which prevents us from using `MustParseProviderPart` with the name in later steps. The invalid test files added in this commit will cause a panic without the code change.	6 years ago
Alisdair McDiarmid	45f7da9678	configs: Fix nested provider requirements bug ... In a recent PR, we changed the provider requirements code to permit per-module requirements gathering, to enhance the provider command output. This had an incorrect implementation of recursive requirements gathering for the normal case, which resulted in only depth-1 modules being inspected. This commit fixes the broken recursion and adds a grandchild module to the unit tests as test coverage. This also demanded fixing the testNestedModuleConfigFromDir helper function to cope with nested modules in test configs.	6 years ago
Alisdair McDiarmid	1c1e4a4de0	command/providers: Show provider requirements tree ... Providers can be required from multiple sources. The previous implementation of the providers sub-command displayed only a flat list of provider requirements, which made it difficult to see which modules required each provider. This commit reintroduces the tree display of provider requirements, and adds a separate output block for providers required by existing state.	6 years ago
Martin Atkins	d1bc412220	configs: Custom variable validation is no longer experimental ... All of the feedback from the experiment described enhancements that can potentially be added later without breaking changes, so this change simply removes the experiment gate from the feature as originally implemented with no changes to its functionality. Further enhancements may follow in later releases, but the goal of this change is just to ship the feature exactly as it was under the experiment. Most of the changes here are cleaning up the experiment opt-ins from our test cases. The most important parts are in configs/experiments.go and in experiments/experiment.go .	6 years ago
Kristin Laemmert	041f4dd8ca	configs: require normalized provider local names (#24945) ... * addrs: replace NewLegacyProvider with NewDefaultProvider in ParseProviderSourceString ParseProviderSourceString was still defaulting to NewLegacyProvider when encountering single-part strings. This has been fixed. This commit also adds a new function, IsProviderPartNormalized, which returns a bool indicating if the string given is the same as a normalized version (as normalized by ParseProviderPart) or an error. This is intended for use by the configs package when decoding provider configurations. * terraform: fix provider local names in tests * configs: validate that all provider names are normalized The addrs package normalizes all source strings, but not the local names. This caused very odd behavior if for e.g. a provider local name was capitalized in one place and not another. We considered enabling case-sensitivity for provider local names, but decided that since this was not something that worked in previous versions of terraform (and we have yet to encounter any use cases for this feature) we could generate an error if the provider local name is not normalized. This error also provides instructions on how to fix it. * configs: refactor decodeProviderRequirements to consistently not set an FQN when there are errors	6 years ago
Alisdair McDiarmid	dcb8b45e0f	configs: Fix for resources with implied providers ... Previously, resources without explicit provider configuration (i.e. a `provider =` attribute) would be assigned a default provider based upon the resource type. For example, a resource `foo_bar` would be assigned provider `hashicorp/foo`. This behaviour did not work well with community or partner providers, with sources configured in `terraform.required_providers` blocks. With the following configuration: terraform { required_providers { foo = { source = "acme/foo" } } } resource foo_bar "a" { } the resource would be configured with the `hashicorp/foo` provider. This commit fixes this implied provider behaviour. First we look for a provider with local name matching the resource type in the module's required providers map. If one is found, this provider is assigned to the resource. Otherwise, we still fall back to a default provider.	6 years ago
Alisdair McDiarmid	7ca7b1f0fe	configs: Simplify required_providers blocks ... We now permit at most one `required_providers` block per module (except for overrides). This prevents users (and Terraform) from struggling to understand how to merge multiple `required_providers` configurations, with `version` and `source` attributes split across multiple blocks. Because only one `required_providers` block is permitted, there is no need to concatenate version constraints and resolve them. This allows us to simplify the structs used to represent provider requirements, aligning more closely with other structs in this package. This commit also fixes a semantic use-before-initialize bug, where resources defined before a `required_providers` block would be unable to use its source attribute. We achieve this by processing the module's `required_providers` configuration (and overrides) before resources. Overrides for `required_providers` work as before, replacing the entire block per provider.	6 years ago
Kristin Laemmert	269d511481	command/providers: refactor with new provider types and functions ... The providers command has been refactored to use the modern provider types and ProviderRequirements() functions. This resulted in a breaking change to the output: it no longer outputs the providers by module and no longer prints `(inherited)` or `(from state)` to show why a provider is included. We decided that at this time it was best to stick with the existing functions and make this change, but if we get feedback from the community we will revisit. Additional tests to exercise providers in modules and providers from state have been included.	6 years ago
Martin Atkins	3e3d8f6764	configs: Hint for a misplaced top-level required_providers block ... With provider dependencies now appearing inside a nested block, it seems likely that configuration examples showing dependencies out of context will sometimes mislead users into thinking that required_providers is toplevel. To give better feedback in that situation, we'll produce a specialized error in that case hinting the correct structure to the user.	6 years ago
Martin Atkins	7caf0b9246	addrs: ImpliedProviderForUnqualifiedType function ... This encapsulates the logic for selecting an implied FQN for an unqualified type name, which could either come from a local name used in a module without specifying an explicit source for it or from the prefix of a resource type on a resource that doesn't explicitly set "provider". This replaces the previous behavior of just directly calling NewDefaultProvider everywhere so that we can use a different implication for the local name "terraform", to refer to the built-in terraform provider rather than the stale one that's on registry.terraform.io for compatibility with other Terraform versions.	6 years ago
Martin Atkins	4061cbed38	internal/getproviders: A new shared model for provider requirements ... We've been using the models from the "moduledeps" package to represent our provider dependencies everywhere since the idea of provider dependencies was introduced in Terraform 0.10, but that model is not convenient to use for any use-case other than the "terraform providers" command that needs individual-module-level detail. To make things easier for new codepaths working with the new-style provider installer, here we introduce a new model type getproviders.Requirements which is based on the type the new installer was already taking as its input. We have new methods in the states, configs, and earlyconfig packages to produce values of this type, and a helper to merge Requirements together so we can combine config-derived and state-derived requirements together during installation. The advantage of this new model over the moduledeps one is that all of recursive module walking is done up front and we produce a simple, flat structure that is more convenient for the main use-cases of selecting providers for installation and then finding providers in the local cache to use them for other operations. This new model is _not_ suitable for implementing "terraform providers" because it does not retain module-specific requirement details. Therefore we will likely keep using moduledeps for "terraform providers" for now, and then possibly at a later time consider specializing the moduledeps logic for only what "terraform providers" needs, because it seems to be the only use-case that needs to retain that level of detail.	6 years ago
Kristin Laemmert	ed1aebbeda	terraform: large refactor to use Provider from configs.Resource (#24396) ... * terraform: large refactor to use Provider from configs.Resource configs.Resource.ImpliedProvider() now returns a string; it is the callers' responsibility to turn that into an addrs.Provider if needed. GraphNodeProviderConsumer ProvidedBy() no longer returns nil (reverting to earlier, pre-provider-fqn behavior): it will return either the provider set in config, provider set in state, or the default provider.	6 years ago
Kristin Laemmert	ef19fb6203	configs: attach provider fqn to Resource (#24382) ... * configs: attach provider fqn to Resource	6 years ago
Kristin Laemmert	1c78b26012	terraform: provider source test (#24342) ... * configs: parse provider source string during module merge This was the smallest unit of work needed to start writing provider source tests! * Update configs/parser_test.go Co-Authored-By: Alisdair McDiarmid <alisdair@users.noreply.github.com>	6 years ago
Martin Atkins	86f0b5191c	addrs: Stronger validation and normalization of provider namespace/type ... The provider FQN is becoming our primary identifier for a provider, so it's important that we are clear about the equality rules for these addresses and what characters are valid within them. We previously had a basic regex permitting ASCII letters and digits for validation and no normalization at all. We need to do at least case folding and UTF-8 normalization because these names will appear in file and directory names in case-insensitive filesystems and in repository names such as on GitHub. Since we're already using DNS-style normalization and validation rules for the hostname part, rather than defining an entirely new set of rules here we'll just treat the provider namespace and type as if they were single labels in a DNS name. Aside from some internal consistency, that also works out nicely because systems like GitHub use organization and repository names as part of hostnames (e.g. with GitHub Pages) and so tend to apply comparable constraints themselves. This introduces the possibility of names containing letters from alphabets other than the latin alphabet, and for latin letters with diacritics. That's consistent with our introduction of similar support for identifiers in the language in Terraform 0.12, and is intended to be more friendly to Terraform users throughout the world that might prefer to name their products using a different alphabet. This is also a further justification for using the DNS normalization rules: modern companies tend to choose product names that make good domain names, and now such names will be usable as Terraform provider names too.	6 years ago
James Bardin	84d1f5c688	convert destroy provisioner warnings to errors	6 years ago
Martin Atkins	8b511524d6	Initial steps towards AbsProviderConfig/LocalProviderConfig separation (#23978) ... * Introduce "Local" terminology for non-absolute provider config addresses In a future change AbsProviderConfig and LocalProviderConfig are going to become two entirely distinct types, rather than Abs embedding Local as written here. This naming change is in preparation for that subsequent work, which will also include introducing a new "ProviderConfig" type that is an interface that AbsProviderConfig and LocalProviderConfig both implement. This is intended to be largely just a naming change to get started, so we can deal with all of the messy renaming. However, this did also require a slight change in modeling where the Resource.DefaultProviderConfig method has become Resource.DefaultProvider returning a Provider address directly, because this method doesn't have enough information to construct a true and accurate LocalProviderConfig -- it would need to refer to the configuration to know what this module is calling the provider it has selected. In order to leave a trail to follow for subsequent work, all of the changes here are intended to ensure that remaining work will become obvious via compile-time errors when all of the following changes happen: - The concept of "legacy" provider addresses is removed from the addrs package, including removing addrs.NewLegacyProvider and addrs.Provider.LegacyString. - addrs.AbsProviderConfig stops having addrs.LocalProviderConfig embedded in it and has an addrs.Provider and a string alias directly instead. - The provider-schema-handling parts of Terraform core are updated to work with addrs.Provider to identify providers, rather than legacy strings. In particular, there are still several codepaths here making legacy provider address assumptions (in order to limit the scope of this change) but I've made sure each one is doing something that relies on at least one of the above changes not having been made yet. * addrs: ProviderConfig interface In a (very) few special situations in the main "terraform" package we need to make runtime decisions about whether a provider config is absolute or local. We currently do that by exploiting the fact that AbsProviderConfig has LocalProviderConfig nested inside of it and so in the local case we can just ignore the wrapping AbsProviderConfig and use the embedded value. In a future change we'll be moving away from that embedding and making these two types distinct in order to represent that mapping between them requires consulting a lookup table in the configuration, and so here we introduce a new interface type ProviderConfig that can represent either AbsProviderConfig or LocalProviderConfig decided dynamically at runtime. This also includes the Config.ResolveAbsProviderAddr method that will eventually be responsible for that local-to-absolute translation, so that callers with access to the configuration can normalize to an addrs.AbsProviderConfig given a non-nil addrs.ProviderConfig. That's currently unused because existing callers are still relying on the simplistic structural transform, but we'll switch them over in a later commit. * rename LocalType to LocalName Co-authored-by: Kristin Laemmert <mildwonkey@users.noreply.github.com>	6 years ago
James Bardin	35107acc46	allow "terraform" in destroy provisioners ... The values supplied in terraform are static, and do not create any dependencies.	6 years ago
James Bardin	c376905adc	it's safe for destroy provisioners to access path ... The path values are statically loaded, and do not create any dependencies that could cause problems with destroy provisioners.	6 years ago
Martin Atkins	ff4ea042c2	config: Allow module authors to specify validation rules for variables ... The existing "type" argument allows specifying a type constraint that allows for some basic validation, but often there are more constraints on a variable value than just its type. This new feature (requiring an experiment opt-in for now, while we refine it) allows specifying arbitrary validation rules for any variable which can then cause custom error messages to be returned when a caller provides an inappropriate value. variable "example" { validation { condition = var.example != "nope" error_message = "Example value must not be \"nope\"." } } The core parts of this are designed to do as little new work as possible when no validations are specified, and thus the main new checking codepath here can therefore only run when the experiment is enabled in order to permit having validations.	6 years ago
Martin Atkins	b90fb25321	experiments: a mechanism for opt-in experimental language features ... Traditionally we've preferred to release new language features in major releases only, because we can then use the beta cycle to gather feedback on the feature and learn about any usability challenges or other situations we didn't consider during our design in time to make those changes before inclusion in a stable release. This "experiments" feature is intended to decouple the feedback cycle for new features from the major release rhythm, and thus allow us to release new features in minor releases by first releasing them as experimental for a minor release or two, adjust for any feedback gathered during that period, and then finally remove the experiment gate and enable the feature for everyone. The intended model here is that anything behind an experiment gate is subject to breaking changes even in patch releases, and so any module using these experimental features will be broken by a future Terraform upgrade. The behavior implemented here is: - Recognize a new "experiments" setting in the "terraform" block which allows module authors to explicitly opt in to experimental features. terraform { experiments = [resource_for_each] } - Generate a warning whenever loading a module that has experiments enabled, to avoid accidentally depending on experimental features and thus risking unexpected breakage on next Terraform upgrade. - We check the enabled experiments against the configuration at module load time, which means that experiments are scoped to a particular module. Enabling an experiment in one module does not automatically enable it in any other module. This experiments mechanism is itself an experiment, and so I'd like to use the resource for_each feature to trial it. Because any configuration using experiments is subject to breaking changes, we are free to adjust this experiments feature in future releases as we see fit, but once for_each is shipped without an experiment gate we'll be blocked from making significant changes to it until the next major release at least.	6 years ago
James Bardin	8547603ff5	deprecation warning for destroy provisioner refs ... Add deprecation warning for references from destroy provisioners or their connections to external resources or values. In order to ensure resource destruction can be completed correctly, destroy nodes must be able to evaluate with only their instance state. We have sufficient information to validate destroy-time provisioners early on during the config loading process. Later on these can be converted to hard errors, and only allow self, count.index, and each.key in destroy provisioners. Limited the provisioner and block evaluation scope later on is tricky, but if the references can never be loaded, then they will never be encountered during evaluation.	6 years ago
Martin Atkins	91752f02da	configs: Warn for deprecated interpolation and quoted type constraints ... Following on from de652e22a2, this introduces deprecation warnings for when an attribute value expression is a template with only a single interpolation sequence, and for variable type constraints given in quotes. As with the previous commit, we allowed these deprecated forms with no warning for a few releases after v0.12.0 to ensure that folks who need to write cross-compatible modules for a while during upgrading would be able to do so, but we're now marking these as explicitly deprecated to guide users towards the new idiomatic forms. The "terraform 0.12upgrade" tool would've already updated configurations to not hit these warnings for those who had pre-existing configurations written for Terraform 0.11. The main target audience for these warnings are newcomers to Terraform who are learning from existing examples already published in various spots on the wider internet that may be showing older Terraform syntax, since those folks will not be running their configurations through the upgrade tool. These warnings will hopefully guide them towards modern Terraform usage during their initial experimentation, and thus reduce the chances of inadvertently adopting the less-readable legacy usage patterns in greenfield projects.	6 years ago
Martin Atkins	79dc808614	configs: Emit warnings for deprecated quoted references/keywords ... Terraform 0.12.0 removed the need for putting references and keywords in quotes, but we disabled the deprecation warnings for the initial release in order to avoid creating noise for folks who were intentionally attempting to maintain modules that were cross-compatible with both Terraform 0.11 and Terraform 0.12. However, with Terraform 0.12 now more widely used, the lack of these warnings seems to be causing newcomers to copy the quoted versions from existing examples on the internet, which is perpetuating the old and confusing quoted form in newer configurations. In preparation for phasing out these deprecated forms altogether in a future major release, and for the shorter-term benefit of giving better feedback to newcomers when they are learning from outdated examples, we'll now re-enable those deprecation warnings, and be explicit that the old forms are intended for removal in a future release. In order to properly test this, we establish a new set of test configurations that explicitly mark which warnings they are expecting and verify that they do indeed produce those expected warnings. We also verify that the "success" tests do _not_ produce warnings, while removing the ones that were previously written to succeed but have their warnings ignored.	6 years ago
Pam Selle	e7d8ac5ad7	Remove panic, update comment	7 years ago
Thayne McCombs	7c678d104f	Add support for for_each for data blocks. ... This also fixes a few things with resource for_each: It makes validation more like validation for count. It makes sure the index is stored in the state properly.	7 years ago
Pam Selle	7d905f6777	Resource for_each	7 years ago
Alex Pilon	0450f487fa	move IsEmptyDir to configs package	7 years ago
James Bardin	8111050c66	ensure we record diagnostics from nested modules ... When loading nested modules, the child module diagnostics were dropped in the recursive function. This mean that the config from the submodules wasn't fully loaded, even though no errors were reported to the user. This caused further problems if the plan was stored in a plan file, when means only the partial configuration was stored for the subsequent apply operation, which would result in unexplained "Resource node has no configuration attached" errors later on. Also due to the child module diagnostics being lost, any newly added nested modules would be silently ignored until `init` was run again manually.	7 years ago
Radek Simko	5b9f2fafc8	Standardise directory name for test data	7 years ago