61 Commits (d857f2013f8ead2f4e8f69a4429c1af15a5d438b)

Author	SHA1	Message	Date
Sarah French	6f07b460a7	Update and add new tests for module-parsing logic (#37176) ... * Add comments, rename tests, reorder tests This commit doesn't change or add any logic * Add test for overriding a cloud block with a backend block * Add tests showing multiple instances of cloud or backend blocks cause an error * Add test showing that backend and cloud blocks cannot be in the same terraform block * Add test case asserting it's not valid to have both cloud and backend blocks across files in a module * Clarify comments and replace some comments with test names * Replace comments with test names	11 months ago
Samsondeen	a5434b421f	Build and Validate list resource nodes (#37011)	11 months ago
Samsondeen	b3c5118ede	Add include_resource attr to list block (#37086)	11 months ago
Samsondeen	44eb0c69ef	Recognize tf query files (#36929)	12 months ago
Kristin Laemmert	386988cb50	Decode action blocks and action_triggers inside resource blocks. (#37030) ... * Decode action blocks and action_triggers inside resource blocks. This commit adds decoding of action and action_triggers inside terraform configuration. I added an Actions experiment as a hacky way of keeping the functionality out of main until we're ready for the alpha; this may never be an experiment but it's a handy feature flag so we don't have to do all the work in a long-lived feature branch. * remove legacy shim handling * validate that the referenced entry in actions is indeed an action	12 months ago
Samsondeen	7f29df96a0	Allow terraform init when only test files are present in directory (#36429)	1 year ago
Samsondeen	2f0f537e69	Terraform test: Execute eligible test runs in parallel (#36300)	1 year ago
James Bardin	be308d07f3	ephemeral configs test files	2 years ago
kmoe	fab3673f8d	lang: stabilise ephemeral values experiment (#35636) ... * lang: stabilise ephemeral values experiment Stabilise the ephemeral values experiment and ephemeralasnull experimental function. * simplify handling of apply time input vars As the final step in stabilising the ephemeral values experiment, we can remove the separate code path for handling input variables supplied via -var and -var-file during apply. The intent here is conveyed in the tests: when applying a saved plan, all ephemeral variables set in the plan (the "apply-time variables"), must be set, and no others. As per the TODO copied from the prototype, there is some more work to be done here in making the handling of undeclared variables during apply as sophisticated as that during plan, emitting helpful warnings (for example) when input variables are supplied unnecessarily via environment variables. * experiments: officially conclude EV experiment --------- Co-authored-by: Radek Simko <radek.simko@gmail.com>	2 years ago
Martin Atkins	6fad32b033	configs: Variables and outputs can be "ephemeral" ... When the ephemeral_values experiment is active, a module author can designate individual input variables and output values as being "ephemeral", which doesn't currently do anything but in future commits will represent that the values can be used only in locations that don't require Terraform to persist the value as part of state snapshots or saved plan files.	2 years ago
James Bardin	acc4c12ac2	variable validation rules must still point to self ... While variable validation rules can now include other references, they also still need to reference the variable itself. If the variable itself wasn't referenced, then the resulting `Invalid value for variable` error would not make sense. Having no self-references will also cause a panic when trying to reinsert the variable name into the evaluation context, because the context variables map could be nil. A check for a nil map does not need to be added however, because ensuring that a self-reference exists means there will always be at least 1 variable in scope.	2 years ago
Martin Atkins	3b620bfe69	terraform: Stabilize the variable_validation_crossref experiment ... Previously we introduced a language experiment that would permit variable validation rules to refer to other objects declared in the same module as the variable. Now that experiment is concluded and its behavior is available for all modules. This final version deviates slightly from the experiment: we learned from the experimental implementation that we accidentally made the "validate" command able to validate constant-valued input variables in child modules despite the usual rule that input variables are unknown during validation, because the previous compromise bypassed the main expression evaluator and built its own evaluation context directly. Even though that behavior was not intended, it's a useful behavior that is protected by our compatibility promises and so this commit includes a slightly hacky emulation of that behavior, in eval_variable.go, that fetches the variable value in the same way the old implementation would have and then modifies the hcl evaluation context to include that value, while preserving anything else that our standard evaluation context builder put in there. That narrowly preserves the old behavior for expressions that compare the variable value directly to a constant, while treating all other references (which were previously totally invalid) in the standard way. This quirk was already covered by the existing test TestContext2Validate_variableCustomValidationsFail, which fails if the special workaround is removed.	2 years ago
Liam Cervante	48fc2d3ac7	terraform test: Disallow version constraints within test files (#35161)	2 years ago
Martin Atkins	79479f52c9	terraform: Experimental support for variable validation cross-references ... When a module opts in to the variable_validation_crossref experiment, Terraform will treat variable validation rules in a similar way as any other dynamic expression, allowing it to refer to other objects that are declared in the same module as the variable. Because the check for a validation rule referring only to itself is now experiments-sensitive we need to do it only once the full module has been assembled, rather than during assembly of individual files, and so the "badref" test in package configs is now classified under "invalid-modules" rather than "invalid-files". It's otherwise unchanged, and the existing test that a validation rule must refer to at least _something_ is still present because that remains true even when opted in to the experiment. This is experimental for now because although it seems pretty plausible that it should be okay, this is allowing full evaluation in a new place where it wasn't allowed before and so we'd like to ask those interested in this feature to try it out with some of their realistic use-cases and make sure this solution is a good fit before we lock it in as a compatibility constraint.	2 years ago
Liam Cervante	3a75362cce	terraform test: run block names should be valid HCL identifiers (#34469) ... * terraform test: run block names should be valid HCL identifiers * fix test	2 years ago
James Bardin	8125fe56a4	cannot statically validate an import target moved ... The validation for checking if an import target is the source of a moved block is incorrect. Besides the noted FIXME of the check being incorrect for moved modules (and also resources scoped within modules); an import can only be statically identified at the granularity of a resource block, but moved block can have individual instances in the `from` field. Since we already validate that a resource block must _not_ exist in the config for `moved`, we can rely on that causing `import` to fail with a nonexistant resource block during evaluation if they have the same target.	2 years ago
Liam Cervante	ee9aa07e10	terraform test: implement the source attribute for mock providers (#34192) ... * complete implementation * add tests * address comments	2 years ago
Liam Cervante	05f877166d	terraform test: parse mocking structures with test files (#34143)	3 years ago
James Bardin	a2b189fd71	handle to expression from json files ... Because expression from JSON files are handled differently from HCL, we need to deal with these manually when evaluating the special "to" syntax. We take the same approach as was done for replace_triggered_by, where the raw string is extracted from the json expression, and re-evaluated as an HCL expression.	3 years ago
Liam Cervante	041b54fb4b	explicitely disallow duplicate run block names in test files (#33836)	3 years ago
Liam Cervante	f397954c52	testing framework: allow users to specify deeply nested testing directories (#33584)	3 years ago
Liam Cervante	4122ba86fc	terraform test: fix crash when using nested modules from test run blocks (#33589)	3 years ago
Liam Cervante	222676390c	testing framework: add validation for provider blocks in test files (#33542) ... * testing framework: add validation for provider blocks in test files * .tftest -> .tftest.hcl	3 years ago
Liam Cervante	c91f91cc9e	testing framework: change test file extension to .tftest.hcl (#33553)	3 years ago
Liam Cervante	4b34902fab	[testing framework] allow tests to define and override providers (#33466) ... * [testing framework] prepare for beta phase of development * [Testing Framework] Add module block to test run blocks * [testing framework] allow tests to define and override providers	3 years ago
Liam Cervante	5acc95dda7	[Testing Framework] Add module block to test run blocks (#33456) ... * [testing framework] prepare for beta phase of development * [Testing Framework] Add module block to test run blocks	3 years ago
Liam Cervante	f74a8d16cf	[testing framework] implement expect_failures functionality (#33443)	3 years ago
Liam Cervante	cad9aa9025	[Testing Framework] Add test file HCL configuration and parser functionality (#33325) ... * Add test structure to views package for rendering test output * Add test file HCL configuration and parser functionality * address comments	3 years ago
CJ Horton	adcecddb4f	Merge pull request #33193 from hashicorp/radditude/no-import-blocks-in-child-modules ... plannable import: disallow import blocks in child modules	3 years ago
kmoe	1172d40d7b	error if import target is move source (#33192) ... It is invalid for any import block to have a "to" argument matching any moved block's "from" argument.	3 years ago
CJ Horton	2dd89d9776	import blocks are only allowed in the root module	3 years ago
Liam Cervante	5d6c5a9a33	plannable import: add a provider argument to the import block (#33175) ... * command: keep our promises * remove some nil config checks Remove some of the safety checks that ensure plan nodes have config attached at the appropriate time. * add GeneratedConfig to plan changes objects Add a new GeneratedConfig field alongside Importing in plan changes. * add config generation package The genconfig package implements HCL config generation from provider state values. Thanks to @mildwonkey whose implementation of terraform add is the basis for this package. * generate config during plan If a resource is being imported and does not already have config, attempt to generate that config during planning. The config is generated from the state as an HCL string, and then parsed back into an hcl.Body to attach to the plan graph node. The generated config string is attached to the change emitted by the plan. * complete config generation prototype, and add tests * plannable import: add a provider argument to the import block * Update internal/configs/config.go Co-authored-by: kmoe <5575356+kmoe@users.noreply.github.com> * Update internal/configs/config.go Co-authored-by: kmoe <5575356+kmoe@users.noreply.github.com> * Update internal/configs/config.go Co-authored-by: kmoe <5575356+kmoe@users.noreply.github.com> * fix formatting and tests --------- Co-authored-by: Katy Moe <katy@katy.moe> Co-authored-by: kmoe <5575356+kmoe@users.noreply.github.com>	3 years ago
Liam Cervante	79f7f59155	Plannable import: Generate config for imported resources during the plan. (#33153) ... * command: keep our promises * remove some nil config checks Remove some of the safety checks that ensure plan nodes have config attached at the appropriate time. * add GeneratedConfig to plan changes objects Add a new GeneratedConfig field alongside Importing in plan changes. * add config generation package The genconfig package implements HCL config generation from provider state values. Thanks to @mildwonkey whose implementation of terraform add is the basis for this package. * generate config during plan If a resource is being imported and does not already have config, attempt to generate that config during planning. The config is generated from the state as an HCL string, and then parsed back into an hcl.Body to attach to the plan graph node. The generated config string is attached to the change emitted by the plan. * complete config generation prototype, and add tests --------- Co-authored-by: Katy Moe <katy@katy.moe>	3 years ago
Eugene Dementyev	36aac6b498	Fixes depends_on when overriding modules (#32796) ... Fixes #32795	3 years ago
Alisdair McDiarmid	551f6316fa	Do not apply type defaults to null values ... Applying object type defaults to null values can convert null to an object with partial attributes. This means that even a specified default value of null will not remain null after variable evaluation. In turn, the result can then be invalid, if not all attributes in an object type have defaults specified. This commit skips the type default application step during config load and variable evaluation if the default or given value is null of any type. We still perform type conversion.	4 years ago
Alisdair McDiarmid	64cf53bd7e	configs: Fix check block configuration diagnostics ... When validating self-references for resource and data source preconditions and postconditions, we previously did not nil-check the block's condition field, which caused a panic when the block had no condition. While fixing this I noticed that we were not validating that there are no self-references in the error message, so fixed that.	4 years ago
Martin Atkins	fda0579537	Experiments supported only in alpha/dev builds ... We originally introduced the idea of language experiments as a way to get early feedback on not-yet-proven feature ideas, ideally as part of the initial exploration of the solution space rather than only after a solution has become relatively clear. Unfortunately, our tradeoff of making them available in normal releases behind an explicit opt-in in order to make it easier to participate in the feedback process had the unintended side-effect of making it feel okay to use experiments in production and endure the warnings they generate. This in turn has made us reluctant to make use of the experiments feature lest experiments become de-facto production features which we then feel compelled to preserve even though we aren't yet ready to graduate them to stable features. In an attempt to tweak that compromise, here we make the availability of experiments _at all_ a build-time flag which will not be set by default, and therefore experiments will not be available in most release builds. The intent (not yet implemented in this PR) is for our release process to set this flag only when it knows it's building an alpha release or a development snapshot not destined for release at all, which will therefore allow us to still use the alpha releases as a vehicle for giving feedback participants access to a feature (without needing to install a Go toolchain) but will not encourage pretending that these features are production-ready before they graduate from experimental. Only language experiments have an explicit framework for dealing with them which outlives any particular experiment, so most of the changes here are to that generalized mechanism. However, the intent is that non-language experiments, such as experimental CLI commands, would also in future check Meta.AllowExperimentalFeatures and gate the use of those experiments too, so that we can be consistent that experimental features will never be available unless you explicitly choose to use an alpha release or a custom build from source code. Since there are already some experiments active at the time of this commit which were not previously subject to this restriction, we'll pragmatically leave those as exceptions that will remain generally available for now, and so this new approach will apply only to new experiments started in the future. Once those experiments have all concluded, we will be left with no more exceptions unless we explicitly choose to make an exception for some reason we've not imagined yet. It's important that we be able to write tests that rely on experiments either being available or not being available, so here we're using our typical approach of making "package main" deal with the global setting that applies to Terraform CLI executables while making the layers below all support fine-grain selection of this behavior so that tests with different needs can run concurrently without trampling on one another. As a compromise, the integration tests in the terraform package will run with experiments enabled _by default_ since we commonly need to exercise experiments in those tests, but they can selectively opt-out if they need to by overriding the loader setting back to false again.	4 years ago
Alisdair McDiarmid	479c71f93d	Merge pull request #31210 from hashicorp/update-optional-type-attributes ... Edit type constraints docs for style and flow	4 years ago
James Bardin	57c0deb01e	check for duplicate defaults from resource names	4 years ago
Alisdair McDiarmid	922de89be1	Conclude module variable optional attrs experiment	4 years ago
James Bardin	2581bc93cb	Check for duplicate types in required_providers ... Adding multiple local names for the same provider type in required_providers was not prevented, which can lead to ambiguous behavior in Terraform. Providers are always indexed by the providers fully qualified name, so duplicate local names cannot be differentiated.	4 years ago
Alisdair McDiarmid	650380f3ae	configs: Add default argument to optional() ... The optional modifier previously accepted a single argument: the attribute type. This commit adds an optional second argument, which specifies a default value for the attribute. To record the default values for a variable's type, we use a separate parallel structure of `typeexpr.Defaults`, rather than extending `cty.Type` to include a `cty.Value` of defaults (which may in turn include a `cty.Type` with defaults, and so on, and so forth). The new `typeexpr.TypeConstraintWithDefaults` returns a type constraint and defaults value. Defaults will be `nil` unless there are default values specified somewhere in the variable's type.	4 years ago
James Bardin	6eb3264d1a	parse replace_triggered_by in resource configs	4 years ago
Alisdair McDiarmid	bb35f02c95	Conclude preconditions/postconditions experiment	4 years ago
Martin Atkins	1879a39d2d	configs: Refined error messages for mismatched provider passing ... This set of diagnostic messages is under a number of unusual constraints that make them tough to get right: - They are discussing a couple finicky concepts which authors are likely to be encountering for the first time in these error messages: the idea of "local names" for providers, the relationship between those and provider source addresses, and additional ("aliased") provider configurations. - They are reporting concerns that span across a module call boundary, and so need to take care to be clear about whether they are talking about a problem in the caller or a problem in the callee. - Some of them are effectively deprecation warnings for features that might be in use by a third-party module that the user doesn't control, in which case they have no recourse to address them aside from opening a feature request with the upstream module maintainer. - Terraform has, for backward-compatibility reasons, a lot of implied default behaviors regarding providers and provider configurations, and these errors can arise in situations where Terraform's assumptions don't match the author's intent, and so we need to be careful to explain what Terraform assumed in order to make the messages understandable. After seeing some confusion with these messages in the community, and being somewhat confused by some of them myself, I decided to try to edit them a bit for consistency of terminology (both between the messages and with terminology in our docs), being explicit about caller vs. callee by naming them in the messages, and making explicit what would otherwise be implicit with regard to the correspondences between provider source addresses and local names. My assumed audience for all of these messages is the author of the caller module, because it's the caller who is responsible for creating the relationship between caller and callee. As much as possible I tried to make the messages include specific actions for that author to take to quiet the warning or fix the error, but some of the warnings are only fixable by the callee's maintainer and so those messages are, in effect, a suggestion to send a request to the author to stop using a deprecated feature. I think these new messages are also not ideal by any means, because it's just tough to pack so much information into concise messages while being clear and consistent, but I hope at least this will give users seeing these messages enough context to infer what's going on, possibly with the help of our documentation. I intentionally didn't change which cases Terraform will return warnings or errors -- only the message texts -- although I did highlight in a comment in one of the tests that what it is a asserting seems a bit suspicious to me. I don't intend to address that here; instead, I intend that note to be something to refer to if we later see a bug report that calls that behavior into question. This does actually silence some _unrelated_ warnings and errors in cases where a provider block has an invalid provider local name as its label, because our other functions for dealing with provider addresses are written to panic if given invalid addresses under the assumption that earlier code will have guarded against that. Doing this allowed for the provider configuration validation logic to safely include more information about the configuration as helpful context, without risking tripping over known-invalid configuration and panicking in the process.	4 years ago
Alisdair McDiarmid	b06fe04621	core: Check rule error message expressions ... Error messages for preconditions, postconditions, and custom variable validations have until now been string literals. This commit changes this to treat the field as an HCL expression, which must evaluate to a string. Most commonly this will either be a string literal or a template expression. When the check rule condition is evaluated, we also evaluate the error message. This means that the error message should always evaluate to a string value, even if the condition passes. If it does not, this will result in an error diagnostic. If the condition fails, and the error message also fails to evaluate, we fall back to a default error message. This means that the check rule failure will still be reported, alongside diagnostics explaining why the custom error message failed to render. As part of this change, we also necessarily remove the heuristic about the error message format. This guidance can be readded in future as part of a configuration hint system.	4 years ago
Alisdair McDiarmid	7ded73f266	configs: Validate pre/postcondition self-refs ... Preconditions and postconditions for resources and data sources may not refer to the address of the containing resource or data source. This commit adds a parse-time validation for this rule.	4 years ago
Martin Atkins	9076400436	configs: Decode preconditions and postconditions ... This allows precondition and postcondition checks to be declared for resources and output values as long as the preconditions_postconditions experiment is enabled. Terraform Core doesn't currently know anything about these features, so as of this commit declaring them does nothing at all.	4 years ago
James Bardin	0771a214d8	skip provider resolution when there are errors ... If there are errors loading the configuration, don't try to resolve the provider names, as those names may not even be valid.	4 years ago
Martin Atkins	affe2c3295	addrs: Expose the registry address parser's error messages ... Previously we ended up losing all of the error message detail produced by the registry address parser, because we treated any registry address failure as cause to parse the address as a go-getter-style remote address instead. That led to terrible feedback in the situation where the user _was_ trying to write a module address but it was invalid in some way. Although we can't really tighten this up in the default case due to our compatibility promises, it's never been valid to use the "version" argument with anything other than a registry address and so as a compromise here we'll use the presence of "version" as a heuristic for user intent to parse the source address as a registry address, and thus we can return a registry-address-specific error message in that case and thus give more direct feedback about what was wrong. This unfortunately won't help someone trying to install from the registry _without_ a version constraint, but I didn't want to let perfect be the enemy of the good here, particularly since we recommend using version constraints with registry modules anyway; indeed, that's one of the main benefits of using a registry rather than a remote source directly.	4 years ago