47 Commits (be2ad69eda9c66ca2c3ee793720d79faee997334)

Author	SHA1	Message	Date
Liam Cervante	4d837df546	Plannable import: Add generated config to JSON and human-readable plan output (#33154) ... * command: keep our promises * remove some nil config checks Remove some of the safety checks that ensure plan nodes have config attached at the appropriate time. * add GeneratedConfig to plan changes objects Add a new GeneratedConfig field alongside Importing in plan changes. * add config generation package The genconfig package implements HCL config generation from provider state values. Thanks to @mildwonkey whose implementation of terraform add is the basis for this package. * generate config during plan If a resource is being imported and does not already have config, attempt to generate that config during planning. The config is generated from the state as an HCL string, and then parsed back into an hcl.Body to attach to the plan graph node. The generated config string is attached to the change emitted by the plan. * complete config generation prototype, and add tests * Plannable import: Add generated config to json and human-readable plan output --------- Co-authored-by: Katy Moe <katy@katy.moe>	3 years ago
Liam Cervante	54c1c1162f	[Plannable Import] Implement human-readable plan rendering (#33113) ... * [plannable import] embed the resource id within the changes * add the plannable imports to the json and human plans * latest importing struct	3 years ago
hashicorp-copywrite[bot]	325d18262e	[COMPLIANCE] Add Copyright and License Headers	3 years ago
Liam Cervante	aae6990d38	Add native `plantimestamp` function to Terraform (#32980) ... * add plantimestamp function * documentation * add missing links * fix typo	3 years ago
Liam Cervante	978263efe9	Checks: Introduce check blocks into the terraform node and transform graph (#32735) ... * Add support for scoped resources * refactor existing checks addrs and add check block addr * Add configuration for check blocks * introduce check blocks into the terraform node and transform graph * address comments * address comments * don't execute checks during destroy operations * don't even include check nodes for destroy operations	3 years ago
Liam Cervante	d212a72d1d	structured run output: impose canonical ordering on jsonstate and jsonplan packages (#32649)	3 years ago
Liam Cervante	d818d7850d	Structured Renderer: use the new renderer when rendering the state in addition to the plan (#32629) ... * Use the new renderer when rendering the state * remove confusing and unneeded comment	3 years ago
Liam Cervante	6dc49150b7	Structured Plan Renderer: Read the data source schemas from the right place (#32532) ... * read the data source schemas from the right place * address comments and add test	3 years ago
Liam Cervante	8d61c5bfc4	Use the new structured renderer in place of the old diffs package (#32520) ... * Use the new structured renderer in place of the old diffs package * remove old plan tests * refresh only plans should show moved resources in the refresh section	3 years ago
Liam Cervante	e015b15f12	Structured Plan Renderer: Remove attributes that do not match the relevant attributes filter (#32509) ... * remove attributes that do not match the relevant attributes filter * fix formatting * fix renderer function, don't drop irrelevant attributes just mark them as no-ops * fix imports	3 years ago
Liam Cervante	95782f2491	Structured plan renderer: Implement the main functionality for the renderer (#32496) ... * raw unmodified broken tests * tests execute, no panics * fix whitespace differences * fix all the tests * fix tests * actually fix tests * add missing plan metadata into the renderer * address comments * complete merge * remove TODO raising questions about outputs, they are fixed * missing bold on plan	3 years ago
Liam Cervante	af0ff90d6e	Remove outputs from the jsonplan that are not from the root module (#32503)	3 years ago
James Bardin	96b1318167	remove deprecated condition_results from plan	3 years ago
Liam Cervante	b917154a97	Make required JSON structured output structures public for processing (#32367) ... * prep for processing the structured run output * undo unwanted change to a json key	3 years ago
kmoe	dec48a8510	plans: indicate when resource deleted due to move (#31695) ... Add a new ChangeReason, ReasonDeleteBecauseNoMoveTarget, to provide better information in cases where a planned deletion is due to moving a resource to a target not in configuration. Consider a case in which a resource instance exists in state at address A, and the user adds a moved block to move A to address B. Whether by the user's intention or not, address B does not exist in configuration. Terraform combines the move from A to B, and the lack of configuration for B, into a single delete action for the (previously nonexistent) entity B. Prior to this commit, the Terraform plan will report that resource B will be destroyed because it does not exist in configuration, without explicitly connecting this to the move. This commit provides the user an additional clue as to what has happened, in a case in which Terraform has elided a user's action and inaction into one potentially destructive change.	4 years ago
Martin Atkins	0e4e9f7706	addrs: Be explicit about checkable object address kinds ... Previously we were attempting to infer the checkable object address kind of a given address by whether it included "output" in the position where a resource type name would otherwise go. That was already potentially risky because we've historically not prevented a resource type named "output", and it's also a forward-compatibility hazard in case we introduce additional object kinds with entirely-new addressing schemes in future. Given that, we'll instead always be explicit about what kind of address we're storing in a wire or file format, so that we can make sure to always use the intended parser when reading an address back into memory, or return an error if we encounter a kind we're not familiar with.	4 years ago
Martin Atkins	fe7e6f970e	command/jsonplan: Include new-style check results in JSON plan output ... This is a new-shaped representation of check results which follows the two-tiered structure of static objects and dynamic instances of objects, thereby allowing consumers to see which checkable objects exist in the configuration even if a dynamic evaluation error prevented actually expanding them all to determine their declared instances. Eventually we'll include this in the state too, but this initially adds it only to the plan in order to replace the now-deprecated experimental conditions result that was present but undocumented in Terraform v1.2.	4 years ago
Martin Atkins	9e4861adbb	states: Two-level representation of check results ... A significant goal of the design changes around checks in earlier commits (with the introduction of package "checks") was to allow us to differentiate between a configuration object that we didn't expand at all due to an upstream error, which has _unknown_ check status, and a configuration object that expanded to zero dynamic objects, which therefore has a _passing_ check status. However, our initial lowering of checks.State into states.CheckResults stayed with the older model of just recording each leaf check in isolation, without any tracking of the containers. This commit therefore lightly reworks our representation of check results in the state and plan with two main goals: - The results are grouped by the static configuration object they came from, and we capture an aggregate status for each of those so that we can differentiate an unknown aggregate result from a passing aggregate result which has zero dynamic associated objects. - The granularity of results is whole checkable objects rather than individual checks, because checkable objects have durable addresses between runs, but individual checks for an object are more of a syntactic convenience to make it easier for module authors to declare many independent conditions that each have their own error messages. Since v1.2 exposed some details of our checks model into the JSON plan output there are some unanswered questions here about how we can shift to reporting in the two-level heirarchy described above. For now I've preserved structural compatibility but not semantic compatibility: any parser that was written against that format should still function but will now see fewer results. We'll revisit this in a later commit and consider other structures and what to do about our compatibility constraint on the v1.2 structure. Otherwise though, this is an internal-only change which preserves all of the existing main behaviors of conditions as before, and just gets us ready to build user-facing features in terms of this new structure.	4 years ago
Martin Atkins	3785619f93	core: Use the new checks package for condition tracking ... The "checks" package is an expansion what we previously called plans.Conditions to accommodate a new requirement that we be able to track which checks we're expecting to run even if we don't actually get around to running them, which will be helpful when we start using checks as part of our module testing story because test reporting tools appreciate there being a relatively consistent set of test cases from one run to the next. So far this should be essentially a no-op change from an external functionality standpoint, aside from some minor adjustments to how we report some of the error and warning cases from condition evaluation in light of the fact that the "checks" package can now track errors as a different outcome than a failure of a valid check. As is often the case with anything which changes what we track in the EvalContext and persist between plan and apply, Terraform Core is pretty brittle and so this had knock-on effects elsewhere too. Again, the goal is for these changes to not create any material externally-visible difference, and just to accommodate the new assumption that there will always be a "checks" object available for tracking during a graph walk.	4 years ago
Martin Atkins	783a07d9e8	build: Use Go 1.19 ... Go 1.19's "fmt" has some awareness of the new doc comment formatting conventions and adjusts the presentation of the source comments to make it clearer how godoc would interpret them. Therefore this commit includes various updates made by "go fmt" to acheve that. In line with our usual convention that we make stylistic/grammar/spelling tweaks typically only when we're "in the area" changing something else anyway, I also took this opportunity to review most of the comments that this updated to see if there were any other opportunities to improve them.	4 years ago
Alisdair McDiarmid	c7bc82bd87	Merge pull request #31235 from hashicorp/alisdair/json-plan-unknown-outputs ... json-output: Extended detail for unknown outputs	4 years ago
Alisdair McDiarmid	9497b2cd6f	json-output: Fix unknowns for tuples and sets ... The JSON output for sequences previously omitted unknown values for tuples and sets, which made it impossible to interpret the corresponding unknown marks. For example, consider this resource: resource "example_resource" "example" { tags = toset(["alpha", timestamp(), "charlie"]) } This would previously be encoded in JSON as: "after": { "tags": ["alpha", "charlie"] }, "after_unknown": { "id": true, "tags": [false, true, false] }, That is, the timestamp value would be omitted from the output altogether, while the corresponding unknown marks would include a value for each of the set members. This commit changes the behaviour to: "after": { "tags": ["alpha", null, "charlie"] }, "after_unknown": { "id": true, "tags": [false, true, false] }, This aligns tuples and sets with the prior behaviour for lists, and makes it clear which elements are known and which are unknown.	4 years ago
Alisdair McDiarmid	48d64eabb2	json-output: Extended detail for unknown outputs ... Planned output changes are represented in the JSON output format using the same change object as planned resource changes. This structure includes an `after` value and a parallel `after_unknown` value, which can be combined to determine which specific parts of a value are known only at apply time. Previously, structured output values would be marked in the JSON plan as coarsely known or unknown, even if only some subset of the structure will be known only at apply time. This simplification was unnecessary, and this commit reuses the same logic for resource changes to give more information to consumers of this format. For example, consider this output: output "bar" { value = tolist([ "hello", timestamp(), "world", ]) } The plan output for this output would be: + bar = [ + "hello", + (known after apply), + "world", ] For the same plan, the JSON output was previously: "bar": { "actions": [ "create" ], "before": null, "after_unknown": true, "before_sensitive": false, "after_sensitive": false } After this commit, the output is instead: "bar": { "actions": [ "create" ], "before": null, "after": [ "hello", null, "world" ], "after_unknown": [ false, true, false ], "before_sensitive": false, "after_sensitive": false }	4 years ago
Martin Atkins	4cffff24b1	core: Report reason for deferring data read until apply ... We have two different reasons why a data resource might be read only during apply, rather than during planning as usual: the configuration contains unknown values, or the data resource as a whole depends on a managed resource which itself has a change pending. However, we didn't previously distinguish these two in a way that allowed the UI to describe the difference, and so we confusingly reported both as "config refers to values not yet known", which in turn led to a number of reasonable questions about why Terraform was claiming that but then immediately below showing the configuration entirely known. Now we'll use our existing "ActionReason" mechanism to tell the UI layer which of the two reasons applies to a particular data resource instance. The "dependency pending" situation tends to happen in conjunction with "config unknown", so we'll prefer to refer that the configuration is unknown if both are true.	4 years ago
Alisdair McDiarmid	12c8f9498c	json-output: Add output type to JSON format ... Previously the supported JSON plan and state formats included only serialized output values, which was a lossy serialization of the Terraform type system. This commit adds a type field in the usual cty JSON format, which allows reconstitution of the original value. For example, previously a list(string) and a set(string) containing the same values were indistinguishable. This change serializes these as follows: { "value": ["a","b","c"], "type": ["list","string"] } and: { "value": ["a","b","c"], "type": ["set","string"] }	4 years ago
James Bardin	3a0a019521	round-trip replace triggers	4 years ago
Alisdair McDiarmid	c5d10bdef1	core: Store condition block results in plan ... In order to include condition block results in the JSON plan output, we must store them in the plan and its serialization. Terraform can evaluate condition blocks multiple times, so we must be able to update the result. Accordingly, the plan.Conditions object is a map with keys representing the condition block's address. Condition blocks are not referenceable in any other context, so this address form cannot be used anywhere in the configuration. The commit includes a new test case for the JSON output of a refresh-only plan, which is currently the only way for a failing condition result to be rendered through this path.	4 years ago
James Bardin	8c5e11d41a	add relevant_attributes to the json plan format ... Add the resource instances and individual attributes which may have contributed to the planned changes to the json format of the plan. We use the existing path encoding for individual attributes, which is already used in the replace_paths change field.	4 years ago
Alisdair McDiarmid	3e4d6b252f	jsonplan: Improve performance for deep objects ... When calculating the unknown values for JSON plan output, we would previously recursively call the `unknownAsBool` function on the current sub-tree twice, if any values were unknown. This was wasteful, but not noticeable for normal Terraform resource shapes. However for deeper nested object values, such as Kubernetes manifests, this was a severe performance problem, causing `terraform show -json` to take several hours to render a plan. This commit reuses the already calculated unknown value for the subtree, and adds benchmark coverage to demonstrate the improvement.	4 years ago
Alisdair McDiarmid	fe8183c4af	json: Increment JSON plan format version ... The JSON plan configuration data now includes a `full_name` field for providers. This addition warrants a backwards compatible increment to the version number.	4 years ago
Martin Atkins	36c4d4c241	core and backend: remove redundant handling of default variable values ... Previously we had three different layers all thinking they were responsible for substituting a default value for an unset root module variable: - the local backend, via logic in backend.ParseVariableValues - the context.Plan function (and other similar functions) trying to preprocess the input variables using terraform.mergeDefaultInputVariableValues . - the newer prepareFinalInputVariableValue, which aims to centralize all of the variable preparation logic so it can be common to both root and child module variables. The second of these was also trying to handle type constraint checking, which is also the responsibility of the central function and not something we need to handle so early. Only the last of these consistently handles both root and child module variables, and so is the one we ought to keep. The others are now redundant and are causing prepareFinalInputVariableValue to get a slightly corrupted view of the caller's chosen variable values. To rectify that, here we remove the two redundant layers altogether and have unset root variables pass through as cty.NilVal all the way to the central prepareFinalInputVariableValue function, which will then handle them in a suitable way which properly respects the "nullable" setting. This commit includes some test changes in the terraform package to make those tests no longer rely on the mergeDefaultInputVariableValues logic we've removed, and to instead explicitly set cty.NilVal for all unset variables to comply with our intended contract for PlanOpts.SetVariables, and similar. (This is so that we can more easily catch bugs in callers where they _don't_ correctly handle input variables; it allows us to distinguish between the caller explicitly marking a variable as unset vs. not describing it at all, where the latter is a bug in the caller.)	4 years ago
Alisdair McDiarmid	b9f3dab035	json-output: Release format version 1.0	4 years ago
Martin Atkins	a1a713cf28	core: Report ActionReasons when we plan to delete "orphans" ... There are a few different reasons why a resource instance tracked in the prior state might be considered an "orphan", but previously we reported them all identically in the planned changes. In order to help users understand the reason for a surprising planned delete, we'll now try to specify an additional reason for the planned deletion, covering all of the main reasons why that could happen. This commit only introduces the new detail to the plans.Changes result, though it also incidentally exposes it as part of the JSON plan result in order to keep that working without returning errors in these new cases. We'll expose this information in the human-oriented UI output in a subsequent commit.	4 years ago
Alisdair McDiarmid	78c4a8c461	json-output: Previous address for resource changes ... Configuration-driven moves are represented in the plan file by setting the resource's `PrevRunAddr` to a different value than its `Addr`. For JSON plan output, we here add a new field to resource changes, `previous_address`, which is present and non-empty only if the resource is planned to be moved. Like the CLI UI, refresh-only plans will include move-only changes in the resource drift JSON output. In normal plan mode, these are elided to avoid redundancy with planned changes.	4 years ago
Alisdair McDiarmid	f0cf4235f9	cli: Refactor resource drift rendering	4 years ago
Alisdair McDiarmid	cd29c3e5fd	Revert "json-output: Release format version 1.0"	5 years ago
Laura Pacilio	a819d7db3a	Merge pull request #29502 from hashicorp/alisdair/json-format-version ... json-output: Release format version 1.0	5 years ago
Alisdair McDiarmid	b60f201eed	json-output: Release format version 1.0	5 years ago
James Bardin	863963e7a6	de-linting	5 years ago
Alisdair McDiarmid	3326ab7dae	json-output: Omit unchanged resource_drift entries ... Previously, if any resources were found to have drifted, the JSON plan output would include a drift entry for every resource in state. This commit aligns the JSON plan output with the CLI UI, and only includes those resources where the old value does not equal the new value---i.e. drift has been detected. Also fixes a bug where the "address" field was missing from the drift output, and adds some test coverage.	5 years ago
Kristin Laemmert	ac03d35997	jsonplan and jsonstate: include sensitive_values in state representations (#28889) ... * jsonplan and jsonstate: include sensitive_values in state representations A sensitive_values field has been added to the resource in state and planned values which is a map of all sensitive attributes with the values set to true. It wasn't entirely clear to me if the values in state would suffice, or if we also need to consult the schema - I believe that this is sufficient for state files written since v0.15, and if that's incorrect or insufficient, I'll add in the provider schema check as well. I also updated the documentation, and, since we've considered this before, bumped the FormatVersions for both jsonstate and jsonplan.	5 years ago
James Bardin	09c33fa449	account for noop deposed instances in json plan ... When rendering a json plan, we need to account for deposed instances that have become a noop rather than a destroy.	5 years ago
Martin Atkins	36d0a50427	Move terraform/ to internal/terraform/ ... This is part of a general effort to move all of Terraform's non-library package surface under internal in order to reinforce that these are for internal use within Terraform only. If you were previously importing packages under this prefix into an external codebase, you could pin to an earlier release tag as an interim solution until you've make a plan to achieve the same functionality some other way.	5 years ago
Martin Atkins	f40800b3a4	Move states/ to internal/states/ ... This is part of a general effort to move all of Terraform's non-library package surface under internal in order to reinforce that these are for internal use within Terraform only. If you were previously importing packages under this prefix into an external codebase, you could pin to an earlier release tag as an interim solution until you've make a plan to achieve the same functionality some other way.	5 years ago
Martin Atkins	034e944070	Move plans/ to internal/plans/ ... This is part of a general effort to move all of Terraform's non-library package surface under internal in order to reinforce that these are for internal use within Terraform only. If you were previously importing packages under this prefix into an external codebase, you could pin to an earlier release tag as an interim solution until you've make a plan to achieve the same functionality some other way.	5 years ago
Martin Atkins	31349a9c3a	Move configs/ to internal/configs/ ... This is part of a general effort to move all of Terraform's non-library package surface under internal in order to reinforce that these are for internal use within Terraform only. If you were previously importing packages under this prefix into an external codebase, you could pin to an earlier release tag as an interim solution until you've make a plan to achieve the same functionality some other way.	5 years ago
Martin Atkins	ffe056bacb	Move command/ to internal/command/ ... This is part of a general effort to move all of Terraform's non-library package surface under internal in order to reinforce that these are for internal use within Terraform only. If you were previously importing packages under this prefix into an external codebase, you could pin to an earlier release tag as an interim solution until you've make a plan to achieve the same functionality some other way.	5 years ago