97 Commits (7bee77bdd3dc67de0973e1c19bd5a1685dc805c0)

Author	SHA1	Message	Date
Martin Atkins	a7807dac16	main: Optionally configure an OpenTelemetry OTLP exporter ... Terraform CLI is sometimes used as part of a larger distributed system, in which case it would be helpful to be able to gather telemetry from it as part of the larger request it's being run in response to. We'll now support optionally enabling an OTLP exporter by setting the environment variable OTEL_TRACES_EXPORTER=otlp (a standard OpenTelemetry convention). As of this commit there isn't actually anything emitting traces to the specified collector, but we'll gradually add tracing instrumentation to parts of Terraform CLI and Core in later commits.	3 years ago
Liam Cervante	c9bc7e8479	Add input validation into the 'checks' outputs and tracking (#33481)	3 years ago
Liam Cervante	3732bffe13	[Testing Framework] Adds TestContext for evaluating test assertions (#33326) ... * Add test structure to views package for rendering test output * Add test file HCL configuration and parser functionality * Adds a TestContext structure for evaluating assertions against the state and plan	3 years ago
Martin Atkins	d49e991c3c	plans/objchange: Support nested unknowns in our unrefinedValue shim ... Several parts of the objchange logic incorrectly use cty.Value.RawEquals for value comparison, instead of more appropriate comparison methods like cty.Value.Equals or c.Value.Range().Includes. That makes them incorrectly consider two unknown values with the same type but different refinements as always non-equal, rather than evaluating based on the overlap between the refinements (if any). As a short-term fix for that we previously added this unrefinedValue shim that just strips away the refinements for comparison, thus allowing callers to continue using RawEquals as long as they've already taken care of all of the other things that can make that go wrong, such as value marks. Unfortunately the shim was too simplistic and only supported direct unknown values. Unknown values with refinements can also appear nested inside known container values such as collections, so the shim needs to recursively un-refine the entire data structure in that case. This is still intended only as a temporary fix until we have time to revisit all of the callers and make them use cty's own logic for comparison. Using cty's own logic will make the results more precise, because e.g. it can notice if two unknown strings have different known prefixes and therefore cannot possibly be equal despite not being fully known. For now this shim will accept any pair of unknown values of the same type as equal, regardless of refinement.	3 years ago
James Bardin	f6c536e436	validate planned set sizes using refinements	3 years ago
James Bardin	3c8a163583	set length is unknown with partially known elems ... If a set contains partially known values the length is unknown which causes assertPlannedObjectValid to fail valid plans. Revert to the old method if using LengthInt for the set lengths, which returns the maximum number of possible elements, with a guard for entirely unknown set values.	3 years ago
Martin Atkins	4c439b099f	plans/objchange: Don't consider refinements when validating plans ... Providers that existed prior to refinements (all of them, at the time of writing) cannot preserve refinements sent in unknown values in the configuration, and even if one day providers _are_ aware of refinements there we might add new ones that existing providers don't know how to handle. For that reason we'll absolve providers of the responsibility of preserving refinements from config into plan by fixing some cases where we were incorrectly using RawEquals to compare values; that function isn't appropriate for comparing values that might be unknown. However, to avoid a disruptive change right now this initial fix just strips off the refinements before comparing. Ideally this should be using Value.Equals and handling unknown values more explicitly, but we'll save that for a possible later improvement. This does not include a similar exception for validating whether a final value conforms to a plan because the plan value and the final value are both produced by the same provider and so providers ought to be able to be consistent with their _own_ treatment of refinements, if any. Configuration is special because Terraform itself generates that, and so it can potentially contain refinements that a particular provider has no awareness of.	3 years ago
Martin Atkins	dfe5e1ddc4	plans/objchange: Providers must honor their unknown value refinements ... If the original value was unknown but its range was refined then the provider must return a value that is within the refined range, because otherwise downstream planning decisions could be invalidated. This relies on cty's definition of whether a value is in a refined range, which has pretty good coverage for the "false" case and so should give a pretty good signal, but it'll probably improve over time and so providers must not rely on any loopholes in the current implementation and must keep their promises even if Terraform can't currently check them.	3 years ago
CJ Horton	258bdbe89f	Merge pull request #33238 from hashicorp/radditude/import-plan-plumbing ... plannable import: correct plumbing when using a plan output file	3 years ago
James Bardin	2f308cf948	Merge pull request #32962 from hashicorp/jbardin/validate-unknown-coll-attrs ... validate unknown nested attribute collections	3 years ago
CJ Horton	40ff381887	plumb import changes to and from binary plan	3 years ago
Liam Cervante	79f7f59155	Plannable import: Generate config for imported resources during the plan. (#33153) ... * command: keep our promises * remove some nil config checks Remove some of the safety checks that ensure plan nodes have config attached at the appropriate time. * add GeneratedConfig to plan changes objects Add a new GeneratedConfig field alongside Importing in plan changes. * add config generation package The genconfig package implements HCL config generation from provider state values. Thanks to @mildwonkey whose implementation of terraform add is the basis for this package. * generate config during plan If a resource is being imported and does not already have config, attempt to generate that config during planning. The config is generated from the state as an HCL string, and then parsed back into an hcl.Body to attach to the plan graph node. The generated config string is attached to the change emitted by the plan. * complete config generation prototype, and add tests --------- Co-authored-by: Katy Moe <katy@katy.moe>	3 years ago
Alisdair McDiarmid	b5658a46a2	Rebuild protobuf	3 years ago
hashicorp-copywrite[bot]	325d18262e	[COMPLIANCE] Add Copyright and License Headers	3 years ago
Liam Cervante	4210d905c0	[plannable import] embed the resource id within the changes (#33134) ... * [plannable import] embed the resource id within the changes * make pointers and update docs	3 years ago
kmoe	28643516b2	Plannable import 3: Make import plannable (#33085) ... During a plan, Terraform now checks for the presence of import blocks. For each resource in config, if an import block is present with a matching address, planning that node will now trigger an ImportResourceState and ReadResource. The resulting state is treated as the node's "refresh state", and planning proceeds as normal from there. The walkImport operation is now only used for the legacy "terraform import" CLI command. This is the only case under which the plan should produce graphNodeImportStates.	3 years ago
kmoe	531efd303b	add types for plannable import (#33080)	3 years ago
Liam Cervante	aae6990d38	Add native `plantimestamp` function to Terraform (#32980) ... * add plantimestamp function * documentation * add missing links * fix typo	3 years ago
James Bardin	8ab7af8c5f	validate unknown nested attribute collections ... It is not valid for a provider to return an unknown value for a configured nested collection, but we need to check for unknowns before comparing the number of values in the collection.	3 years ago
Liam Cervante	978263efe9	Checks: Introduce check blocks into the terraform node and transform graph (#32735) ... * Add support for scoped resources * refactor existing checks addrs and add check block addr * Add configuration for check blocks * introduce check blocks into the terraform node and transform graph * address comments * address comments * don't execute checks during destroy operations * don't even include check nodes for destroy operations	3 years ago
kmoe	76b34e891c	remove unused actionforchange (#32759)	3 years ago
James Bardin	b6906f3572	Merge pull request #32563 from hashicorp/jbardin/optional-computed-comparison-next ... a new method of `ProposedNew` set comparison	3 years ago
James Bardin	799f4a76ab	Merge pull request #32554 from hashicorp/jbardin/optional-computed-comparison ... refactor ProposedNew	3 years ago
James Bardin	33c1ffa7c2	Merge pull request #32553 from hashicorp/jbardin/nesting-map-objchange ... Fix crashes with NestingMap values	3 years ago
James Bardin	d1d55653e4	update comment for fixed test	3 years ago
James Bardin	60d6e52021	a new method of ProposedNew set comparison ... The existing set comparison method uses the prior elements with the computed portions nulled out to find candidates to match the configuration. This has the shortcoming of always removing optional+computed attributes, because we have not yet found the configuration to know if attribute was set or not. Rather than having to take the most pessimistic value before comparison to precompute the nulled values, we can compare each candidate directly, walking the values in tandem. Each prior value is compared against the config and checked to see if it could have been derived from that configuration value, which allows us to treat optional+computed as optional if there is config and computed if there is not. This removes the ambiguity from having optional+computed attributes within sets, giving us consistent plans when all values are known. Unknown values of course are still undecidable, as are edge cases were providers refresh with altered values or retained changed prior values plan that were deemed not functionally significant.	3 years ago
James Bardin	9459f0b214	refactor ProposedNew ... Unify the ProposedNew paths for Blocks and Objects. Break out the individual case blocks into functions, then use a common interface to dispatch the object creation to the correct function based on schema type. This cuts the code in half, and prevents the block and object behavior from diverging.	3 years ago
James Bardin	375c2da3e3	update NestingMap logic ... Simplify the logic in the NestingMap cases. Prevent uninitialized cty.NilVal from appearing in block case.	3 years ago
James Bardin	fcbfc365e6	fix panics when handling null values in maps ... NestingMap structures are not well tested, and we panic in many situations when null crops up. Fix the first test cases and start refactoring best we can. This probably won't go so far as making all the objchange functions generic over Block and Object, but we can simplify a lot and verify parity in implementations for now.	3 years ago
James Bardin	8e917e5513	check for non-computed within optional+computed ... We can check if an object in state must have at least partially come from configuration, by seeing if the prior value has any non-null attributes which are not computed in the schema. This is used when the configuration contains a null optional+computed value, and we want to know if we should plan to send the null value or the prior state.	3 years ago
James Bardin	e16b848d00	expand test case	3 years ago
James Bardin	7ca9abe463	optional+computed ... Simplify the proposedNewAttributes cases, and add another test for coverage.	3 years ago
James Bardin	470ed22730	failing tests for nested sets in objchange ... Add a number of test cases which fail without the prior changes.	3 years ago
James Bardin	048ee3432c	better handling of nested sets in objchange ... Combine and simplify the set comparison functions for NestingSet blocks and attribute types. The set handling for structural attributes was not recursing into nested values. Once a simplified method for comparing set elements was devised for nested types, it turns out the same method could be applied to nested set blocks as well.	3 years ago
James Bardin	ac4f5fe2f9	reformat	3 years ago
James Bardin	986127e619	remove unused isConfig arg	3 years ago
James Bardin	e661e91bff	failing test for nested set types	3 years ago
James Bardin	93f739e927	optional+computed can contain nested objects ... When structural attributes were added, optional+computed were not correctly handled when containing nested values which could themselves be computed. This would cause terraform to ignore previously computed values from state when generating the proposed plan. The special case for optional+computed was incorrect, but isn't needed in the context of planning new values anyway. Attributes are either computed, or not computed. When optional+computed is set and there is no configuration, the attribute is treated as computed. It is up to the provider to determine how and when to deal with any changes to that computed value.	3 years ago
James Bardin	0a73e216e5	Merge pull request #32463 from hashicorp/jbardin/nesting-single-null ... NestingSingle blocks can be null	3 years ago
Liam Cervante	38afb41787	Structured plan renderer: Introduce support for multiline and JSON strings (#32488) ... * pause implementation * change -> diff, value -> change * add support for json and multiline strings to the primitive renderer * goimports * remove unused function * go fmt * address comments	3 years ago
James Bardin	d5d6d61c4c	NestingSingle blocks can be null ... NestingSingle blocks removed from from the config were causing a plan to error out with "... planned for existence but config wants absence". Terraform core was proposing an incorrect value in this case, taking the prior instead as a fallback because a null value was not expected. Unlike other collection nesting modes, a NestingSingle block not present in the configuration is a null value, and should be allowed when planning a new value rather than building an empty object or falling back to the prior value.	3 years ago
James Bardin	721df0e68d	Merge pull request #32395 from hashicorp/jbardin/plan-output-on-error ... Always show and store planned actions and checks even when planning fails	3 years ago
James Bardin	ea193d5ce6	don't panic with a null list block value in config ... Using ignore_changes with a list block, where the provider returned an invalid null value for that block, can result in a panic when validating the plan. Future releases may prevent providers from storing a null block in state, however we can avoid the panic for now. Only the NestingList case needs to be handled, because legacy providers only have list and set blocks, and the set case does not use the config value.	3 years ago
Martin Atkins	6d9ddbacec	core: Mark errored plans as errored ... In any situation where we return a plan object along with some errors we'll also explicitly annotate the plan object as being errored so that we can catch if someone accidentally tries to apply that incomplete plan. At the moment this situation is impossible to reach but in a later commit we'll make it possible to save errored plans to disk for further inspection, at which point it'll become important to not allow applying them.	3 years ago
Sarah French	6fd3a8cdf4	go get cloud.google.com/go/storage@v1.28.0 (#32203) ... * go get cloud.google.com/go/storage@v1.28.0 * go mod tidy * Run `make generate` & `make protobuf` using go1.19.3	3 years ago
James Bardin	a44c859c86	update UIMode comment	4 years ago
James Bardin	47b6386348	remove IsFullDestroy workaround ... IsFullDestroy was a workaround during apply to detect when the change set was created by a destroy plan. This no longer works correctly, and we need to fall back to the UIMode set in the plan.	4 years ago
kmoe	dec48a8510	plans: indicate when resource deleted due to move (#31695) ... Add a new ChangeReason, ReasonDeleteBecauseNoMoveTarget, to provide better information in cases where a planned deletion is due to moving a resource to a target not in configuration. Consider a case in which a resource instance exists in state at address A, and the user adds a moved block to move A to address B. Whether by the user's intention or not, address B does not exist in configuration. Terraform combines the move from A to B, and the lack of configuration for B, into a single delete action for the (previously nonexistent) entity B. Prior to this commit, the Terraform plan will report that resource B will be destroyed because it does not exist in configuration, without explicitly connecting this to the move. This commit provides the user an additional clue as to what has happened, in a case in which Terraform has elided a user's action and inaction into one potentially destructive change.	4 years ago
Martin Atkins	0e4e9f7706	addrs: Be explicit about checkable object address kinds ... Previously we were attempting to infer the checkable object address kind of a given address by whether it included "output" in the position where a resource type name would otherwise go. That was already potentially risky because we've historically not prevented a resource type named "output", and it's also a forward-compatibility hazard in case we introduce additional object kinds with entirely-new addressing schemes in future. Given that, we'll instead always be explicit about what kind of address we're storing in a wire or file format, so that we can make sure to always use the intended parser when reading an address back into memory, or return an error if we encounter a kind we're not familiar with.	4 years ago
Martin Atkins	9e4861adbb	states: Two-level representation of check results ... A significant goal of the design changes around checks in earlier commits (with the introduction of package "checks") was to allow us to differentiate between a configuration object that we didn't expand at all due to an upstream error, which has _unknown_ check status, and a configuration object that expanded to zero dynamic objects, which therefore has a _passing_ check status. However, our initial lowering of checks.State into states.CheckResults stayed with the older model of just recording each leaf check in isolation, without any tracking of the containers. This commit therefore lightly reworks our representation of check results in the state and plan with two main goals: - The results are grouped by the static configuration object they came from, and we capture an aggregate status for each of those so that we can differentiate an unknown aggregate result from a passing aggregate result which has zero dynamic associated objects. - The granularity of results is whole checkable objects rather than individual checks, because checkable objects have durable addresses between runs, but individual checks for an object are more of a syntactic convenience to make it easier for module authors to declare many independent conditions that each have their own error messages. Since v1.2 exposed some details of our checks model into the JSON plan output there are some unanswered questions here about how we can shift to reporting in the two-level heirarchy described above. For now I've preserved structural compatibility but not semantic compatibility: any parser that was written against that format should still function but will now see fewer results. We'll revisit this in a later commit and consider other structures and what to do about our compatibility constraint on the v1.2 structure. Otherwise though, this is an internal-only change which preserves all of the existing main behaviors of conditions as before, and just gets us ready to build user-facing features in terms of this new structure.	4 years ago