15 Commits (stacks-variable-validations)

Author	SHA1	Message	Date
Liam Cervante	bfa320c7b0	stacks: represent ephemeral inputs and outputs as null (#35824) ... * stacks: represent ephemeral inputs and outputs as null * quick fix: remove unnecessary check	1 year ago
Liam Cervante	d142486a40	stacks: expand plan and apply outputs for inputs (#35724)	1 year ago
Liam Cervante	73e3f8096b	stacks: complete stack output implementation for plan and apply (#35723)	1 year ago
Liam Cervante	f8fe397d88	stacks: add removed block functionality (#35671) ... * stacks: add removed block functionality * fix compile errors	1 year ago
Liam Cervante	7163c4b6d5	stacks: fix destroy ordering of stacks destroy plans (#35665) ... * stacks: remove unneeded required components functions * stacks: refactor the mock provider for realism * deferred actions: fix missing features in deletions * stacks: fix destroy ordering of stacks destroy plans * fix missing import	1 year ago
Alisdair McDiarmid	d828776757	stacks+rpcapi: Load prior state and plan separately ... Previously we expected clients to provide an inline raw prior state to PlanStackChanges and an inline raw plan to ApplyStackChanges, which was a simpler design but meant that we might end up generating a state or plan that's too large to be submitted in a single gRPC request, which would then be difficult to resolve. Instead we'll offer separate RPC functions for loading raw state and plan using a gRPC streaming approach, which better mirrors the streaming approach we use to _emit_ these artifacts. Although we don't actually need this benefit right now, this makes it possible in principle for a client that's running PlanStackChanges to feed back the raw planned actions concurrently into OpenPlan and thus avoid buffering the whole plan on the client side at all. This required resolving the pre-existing FIXME about the inconsistency where stackeval wants a raw plan for apply but expects the caller to have dealt with loading the prior state for planning. Here it's resolved in the direction of the caller (rpcapi) always being responsible for loading both artifacts, because that means we can continue supporting the old inline approach for a while without that complexity having to infect the lower layers. Ideally we should remove the legacy approach before this API becomes constrained by compatibility promises, but I've preserved the old API for now to give us some flexibility in when we update the existing clients of this API to use the new approach. Co-authored-by: Martin Atkins <mart@degeneration.co.uk>	2 years ago
Martin Atkins	30e2fd6525	Handle marks a little more consistently ... In the very first implementation of "sensitive values" we were unfortunately not disciplined about separating the idea of "marked value" from the idea of "sensitive value" (where the latter is a subset of the former). The first implementation just assumed that any marking whatsoever meant "sensitive". We later improved that by adding the marks package and the marks.Sensitive value to standardize on the representation of "sensitive value" as being a value marked with _that specific mark_. However, we did not perform a thorough review of all of the mark-handling codepaths to make sure they all agreed on that definition. In particular, the state and plan models were both designed as if they supported arbitrary marks but then in practice marks other than marks.Sensitive would be handled in various inconsistent ways: dropped entirely, or interpreted as if marks.Sensitive, and possibly do so inconsistently when a value is used only in memory vs. round-tripped through a wire/file format. The goal of this commit is to resolve those oddities so that there are now two possible situations: - General mark handling: some codepaths genuinely handle marks generically, by transporting them from input value to output value in a way consistent with how cty itself deals with marks. This is the ideal case because it means we can add new marks in future and assume these codepaths will handle them correctly without any further modifications. - Sensitive-only mark preservation: the codepaths that interact with our wire protocols and file formats typically have only specialized support for sensitive values in particular, and lack support for any other marks. Those codepaths are now subject to a new rule where they must return an error if asked to deal with any other mark, so that if we introduce new marks in future we'll be forced either to define how we'll avoid those markings reaching the file/wire formats or extend the file/wire formats to support the new marks. Some new helper functions in package marks are intended to standardize how we deal with the "sensitive values only" situations, in the hope that this will make it easier to keep things consistent as the codebase evolves in future. In practice the modules runtime only ever uses marks.Sensitive as a mark today, so all of these checks are effectively covering "should never happen" cases. The only other mark Terraform uses is an implementation detail of "terraform console" and does not interact with any of the codepaths that only support sensitive values in particular.	2 years ago
Martin Atkins	7d2d4dec5e	stackeval: When being destroyed, component instance result comes from plan ... Because we treat dependency edges as reversed when a component instance is being destroyed, the final result (an object representing output values) for a component instance being destroyed must not depend on anything else in the evaluation graph, or else we'd cause a promise self-reference as the downstream component tries to configure itself based on our outputs. As a special case then, for a component instance being destroyed we take the planned output values directly from the plan, relying on the fact that the plan phase sets them to the prior state output values in that case, and therefore the result for such a component is available immediately without blocking on any other expression evaluation during the apply phase. This combines with several previous commits to create a first pass at handling ordering correctly when planning and applying a full destroy. This commit also incorporates some fixes and improvements to stackeval's apply-time testing helpers, which had some quirks and bugs when first added in a recent commit. One of those problems also revealed that the raw state loader was not resilient to a buggy caller setting a state entry to nil instead of removing it altogether, and that mistake seems relatively easy to make (as I did here in the test helper) so we'll tolerate it to make it possible to recover if such a bug does end up occurring in real code too.	2 years ago
CJ Horton	89aea31bbd	stackstate: sensitive values are preserved when deserializing ... Without this, sensitive values cause a permadiff, since their sensitivity is lost on every round trip through state.	2 years ago
Martin Atkins	a47dc447c1	stackeval: Additional testing utilities ... In earlier commits we stubbed out some testing utilities as part of adding some unit tests, but we didn't end up using all of them in the initial round and it turned out that some of them were incorrect or incomplete. Now we'll improve the test utilities, this time with a focus mainly on integration-type tests, because we'll start adding those in the near future. Specifically: - utilities for rendering diagnostics, mainly in situations where they are unexpected and thus we just want to see the full information as part of the test results - helper functions for concisely asserting that there should be no diagnostics at all, or that there should be no error diagnostics - the existing helper for faking up a prior state based on hard-coded raw state messages was incorrect in that it was trying to use statekeys.Key values as map keys, and those values are not typically comparable. Instead, we should use the string representations that are designed to be used as unique map keys. - the testPlan function, and the lower-level testPlanOutput it's built with, help to encapsulate the asynchronous event-emitting behavior of Main.PlanAll into a "normal-looking" function that just blocks until it's done and then returns all of its results. This automatically round-trips the planned changes through serialization and deserialization as we would between a normal plan and apply phase, and so the calling test can make its assertions against the applyable plan object instead of the raw component parts it's made from.	2 years ago
Martin Atkins	3f569b55c5	stacks: Some code consistency cleanups in preparation for first merge	2 years ago
Martin Atkins	91b8ea3d76	stackstate: A helper for loading state during tests ... The main entry point here assumes the caller is providing serialized state objects wrapped in anypb.Any messages. That's an inconvenient representation for hand-writing in tests, so the new function LoadFromDirectProto allows skipping the deserialization steps and instead has the caller write the wanted values as if they had already been parsed/unmarshaled.	2 years ago
Martin Atkins	8fd29e9439	stacks: Preserve prior state from plan to apply ... We need to retain the prior state (in a form that Terraform Core's modules runtime can accept) between the plan and apply phases because Terraform uses it to verify that the provider's "final plan" is consistent with the initial plan. We previously tried to do this by reusing the "old" value from the planned change, but that's not really possible in practice because of the historical implementation detail that Terraform wants state information in a JSON format and plan information in MessagePack. This also contains the beginnings of handling the "discarding" of unrecognized keys from the state data structures, although we'll probably need to do more in that area later since this is just the bare minimum.	2 years ago
Martin Atkins	49511fe9a8	stackstate: Fuller round-tripping of resource instance object state ... This gets us ever closer to being able to preserve resource instance objects from one run to the next. In subsequent commits we'll make use of the "LoadFromProto" option to load the prior state during the planning phase and take it into account when we're deciding what actions to propose.	2 years ago
Martin Atkins	89776cd2a6	stackstate: Initial work on decoding and the state model ... This is a sketch of the overall structure of the prior state decoder and the model type it populates. Before we can complete this we'll need to slightly rework how the apply phase emits the raw events that this is consuming, and in particular to change the raw state representation to be JSON-based to match with how Terraform Core expects to receive it once reloaded. That will follow in later commits.	2 years ago