75 Commits (sarah/init-cmd-testing-comments)

Author	SHA1	Message	Date
Liam Cervante	b6be635748	stacks migrate: allow resource mapping to include nested resources (#37060) ... * stacks migrate: allow resource mapping to include nested resources * make protobuf	1 year ago
James Bardin	807e084130	final renaming for function results	1 year ago
James Bardin	a6ec17cb77	rename function results table ... Remove "provider" from the function results fields since it's not going to be exclusively used for providers any longer.	1 year ago
James Bardin	d016070564	move function results hashing to lang ... We need to abstract the function results verification to use internally too, so start by moving it out of the providers code.	1 year ago
Liam Cervante	4eaa9d7fa0	stacks: removing embedded stacks should ignore stacks not in state (#36901)	1 year ago
Liam Cervante	063757ff45	stacks: refactor plan, state, and removed tracking with tree structures for efficient lookups (#36850)	1 year ago
Liam Cervante	a06f82746a	stacks: update removed blocks to allow targeting of embedded stacks (#36814) ... * stacks: update removed blocks to allow targeting of embedded stacks * copywrite headers	1 year ago
Liam Cervante	2b5101f734	stacks: include existing components when deferring nested stacks (#36788) ... * stacks: include existing components when deferring nested stacks * improve comments	1 year ago
Daniel Banck	b2b42c0fb4	Store resource identities in state (TF-23255) (#36464) ... * Persist resource identity in Terraform state * make syncdeps * Move identity schema merging closer to the protocol * mock GetResourceIdentitySchemas * Fix identity refresh tests * Add more tests * Change grcpwrap upgrade identity * Review feedback * Remove unnecessary version conversion * Check if GetResourceIdentitySchemas RPC call is implemented * Update function signature docs * Adapt protocol changes * Check unimplemented error for identities in GetSchema	1 year ago
Liam Cervante	a384d2daa7	stacks: fix deferred data sources and unknown component applies (#35876)	2 years ago
Liam Cervante	bfa320c7b0	stacks: represent ephemeral inputs and outputs as null (#35824) ... * stacks: represent ephemeral inputs and outputs as null * quick fix: remove unnecessary check	2 years ago
James Bardin	384f2d4fab	update collections to use for-range method	2 years ago
Liam Cervante	d93e18d155	stacks: separate refresh and destroy stages during destroy plans (#35744) ... * add tests that highlight known issues in the destroy mechanism * separate refresh during destroy plans * use the refresh outputs during destroy plans * copywrite headers	2 years ago
Liam Cervante	b38fd17cf9	stacks: emit removal notices for empty components (#35738) ... * stacks: emit removal notices for empty components * fix tests and checks	2 years ago
Liam Cervante	e78294d42b	stacks: stop encoding values into msgpack twice (#35734)	2 years ago
Liam Cervante	598648b66f	stacks: handle input and output state during delete plans (#35726)	2 years ago
Liam Cervante	d142486a40	stacks: expand plan and apply outputs for inputs (#35724)	2 years ago
Liam Cervante	73e3f8096b	stacks: complete stack output implementation for plan and apply (#35723)	2 years ago
Liam Cervante	6b2cc9c379	stacks: include existing instances for unknown removed and component blocks (#35691)	2 years ago
Liam Cervante	b6ac98122b	stacks: refactor shared functionality in prep for removed blocks (#35670)	2 years ago
Liam Cervante	50233d7a78	stacks: support sensitive input values in planned changes (#35640) ... * stacks: support sensitive input values in planned changes * stacks: Fix panic when applying sensitive inputs The stacks-specific check that a given plan's input variable values remain unchanged at apply time would previously panic when given a marked value. Marks are not important for this check, so we simply remove them to ensure that it passes. This seems in line with how AssertObjectCompatible works, which I'm taking as precedent. The new test added in this commit panics without the corresponding code change. --------- Co-authored-by: Alisdair McDiarmid <alisdair@users.noreply.github.com>	2 years ago
James Bardin	2ffbd0eb4e	remove old refactoring comments	2 years ago
James Bardin	bc750192e4	POC for always using decoded changes in core ... We should not need to encode and decode change values within core, since the encoded version is only technically needed for serialization. This pattern stems from the conversion to current changes system, but back then we did not have easy access to the correct schemas at the time to encode and decode the entire set of changes. Moving the core handling of changes to only use the decoded values will drastically improve evaluation efficiency, removing a round trip through encoded values for every resource reference.	2 years ago
Liam Cervante	f1ae07b1af	stacks: add support for provider functions in .tfstack.hcl files (#35610) ... * stacks: fix bug preventing cross-provider move refactorings * also make provider functions work * stacks: add support for provider functions in .tfstack.hcl files	2 years ago
Liam Cervante	915b174da3	stacks: split the terraform1 RPC package into per-service packages (#35513) ... * stacks: split the terraform1 RPC package into per-service packages * pull latest changes	2 years ago
Liam Cervante	471bddf3ba	json plan: update resource index and import ids to support unknown values (#35518)	2 years ago
Daniel Schmidt	429ccbb88e	Merge pull request #35512 from hashicorp/TF-18555 ... stacks: add fields required to display plan	2 years ago
Daniel Schmidt	4faf0f25dd	guard against nil resource key	2 years ago
Daniel Schmidt	c693c39677	use resource_name instead of name ... Co-authored-by: Liam Cervante <liam.cervante@hashicorp.com>	2 years ago
Liam Cervante	0c67edd598	stacks: ensure input values for components don't change between plan and apply (#35489) ... * stacks: ensure input values for components don't change between plan and apply * skip ephemeral values	2 years ago
Daniel Schmidt	df85a22343	stacks: add fields required to display plan	2 years ago
Liam Cervante	87bbc47737	stacks: include plan mode in stacks plan format (#35405) ... * stacks: include plan mode in stacks plan format * fix tests * fix missed files	2 years ago
Martin Atkins	ec2510fc3d	stacks: Track raw stack as separate messages in raw plan ... Previously we had the entire raw prior state serialized as part of the "plan header" message in the raw plan, which meant that the maximum state size was constrained by the maximum allowed gRPC message size. Instead now we'll use a separate raw plan element for each raw prior state element, so that we're limited only in the size of individual items rather than size of the state as a whole. This deals with the last remaining (non-deprecated) case where our RPC protocol tries to pack an entire raw state or plan into a single protobuf message, and so we're now standardized on using a streaming approach in all cases.	2 years ago
Alisdair McDiarmid	d828776757	stacks+rpcapi: Load prior state and plan separately ... Previously we expected clients to provide an inline raw prior state to PlanStackChanges and an inline raw plan to ApplyStackChanges, which was a simpler design but meant that we might end up generating a state or plan that's too large to be submitted in a single gRPC request, which would then be difficult to resolve. Instead we'll offer separate RPC functions for loading raw state and plan using a gRPC streaming approach, which better mirrors the streaming approach we use to _emit_ these artifacts. Although we don't actually need this benefit right now, this makes it possible in principle for a client that's running PlanStackChanges to feed back the raw planned actions concurrently into OpenPlan and thus avoid buffering the whole plan on the client side at all. This required resolving the pre-existing FIXME about the inconsistency where stackeval wants a raw plan for apply but expects the caller to have dealt with loading the prior state for planning. Here it's resolved in the direction of the caller (rpcapi) always being responsible for loading both artifacts, because that means we can continue supporting the old inline approach for a while without that complexity having to infect the lower layers. Ideally we should remove the legacy approach before this API becomes constrained by compatibility promises, but I've preserved the old API for now to give us some flexibility in when we update the existing clients of this API to use the new approach. Co-authored-by: Martin Atkins <mart@degeneration.co.uk>	2 years ago
Liam Cervante	46a6d3f637	stacks: include moved and imported metadata in change descriptions (#35439)	2 years ago
Daniel Schmidt	b62a67394b	stacks: save plantimestamp to plan	2 years ago
Liam Cervante	0f0414d484	Fix partial address representation in plan for deferred actions (#34966) ... * Fix partial address representation in plan * reflect new type in stacks	2 years ago
Martin Atkins	e74896bd24	stackruntime: Handle apply-time-specified input variables ... When the topmost stack configuration declares an ephemeral input variable, its values must be provided separately for each of the plan and apply phases. Therefore here we extend the API to allow specifying input variable values during the apply phase, and add rules to check whether all of the apply-time-required input variables have been specified and whether any non-ephemeral variables are either unspecified or re-specified with equal values during the apply phase. This also extends the FindStackConfigurationComponents response to include more metadata about the input variables and output values so that a caller can know which ones are ephemeral. The name of that RPC function had already become a little too specific with the inclusion of embedded stack information and is even moreso now; we might choose to rename it to something more generic like "AnalyzeStackConfiguration" in future, but that'd be a breaking change and therefore requires more coordination.	2 years ago
Liam Cervante	9b95898f63	stacks: emit events for deferred actions (#35325) ... * stacks: emit events for deferred actions * deferral allowed is always on * Update internal/rpcapi/stacks.go Co-authored-by: Daniel Schmidt <danielmschmidt92@gmail.com> --------- Co-authored-by: Daniel Schmidt <danielmschmidt92@gmail.com>	2 years ago
Daniel Schmidt	59ead5356f	stacks: update RPC APIs with deferral information (#35125) ... * stacks: add deferred resource instance planned change to protobuf * stacks: add deferred resource instance to stack plan sequence * stacks: add planned change for deferred actions * stacks: refactor planned change resource instance planned moving the components out of the main function definition so that we can reuse the implementation for deferred resource instances which wraps the message used for PlannedChangeResourceInstancePlanned * stacks: track deferred changes in stackplan * add simple tests * fix tests * address comments --------- Co-authored-by: Liam Cervante <liam.cervante@hashicorp.com>	2 years ago
Martin Atkins	30e2fd6525	Handle marks a little more consistently ... In the very first implementation of "sensitive values" we were unfortunately not disciplined about separating the idea of "marked value" from the idea of "sensitive value" (where the latter is a subset of the former). The first implementation just assumed that any marking whatsoever meant "sensitive". We later improved that by adding the marks package and the marks.Sensitive value to standardize on the representation of "sensitive value" as being a value marked with _that specific mark_. However, we did not perform a thorough review of all of the mark-handling codepaths to make sure they all agreed on that definition. In particular, the state and plan models were both designed as if they supported arbitrary marks but then in practice marks other than marks.Sensitive would be handled in various inconsistent ways: dropped entirely, or interpreted as if marks.Sensitive, and possibly do so inconsistently when a value is used only in memory vs. round-tripped through a wire/file format. The goal of this commit is to resolve those oddities so that there are now two possible situations: - General mark handling: some codepaths genuinely handle marks generically, by transporting them from input value to output value in a way consistent with how cty itself deals with marks. This is the ideal case because it means we can add new marks in future and assume these codepaths will handle them correctly without any further modifications. - Sensitive-only mark preservation: the codepaths that interact with our wire protocols and file formats typically have only specialized support for sensitive values in particular, and lack support for any other marks. Those codepaths are now subject to a new rule where they must return an error if asked to deal with any other mark, so that if we introduce new marks in future we'll be forced either to define how we'll avoid those markings reaching the file/wire formats or extend the file/wire formats to support the new marks. Some new helper functions in package marks are intended to standardize how we deal with the "sensitive values only" situations, in the hope that this will make it easier to keep things consistent as the codebase evolves in future. In practice the modules runtime only ever uses marks.Sensitive as a mark today, so all of these checks are effectively covering "should never happen" cases. The only other mark Terraform uses is an implementation detail of "terraform console" and does not interact with any of the codepaths that only support sensitive values in particular.	2 years ago
Daniel Schmidt	949a0e5585	Merge pull request #34755 from hashicorp/TF-10993 ... stacks: expose replace paths in grpc api	2 years ago
Daniel Schmidt	2fa4cc7b53	stacks: use existing conversion for attribute paths	2 years ago
CJ Horton	5a750561e6	round-trip check results through stack plans ... Components expect that check results will be round-tripped through the plan in order to update unknown check results during the apply. We hadn't wired this up for stack plans, resulting in panics when the apply tries to update a check status that doesn't exist.	2 years ago
Daniel Schmidt	57dd94fcde	stacks: expose replace paths in grpc api ... Co-authored-by: Katy Moe <katy@katy.moe>	2 years ago
Martin Atkins	a45d4467f1	stackeval: Don't fail when data resource is removed ... Due to an oversight in our handling of resource instance objects that are neither in configuration nor plan -- which is true for data resources that have since been removed from the configuration -- we were generating plan change objects that were lacking a provider configuration address, which made them syntactically invalid and thus not reloadable using the raw plan parser. This is a bit of a strange situation since we don't technically _need_ a provider configuration address for these; all we're going to do is just unceremoniously delete them from the state during apply anyway. However, we always have the provider configuration address available anyway, so adding this in here is overall simpler than changing the parser, the models it populates, and all of the downstream users of those models to treat this field as optional. This commit is more test case than it is fix, since the fix was relatively straightforward once I had a test case to reproduce the problem it's fixing.	2 years ago
Liam Cervante	07f6621091	stacks: include resources in state when calculating required providers (#34645) ... * stacks: include resources in state when calculating required providers * also support apply time * add copywrite headers	2 years ago
Martin Atkins	884e1fb2a4	terraform: Plans can be "complete" and "applyable" ... These ideas are both already implied by some logic elsewhere in the system, but until now we didn't have the decision logic centralized in a single place that could therefore evolve over time without necessarily always updating every caller together. We'll now have the modules runtime produce its own boolean ruling about each characteristic, which callers can rely on for the mechanical decision-making of whether to offer the user an "approve" prompt, and whether to remind the user after apply that it was an incomplete plan that will probably therefore need at least one more plan/apply round to converge. The "Applyable" flag directly replaces the previous method Plan.CanApply, with equivalent logic. Making this a field instead of a method means that we can freeze it as part of a saved plan, rather than recalculating it when we reload the plan, and we can export the field value in our export formats like JSON while ensuring it'll always be consistent with what Terraform is using internally. Callers can (and should) still use other context in the plan to return more tailored messages for specific situations they already know about that might be useful to users, but with these flags as a baseline callers can now just fall back to a generic presentation when encountering a situation they don't yet understand, rather than making the wrong decision and causing something strange to happen. That is: a lack of awareness of a new rule will now cause just a generic message in the UI, rather than incorrect behavior. This commit mostly just deals with populating the flags, and then all of the direct consequences of that on our various tests. Further changes to actually make use of these flags elsewhere in the system will follow in later commits, both in this repository and in other repositories.	2 years ago
Alisdair McDiarmid	1d3f863f2b	stackruntime: Support sensitive component inputs ... Components can emit sensitive values as outputs, which can be consumed as inputs to other components. This commit ensures that such values are correctly processed in order to pass their sensitivity to the modules runtime.	2 years ago
Martin Atkins	a4d61733f6	stackplan: Include extra info in raw component instance plan ... We'll now track the planned action, the component addresses we depend on, and the planned output values.	2 years ago