355 Commits (8fd8a48a06af33d3a3614b4b48768bd1516632fe)

Author	SHA1	Message	Date
Liam Cervante	7163c4b6d5	stacks: fix destroy ordering of stacks destroy plans (#35665) ... * stacks: remove unneeded required components functions * stacks: refactor the mock provider for realism * deferred actions: fix missing features in deletions * stacks: fix destroy ordering of stacks destroy plans * fix missing import	2 years ago
Liam Cervante	36f11851a6	stacks: refactor the mock provider for realism (#35654)	2 years ago
Liam Cervante	11f431196d	stacks: remove unneeded required components functions (#35648)	2 years ago
Liam Cervante	cb7bae3217	stacks: remove stale state during Stack apply operations (#35639)	2 years ago
Liam Cervante	50233d7a78	stacks: support sensitive input values in planned changes (#35640) ... * stacks: support sensitive input values in planned changes * stacks: Fix panic when applying sensitive inputs The stacks-specific check that a given plan's input variable values remain unchanged at apply time would previously panic when given a marked value. Marks are not important for this check, so we simply remove them to ensure that it passes. This seems in line with how AssertObjectCompatible works, which I'm taking as precedent. The new test added in this commit panics without the corresponding code change. --------- Co-authored-by: Alisdair McDiarmid <alisdair@users.noreply.github.com>	2 years ago
Liam Cervante	40dc5c02d5	stacks: ensure providers that should not be configured cannot be (#35624) ... * stacks: ensure providers that should not be configured cannot be * address comments	2 years ago
James Bardin	20f66dffd0	fixup new test comparator	2 years ago
James Bardin	0b8fbfe323	ignore unexported state field in comparison ... and unify the options used to compare changes in tests	2 years ago
Liam Cervante	f1ae07b1af	stacks: add support for provider functions in .tfstack.hcl files (#35610) ... * stacks: fix bug preventing cross-provider move refactorings * also make provider functions work * stacks: add support for provider functions in .tfstack.hcl files	2 years ago
Liam Cervante	964c64157c	stacks: fix bug preventing unconfigured clients within Terraform (#35571) ... * stacks: fix bug preventing cross-provider move refactorings * also make provider functions work	2 years ago
Liam Cervante	7660d8d086	stacks: fix suggested command when lock file is out of date (#35616)	2 years ago
Liam Cervante	047c1af0b4	stacks: add tests for state manipulation failure cases (#35589)	2 years ago
Liam Cervante	bb73fb1d69	stacks: don't validate input variables during the apply phase (#35561)	2 years ago
Liam Cervante	5939efb798	stacks: add depends_on for embedded stacks and components (#35549)	2 years ago
Liam Cervante	38033b5c45	stacks: sort diags in test before comparison (#35550)	2 years ago
Liam Cervante	915b174da3	stacks: split the terraform1 RPC package into per-service packages (#35513) ... * stacks: split the terraform1 RPC package into per-service packages * pull latest changes	2 years ago
Liam Cervante	46393f2a92	stacks: apply automatic type conversion before comparing inputs (#35530) ... * stacks: apply automatic type conversion before comparing inputs * correct error message for missing ephemeral value during apply	2 years ago
Liam Cervante	0c67edd598	stacks: ensure input values for components don't change between plan and apply (#35489) ... * stacks: ensure input values for components don't change between plan and apply * skip ephemeral values	2 years ago
Liam Cervante	33c0ae7f09	stacks: add additional provider validation tests (#35471)	2 years ago
Liam Cervante	e704da535e	stack: stacks should be applyable even if no components are (#35498)	2 years ago
Liam Cervante	87bbc47737	stacks: include plan mode in stacks plan format (#35405) ... * stacks: include plan mode in stacks plan format * fix tests * fix missed files	2 years ago
Martin Atkins	170cca1332	stacks+promising: Better error messages for promise resolution failure ... We were previously just treating promising.ErrUnresolved as a singleton that doesn't carry any context along with it. This particular error always represents a bug in the stack runtime rather than a problem with the input, because resolving all of the promises is a key part of the contract for package promising, but it's still helpful to know a little more about what exactly has failed when debugging it. Therefore promising.ErrUnresolved is now a type rather than a value, and its carries with it a set of promise ids that were left unresolved. The stackeval package then recognizes this error type and uses its own records of what each promise was supposed to produce to hopefully return a useful error message that would narrow down what part of the system is buggy. Since this message always reflects a bug in Terraform the error message will probably just be copy-pasted into a bug report, and so this particular diagnostic type (unlike the one we use for promise self-references) only has one presentation form that we use regardless of how many promises we have to return.	2 years ago
Martin Atkins	ec2510fc3d	stacks: Track raw stack as separate messages in raw plan ... Previously we had the entire raw prior state serialized as part of the "plan header" message in the raw plan, which meant that the maximum state size was constrained by the maximum allowed gRPC message size. Instead now we'll use a separate raw plan element for each raw prior state element, so that we're limited only in the size of individual items rather than size of the state as a whole. This deals with the last remaining (non-deprecated) case where our RPC protocol tries to pack an entire raw state or plan into a single protobuf message, and so we're now standardized on using a streaming approach in all cases.	2 years ago
Alisdair McDiarmid	d828776757	stacks+rpcapi: Load prior state and plan separately ... Previously we expected clients to provide an inline raw prior state to PlanStackChanges and an inline raw plan to ApplyStackChanges, which was a simpler design but meant that we might end up generating a state or plan that's too large to be submitted in a single gRPC request, which would then be difficult to resolve. Instead we'll offer separate RPC functions for loading raw state and plan using a gRPC streaming approach, which better mirrors the streaming approach we use to _emit_ these artifacts. Although we don't actually need this benefit right now, this makes it possible in principle for a client that's running PlanStackChanges to feed back the raw planned actions concurrently into OpenPlan and thus avoid buffering the whole plan on the client side at all. This required resolving the pre-existing FIXME about the inconsistency where stackeval wants a raw plan for apply but expects the caller to have dealt with loading the prior state for planning. Here it's resolved in the direction of the caller (rpcapi) always being responsible for loading both artifacts, because that means we can continue supporting the old inline approach for a while without that complexity having to infect the lower layers. Ideally we should remove the legacy approach before this API becomes constrained by compatibility promises, but I've preserved the old API for now to give us some flexibility in when we update the existing clients of this API to use the new approach. Co-authored-by: Martin Atkins <mart@degeneration.co.uk>	2 years ago
Liam Cervante	3ecf7655ba	stacks: insert unknown values for missing outputs during applies (#35463) ... * stacks: insert unknown values for missing outputs during apply operations * fix tests	2 years ago
Daniel Schmidt	1d45d12f40	stacks: track resource deferrals in apply	2 years ago
Daniel Schmidt	591c3dea36	Merge pull request #35433 from hashicorp/TF-10919 ... stacks: add new deferred plan status	2 years ago
Liam Cervante	634155f56b	stacks: extend diagnostic message to capture error from unscheduled component (#35434)	2 years ago
Daniel Schmidt	93cf093168	stacks: add new deferred plan status	2 years ago
Liam Cervante	8e47b5907c	stacks: provide more helpful diagnostics when providers types are mismatched (#35422) ... * stacks: provide more helpful diagnostics when providers types are mismatched * remove left over implementation experiments * remove unnecessary fmt.Sprintf * address comments * fix incorrect pluralisation * fix tests	2 years ago
Daniel Schmidt	4f401d9cd3	stacks: add dependency lock to new test case	2 years ago
Daniel Schmidt	bf5037fa05	stacks: add test case	2 years ago
Daniel Schmidt	9f8ab0450c	stacks: fix move apply test	2 years ago
Daniel Schmidt	2a3c579693	stacks: use inspecting phase for provider config tests	2 years ago
Daniel Schmidt	05755422f3	stacks: pass dependency lock in tests	2 years ago
Daniel Schmidt	4433239da4	stacks: validate dependency lockfile in apply phase	2 years ago
Daniel Schmidt	0e5f7acda0	stacks: validate dependency lockfile in planning phase	2 years ago
Daniel Schmidt	fb2f3ca276	stacks: refactor to get dependency locks through main	2 years ago
Daniel Schmidt	2ed6e1facd	stacks: validate lockfile containing all providers ... If the providers are not in the lockfile fetching the schema fails. This leads to follow up diagnostics that send an unclear message. Now we validate the lockfile and if something is missing we skip further validation that relies on the provider schema being present.	2 years ago
Liam Cervante	587a7e88ff	stacks: include move, forget, import counts in apply summaries (#35391)	2 years ago
Liam Cervante	27f26bd1b5	stacks: emit specific moved notification during apply (#35387)	2 years ago
Daniel Schmidt	ae05a0d27b	stacks: use plantimestamp from opts while planning	2 years ago
Daniel Schmidt	a51938c246	stacks: move PlanTimestamp method for consistency	2 years ago
Daniel Schmidt	46999364f0	stacks: refactor for plantimestamp to come from the outside	2 years ago
Daniel Schmidt	49ba330fb5	stacks: force plan timestamp in module runtime	2 years ago
Daniel Schmidt	b62a67394b	stacks: save plantimestamp to plan	2 years ago
Daniel Schmidt	8fa922f912	stacks: implement PlanTimestamp method on ExpressionScope ... This allows us to set the plantimestamp methods value in the stacks language	2 years ago
Mutahhir Hayat	01c5ddaecc	Merge pull request #34797 from hashicorp/mutahhir/stacks/local-value ... stackruntime: Support referencing Local Values within the `tfstacks.hcl` language	2 years ago
Liam Cervante	c0499647fa	stacks: include moved and removed in API (#35360) ... * stacks: include moved and removed in API * fix race test	2 years ago
Mutahhir Hayat	148b19b0d7	return validate value instead of dynamic value	2 years ago
Daniel Schmidt	3aa8e43100	Merge pull request #35356 from hashicorp/b-stacks-apply-variables-not-passed	2 years ago
Martin Atkins	c57f709b1d	stackruntime: Pass the apply-time variable values to stackeval ... This seems to have got lost while resolving some conflicts in 88eec94896. Without this, apply-time input variable values just get ignored entirely and if any are required the plan is not applyable at all. Now we'll pass through what the caller provided so stackeval can validate and use it properly.	2 years ago
Liam Cervante	b646dff26a	stacks: apply nested default values to inputs (#35349) ... * stacks: apply nested default values to inputs * group similar tests	2 years ago
Liam Cervante	284ce63947	stacks: applyable status should be dependent on applyable components (#35350)	2 years ago
Martin Atkins	1714729f87	stackeval: terraform.applying symbol ... This symbol produces an ephemeral boolean value that's true when the stack evaluator is configured for applying and false otherwise. The primary purpose of this is to allow specifying a more privileged auth role in a provider configuration during the apply phase, while allowing the plan phase to use a read-only or otherwise-less-privileged role.	2 years ago
Martin Atkins	e74896bd24	stackruntime: Handle apply-time-specified input variables ... When the topmost stack configuration declares an ephemeral input variable, its values must be provided separately for each of the plan and apply phases. Therefore here we extend the API to allow specifying input variable values during the apply phase, and add rules to check whether all of the apply-time-required input variables have been specified and whether any non-ephemeral variables are either unspecified or re-specified with equal values during the apply phase. This also extends the FindStackConfigurationComponents response to include more metadata about the input variables and output values so that a caller can know which ones are ephemeral. The name of that RPC function had already become a little too specific with the inclusion of embedded stack information and is even moreso now; we might choose to rename it to something more generic like "AnalyzeStackConfiguration" in future, but that'd be a breaking change and therefore requires more coordination.	2 years ago
Martin Atkins	79f78b268f	stackeval: Ephemeral output values ... Now we'll mark the result of an ephemeral output value as ephemeral, and reject attempts to assign any ephemeral values to a non-ephemeral output value.	2 years ago
Martin Atkins	71749d9682	stackeval: Ephemeral input variables ... We'll now handle input variables that are declared as ephemeral = true, and reject attempts to assign ephemeral values to any variables that are not so declared.	2 years ago
Liam Cervante	517f9944f7	stacks: support deferred imports in stacks events (#35334) ... * stacks: support deferred imports in stacks events * fix indentation * fix tests	2 years ago
Liam Cervante	108bd85053	Refactor deferred variable name to defer in terraform1 API (#35343) ... * Refactor deferred variable name to defer in terraform1 API * fix tests	2 years ago
Mutahhir Hayat	d398fa50c6	handle overlooked error in test	2 years ago
Mutahhir Hayat	cb28bf3379	remove unused test	2 years ago
Mutahhir Hayat	e20891ac1d	add tests for local values through data sources	2 years ago
Liam Cervante	9b95898f63	stacks: emit events for deferred actions (#35325) ... * stacks: emit events for deferred actions * deferral allowed is always on * Update internal/rpcapi/stacks.go Co-authored-by: Daniel Schmidt <danielmschmidt92@gmail.com> --------- Co-authored-by: Daniel Schmidt <danielmschmidt92@gmail.com>	2 years ago
Liam Cervante	1bb78c14c2	deferred actions: always enable deferred actions in stacks (#35321)	2 years ago
Liam Cervante	fc390038f3	stacks: add tests for and refactor steps when a provider can't be configured (#35294)	2 years ago
Liam Cervante	eeb2c53385	stacks: remove wildcard key from computed instances (#35284) ... * stacks: remove wildcard key from computed instances * fix compliation error	2 years ago
Liam Cervante	dc06717f69	stacks: add support for unknown for_each expressions in providers (#35267) ... * stacks: add support for unknown for_each expressions in providers * return more precise values for the unknown provider	2 years ago
Liam Cervante	1e9a186059	stacks: return dynamic types when referencing partial components (#35254)	2 years ago
Liam Cervante	7096ec85cc	stacks: Enable resource deferrals within stack tests (#35168)	2 years ago
Daniel Schmidt	59ead5356f	stacks: update RPC APIs with deferral information (#35125) ... * stacks: add deferred resource instance planned change to protobuf * stacks: add deferred resource instance to stack plan sequence * stacks: add planned change for deferred actions * stacks: refactor planned change resource instance planned moving the components out of the main function definition so that we can reuse the implementation for deferred resource instances which wraps the message used for PlannedChangeResourceInstancePlanned * stacks: track deferred changes in stackplan * add simple tests * fix tests * address comments --------- Co-authored-by: Liam Cervante <liam.cervante@hashicorp.com>	2 years ago
Alisdair McDiarmid	c8eec89c3d	Merge pull request #35247 from hashicorp/alisdair/rpcapi-stop ... rpcapi: Allow stopping long-running operations	2 years ago
Liam Cervante	b5d6b13b3b	stacks: convert sourcebundle source into directory for config SourceDir field (#35245)	2 years ago
Martin Atkins	5003af8335	stackeval: If context is cancelled, ask provider instances to stop ... Co-authored-by: Alisdair McDiarmid <alisdair@users.noreply.github.com>	2 years ago
Alisdair McDiarmid	39373cc0d6	stackeval: Stop operations on context cancellation ... When our context is cancelled, we instruct the modules runtime to stop. This aborts the graph traversal after any current node operations are completed.	2 years ago
Martin Atkins	9448845e91	stackeval: Improve coverage of namedPromiseReporter impls ... The "namedPromiseReporter" interface is a helper we use to collect user-friendly names for promises when we need to report a promise self-reference error to the user. We collect names for promises reactively on error, rather than proactively in the normal course of work, because proactively maintaining a suitable data structure for this cross-cutting concern would be both expensive and complicated due to it being a cross-cutting concern. This compromise therefore allows us to pay the cost of walking over all of the promise-owning objects only in the exceptional case where we need to report a self-reference error, but it does come at the expense of it being possible for us to forget to implement this interface for certain types and thus be unable to name a subset of the affected promises. This commit implements reportNamedPromises for all promise-owning types that are reachable from a stackeval.Main, although there is a notable gap here where we can't actually report promises from the individual instances of a multi-instance object (e.g. a component) because the promising API doesn't let us peek into a promising.Once to see if it's resolved without blocking until it becomes resolved. There are FIXME comments for that limitation in all of the places where it's relevant, and so hopefully in a future commit we'll devise a suitable way to perform a fallible non-blocking peek of a promise so that we can report promises nested in its result only if they are already present.	2 years ago
Martin Atkins	a4d19a8d13	stackeval: Stack.ProviderByLocalAddr must memoize its results ... A Provider object owns promises for related work, so we need to make sure we treat these as singletons to ensure that the associated work gets done only once and all callers end up sharing the same result. We've got away with this so far because the "related work" for providers is all deterministic based on the configuration anyway, and so without this we're just inefficient (doing the same work multiple times) rather than incorrect (producing inconsistent results). However, we should preserve the typical contract that objects containing promises are always treated as singletons, because otherwise this inconsistency will be a maintenance hazard for anyone who (reasonably) expects objects of this type to be treated the same as objects of other promise-owning types.	2 years ago
Liam Cervante	7a8ffff66e	stacks: Fix unknown provider values crashing validation phase (#35155) ... * test to reproduce * stacks: Fix unknown provider values crashing validation phase * remove extra type information --------- Co-authored-by: CJ Horton <cjhorton@hashicorp.com>	2 years ago
Daniel Schmidt	2c65affe05	stacks: add comments to tests	2 years ago
Daniel Schmidt	b1e50d0475	chore: fix comment ... Co-authored-by: Nick Fagerlund <nick.fagerlund@gmail.com>	2 years ago
Daniel Schmidt	664bb21738	stacks: add comment ... Co-authored-by: Nick Fagerlund <nick.fagerlund@gmail.com>	2 years ago
Mutahhir Hayat	6bbc8cdc76	Add more tests for local values	2 years ago
Mutahhir Hayat	2667a0f2c3	cleanup comments	2 years ago
Mutahhir Hayat	5e12bdf048	Stack evaluation should also visit local values when walking the stack	2 years ago
Mutahhir Hayat	be317d4d3a	Make Stack and Stack config aware of local values	2 years ago
Mutahhir Hayat	6554ce6bfd	Add planning test to test for locals usage	2 years ago
Mutahhir Hayat	129d58fed3	Create local value structs, including config and tests	2 years ago
Mutahhir Hayat	d695f8197c	Add local values test	2 years ago
Daniel Schmidt	3267718d3d	stacks: support unknown for_each on embedded stacks	2 years ago
kmoe	b74715e5fb	stacks: fix invalid filename (#35111)	2 years ago
Daniel Schmidt	35f44b01f0	stacks: fix comment and add assertion for diagnostics	2 years ago
Daniel Schmidt	aeb5b47723	stacks: remove unused test case	2 years ago
Daniel Schmidt	f847360a40	stacks: component.CheckInstances now reports on for each value diagnostics ... So there is no need for checkValid to do this during th walkDynamicObjects phase	2 years ago
Daniel Schmidt	c370e5e71d	stacks: use more precise name for variable	2 years ago
Daniel Schmidt	1afc9789ef	stacks: use UnknownForEachRepetitionData for instance map repetition data in for_each	2 years ago
Daniel Schmidt	91a18a6b30	stacks: fix typo ... Co-authored-by: Liam Cervante <liam.cervante@hashicorp.com>	2 years ago
Daniel Schmidt	7d0c4cfc4d	stacks: early exit on invalid for_each values for components	2 years ago
Daniel Schmidt	d1168ffa62	stacks: return instance value for wildcard component instances	2 years ago
Daniel Schmidt	6f95225a91	stacks: handle unknown dynamic values	2 years ago
Daniel Schmidt	3cbfd39721	stacks: validate unknown component for_each type	2 years ago
Daniel Schmidt	3a7150b27e	stacks: handle unknown values in component for_each	2 years ago
Martin Atkins	30e2fd6525	Handle marks a little more consistently ... In the very first implementation of "sensitive values" we were unfortunately not disciplined about separating the idea of "marked value" from the idea of "sensitive value" (where the latter is a subset of the former). The first implementation just assumed that any marking whatsoever meant "sensitive". We later improved that by adding the marks package and the marks.Sensitive value to standardize on the representation of "sensitive value" as being a value marked with _that specific mark_. However, we did not perform a thorough review of all of the mark-handling codepaths to make sure they all agreed on that definition. In particular, the state and plan models were both designed as if they supported arbitrary marks but then in practice marks other than marks.Sensitive would be handled in various inconsistent ways: dropped entirely, or interpreted as if marks.Sensitive, and possibly do so inconsistently when a value is used only in memory vs. round-tripped through a wire/file format. The goal of this commit is to resolve those oddities so that there are now two possible situations: - General mark handling: some codepaths genuinely handle marks generically, by transporting them from input value to output value in a way consistent with how cty itself deals with marks. This is the ideal case because it means we can add new marks in future and assume these codepaths will handle them correctly without any further modifications. - Sensitive-only mark preservation: the codepaths that interact with our wire protocols and file formats typically have only specialized support for sensitive values in particular, and lack support for any other marks. Those codepaths are now subject to a new rule where they must return an error if asked to deal with any other mark, so that if we introduce new marks in future we'll be forced either to define how we'll avoid those markings reaching the file/wire formats or extend the file/wire formats to support the new marks. Some new helper functions in package marks are intended to standardize how we deal with the "sensitive values only" situations, in the hope that this will make it easier to keep things consistent as the codebase evolves in future. In practice the modules runtime only ever uses marks.Sensitive as a mark today, so all of these checks are effectively covering "should never happen" cases. The only other mark Terraform uses is an implementation detail of "terraform console" and does not interact with any of the codepaths that only support sensitive values in particular.	2 years ago
Nick Fagerlund	4d46cc81e3	Fix several deferred actions tests by setting DeferralAllowed	2 years ago
Nick Fagerlund	2b4147e0ba	Add DeferralAllowed to stackeval.PlanOpts (temporary) ... Ultimately, we want deferred actions to always be enabled for Stacks. However, it's not ready to turn on yet, as it currently regresses other features. So, adding a top-level option that flows through to the main terraform.PlanOpts field will allow us to enable it selectively in tests while we sort out various behaviors.	2 years ago
Nick Fagerlund	d5be799cb9	End unknown_instances language experiment, in favor of whole-plan option ... We're still iterating on the ability to defer unplannable changes to a later plan. However, instead of enabling it via a "contagious" experimental feature switch that can bubble up from any module, it's now behind a per-plan option in the PlanOpts struct. That option travels a winding path into the internals (see previous commit), and node execution logic can then access it during the plan walk via the `Deferred.DeferralAllowed()` method. This commit updates the spot where we switch between the deferral yea/nay code paths to respect this top-level plan option rather than the experiment flag.	2 years ago
Alisdair McDiarmid	f409639934	Merge pull request #34765 from hashicorp/alisdair/stack-root-input-variable-defaults ... stackruntime: Apply defaults to root variables	2 years ago
Alisdair McDiarmid	f3ec86b17b	stackruntime: Treat unset and null equally ... When handling root input variable values, we now consider unset and null values to be equivalent to each other. This is consistent with how we handle variables in embedded stacks, and very similar to how we handle variable in the modules runtime with `nullable = false`. One difference from the modules runtime case is that we do not prevent a null default value for stack variables.	2 years ago
Alisdair McDiarmid	0fe26468cd	stackruntime: Apply defaults to root variables ... When evaluating a stack's root input variables, supplied by the caller, we must apply any default values specified in the variable configuration for variables with no specified value. This commit adds this default fallback case, using NilVal as a marker indicating the lack of a specified value. If no default value exists for a variable, it is therefore required to be supplied by the caller. This commit also reports a diagnostic error in this case.	2 years ago
CJ Horton	5a750561e6	round-trip check results through stack plans ... Components expect that check results will be round-tripped through the plan in order to update unknown check results during the apply. We hadn't wired this up for stack plans, resulting in panics when the apply tries to update a check status that doesn't exist.	2 years ago
Daniel Schmidt	49ad27f1c9	Merge pull request #34737 from hashicorp/stacks-for-each ... stacks: make component for_each more robust	2 years ago
Nick Fagerlund	c9c24706aa	Merge pull request #34741 from hashicorp/nf/feb24-provider-addrs-in-statuses ... Stacks, Hooks: Include full provider addresses	2 years ago
Daniel Schmidt	2a0962328d	stacks: make error message more precise	2 years ago
Daniel Schmidt	8da61e2d20	stacks: reword for_each error message	2 years ago
Daniel Schmidt	217879e510	stacks: make component for_each more robust	2 years ago
Liam Cervante	cec4b7b74c	stacks: validate providers based on types instead of local names (#34735) ... * stacks: validate providers based on types instead of local names * fix tests	2 years ago
Nick Fagerlund	f58c71e77c	Stacks: Update expectations in TestTerraformHook	2 years ago
Nick Fagerlund	7703a5aead	Stacks: Add ProviderAddr field to ResourceInstanceStatusHookData ... This updates the stackruntime `hooks.ResourceInstanceStatusHookData` struct to include the provider address, and updates everything that instantiates that struct to pass along the valid provider address it received from its own caller. In other words, this commit is a bridge between the terraform.Hook interface methods (which already have access to the provider address) and the stacks hook callbacks that result in RPC messages being sent to the agent.	2 years ago
Nick Fagerlund	94ea534e40	Refactor terraform.Hook to use a resource-identifying wrapper struct ... The terraform.Hook interface lets other areas of code perform streaming reactions to various events, generally in the service of some UI somewhere. Nearly all of the methods on this interface take an `addrs.AbsResourceInstance` as their first argument, to identify the resource that's being operated on. However, that addrs struct doesn't necessarily contain everything you might want in order to uniquely and usefully identify a resource. It has the module instance and resource instance addresses, but it lacks the provider source address, which can affect how the consuming UI should display the resource's events. (For example, Terraform Cloud wants reliable info about who maintains a given provider, what cloud provider it operates on, and where to find its documentation.) Instead of polluting `addrs.AbsResourceInstance` with extra information that isn't relevant to other call sites, let's change the first argument of each Hook method to be a wrapper struct defined in the package that owns the Hook interface, and add the provider address to that wrapper as a sibling of the resource address. This causes a big noisy commit today, but should streamline future updates to the UI-facing "identity" of a resource; existing callers can ignore any new fields they're uninterested in, or exploit new info as needed. Other than making new information available for future edits to Hook implementing types, this commit should have no effect on existing behavior.	2 years ago
Liam Cervante	31a7fa88d5	stacks: validate provider configurations during static analysis (#34730)	2 years ago
Alisdair McDiarmid	1459825e53	Merge pull request #34725 from hashicorp/alisdair/configs-sourcebundle-parser ... Add and use a source bundle aware configuration parser for stacks	2 years ago
Alisdair McDiarmid	2c22c213e9	stackeval: More robust absolute source addrs ... Instead of relying on the module call's source being unique, we now use the entire path as the key for looking up the parent's absolute source address.	2 years ago
Alisdair McDiarmid	7a043fe54a	stackeval: Another module source test case ... If multiple submodules of a given component share the same relative source, they should result in distint source addresses in the diagnostics. This commit introduces an example with the following structure: - component.remote_invalid_grandchildren - module.first_child - module.child (source = "./child") - module.second_child - module.child (source = "./child") Both of these use the same invalid module as the rest of the examples in this test, so we should (and do) see equivalent diagnostics differing only in filename.	2 years ago
Alisdair McDiarmid	7d956bbc11	stackeval: Fix module loader source addr building ... The source bundle aware module loader requires "absolute" source addresses, which are fully qualified rather than relative. We generate these during the module loading process. The previous implementation assumed that any local module source address should be parented by the root module source, but this is incorrect when a descendant module targets a remote or registry source. This commit addresses this by tracking each module request's generated absolute source address, and using it as the base for any descendant local module requests.	2 years ago
Martin Atkins	a45d4467f1	stackeval: Don't fail when data resource is removed ... Due to an oversight in our handling of resource instance objects that are neither in configuration nor plan -- which is true for data resources that have since been removed from the configuration -- we were generating plan change objects that were lacking a provider configuration address, which made them syntactically invalid and thus not reloadable using the raw plan parser. This is a bit of a strange situation since we don't technically _need_ a provider configuration address for these; all we're going to do is just unceremoniously delete them from the state during apply anyway. However, we always have the provider configuration address available anyway, so adding this in here is overall simpler than changing the parser, the models it populates, and all of the downstream users of those models to treat this field as optional. This commit is more test case than it is fix, since the fix was relatively straightforward once I had a test case to reproduce the problem it's fixing.	2 years ago
Liam Cervante	b2cc7dbadf	stacks: validate the types of input variables during validation (#34722)	2 years ago
Liam Cervante	68fd15dfdd	stacks: expand reference scope of static validations (#34721)	2 years ago
Liam Cervante	831630fabe	stacks: the plan function should stop on validation errors (#34720)	2 years ago
Alisdair McDiarmid	97a220fb82	stacks: Use source bundle aware config parser	2 years ago
Liam Cervante	b3abff5750	stacks: check providers blocks in components during validatation (#34707)	2 years ago
Liam Cervante	bced645a4d	stacks: call terraform.Validate on stack component configs (#34705)	2 years ago
Liam Cervante	2472f7c773	stacks: fix flaky tests (#34708)	2 years ago
Alisdair McDiarmid	48148eda5f	Merge pull request #34687 from hashicorp/alisdair/stackeval-hook-replace-action-counts ... stackeval: Add support for replace actions to hook	2 years ago
Liam Cervante	64b311d53a	stacks: apply sensitive marks to outputs from components (#34684) ... * stacks: apply sensitive marks to outputs from components * gen headers * ensure consistent test ordering	2 years ago
Liam Cervante	f0b61090a2	Move the testing MockProvider into a dedicated shared package (#34681) ... * move the testing MockProvider into a dedicated shared package * address comments * fix new changes	2 years ago
Alisdair McDiarmid	06723d3d4b	stackeval: Add support for replace actions to hook ... The terraform.Hook implementation in stackeval is used to track the operations performed during stack runtime operations, for later reporting to the caller. This hook did not correctly support the replace actions (create-then-destroy, destroy-then-create), resulting in a loss of data between plan and apply. This exhibited as a plan with a replace reporting 1 add and 1 remove operation, which when applied would report only 1 add operation. Previously, we stored the action performed for each apply operation on a given resource instance in the PreApply hook, then allowed access to it via the ResourceInstanceObjectAppliedAction method. Here we extend the PostApply hook to look for a previous action performed on this instance, and use that to reconstruct the planned replace action. This method is called in the stackruntime package, where replace operations are counted for both add and remove. No changes are needed at the call site to fix the bug.	2 years ago
Martin Atkins	fde6277565	stackruntime: Pass ExperimentsAllowed into stackeval for apply ... This was originally part of 7dad938fdb but unfortunately seems to have got lost during some rebasing, or some other similar sort of annoying reason. This now allows the "experiments allowed" flag to propagate into the stackeval package when we're running the apply phase, for consistency with all of the other phases. Without this, it's possible to plan a configuration that's participating in experiments, but then it fails in a strange way during the apply step due to Terraform suddenly thinking it's a stable release where experiments are disabled.	2 years ago
Liam Cervante	07f6621091	stacks: include resources in state when calculating required providers (#34645) ... * stacks: include resources in state when calculating required providers * also support apply time * add copywrite headers	2 years ago
Martin Atkins	e6665c4f43	stackeval: Basic support for deferred-change propagation ... This is the bare minimum functionality to ensure that we defer all actions in any component that depends on a component that already had deferred actions. We will also eventually need to propagate out a signal to the caller for whether the stack plan as a whole is complete or incomplete, but we'll save that for later commits, since the stack orchestration in Terraform Cloud will do the right thing regardless, aside from the cosmetic concern that it won't yet know to show a message to the user saying that there are deferred changes.	2 years ago
Martin Atkins	67d8a5137b	rpcapi+stacks: Stacks runtime can see whether experiments are allowed ... We allow experiments only in alpha builds, and so we propagate the flag for whether that's allowed in from "package main". We previously had that plumbed in only as far as the rpcapi startup. This plumbs the flag all the way into package stackeval so that we can in turn propagate it to Terraform's module config loader, which is ultimately the one responsible for ensuring that language experiments can be enabled only when the flag is set. Therefore it will now be possible to opt in to language experiments in modules that are used in stack components.	2 years ago
Liam Cervante	3475d22741	Fix nil pointer exception when planning stacks with undeclared variables (#34638) ... * fix nil pointer exception when planning with undeclared variables * another function does the same	2 years ago
Alisdair McDiarmid	55675b990a	stackeval: Unmark provider config for plugin RPC ... When validating or applying a provider configuration, we must unmark the value before attempting to use plugin RPC, as we cannot serialize marks.	2 years ago
Martin Atkins	884e1fb2a4	terraform: Plans can be "complete" and "applyable" ... These ideas are both already implied by some logic elsewhere in the system, but until now we didn't have the decision logic centralized in a single place that could therefore evolve over time without necessarily always updating every caller together. We'll now have the modules runtime produce its own boolean ruling about each characteristic, which callers can rely on for the mechanical decision-making of whether to offer the user an "approve" prompt, and whether to remind the user after apply that it was an incomplete plan that will probably therefore need at least one more plan/apply round to converge. The "Applyable" flag directly replaces the previous method Plan.CanApply, with equivalent logic. Making this a field instead of a method means that we can freeze it as part of a saved plan, rather than recalculating it when we reload the plan, and we can export the field value in our export formats like JSON while ensuring it'll always be consistent with what Terraform is using internally. Callers can (and should) still use other context in the plan to return more tailored messages for specific situations they already know about that might be useful to users, but with these flags as a baseline callers can now just fall back to a generic presentation when encountering a situation they don't yet understand, rather than making the wrong decision and causing something strange to happen. That is: a lack of awareness of a new rule will now cause just a generic message in the UI, rather than incorrect behavior. This commit mostly just deals with populating the flags, and then all of the direct consequences of that on our various tests. Further changes to actually make use of these flags elsewhere in the system will follow in later commits, both in this repository and in other repositories.	2 years ago
Martin Atkins	0994e6fbf9	stacks: The terraform.workspace attr is not available in Stacks ... The terraform.workspace attribute is a rare example of a CLI- and Cloud- specific concern bleeding into the Terraform language, and it can only really have meaning when used in the traditional Terraform workflow because otherwise there's no workspace to return the name of. In Stacks any variations between instances of a module must be created through input variables. Within Terraform Cloud in particular it's also possible to use stack-level input variables that are assigned different values from different stack deployments, and thus an author can recreate the effect of terraform.workspace using a stack-level input variable that has a different value for each deployment. This is one of the few cases where the Terraform module language differs in stacks compared to traditional Terraform. Any module that makes use of terraform.workspace will need to be generalized to use input variables instead before it can be used within a stack component. Prior to this change, references to terraform.workspace from a module used in a stack component would just panic altogether, because the stacks runtime doesn't provide the object that the workspace name would be taken from. Now we'll return a user-oriented error instead.	2 years ago
Martin Atkins	7d2d4dec5e	stackeval: When being destroyed, component instance result comes from plan ... Because we treat dependency edges as reversed when a component instance is being destroyed, the final result (an object representing output values) for a component instance being destroyed must not depend on anything else in the evaluation graph, or else we'd cause a promise self-reference as the downstream component tries to configure itself based on our outputs. As a special case then, for a component instance being destroyed we take the planned output values directly from the plan, relying on the fact that the plan phase sets them to the prior state output values in that case, and therefore the result for such a component is available immediately without blocking on any other expression evaluation during the apply phase. This combines with several previous commits to create a first pass at handling ordering correctly when planning and applying a full destroy. This commit also incorporates some fixes and improvements to stackeval's apply-time testing helpers, which had some quirks and bugs when first added in a recent commit. One of those problems also revealed that the raw state loader was not resilient to a buggy caller setting a state entry to nil instead of removing it altogether, and that mistake seems relatively easy to make (as I did here in the test helper) so we'll tolerate it to make it possible to recover if such a bug does end up occurring in real code too.	2 years ago
Martin Atkins	dcb176a557	stackeval: A Main for applying knows the plan it's applying	2 years ago
Martin Atkins	c06ad051a1	stackeval: Respect inter-component dependencies during apply ... Before we start any apply-time actions for a particular component, we'll block until other components' apply are complete. In the normal case a component instance waits until its dependencies have been applied. However, if component instances are being destroyed then they instead wait for their _dependents_ to complete being applied, since a component must outlive all other components that depend on it.	2 years ago
Martin Atkins	c345cde93c	stackeval: Extended utilities for plan and apply testing ... These helpers make it easier to write our tests because the plan and apply phases both work in a callback-based fashion where they gradually emit events, and that's not a very convenient API for test code to interact with. This exposes some more details about the planning results, and also adds a new similar helper for the apply phase. We'll make use of both of these in future commits.	2 years ago
Liam Cervante	9246b3a994	stacks: on module load convert relative paths to absolute (#34523)	2 years ago
Liam Cervante	7d143388f3	Add support for the cross resource type move operation in the proto schema (#34480) ... * Update proto schema and provider interfaces with support for moved across resource type RPCs * address comments * remove unused functions * remove support for flatmap format	2 years ago
Martin Atkins	3c14eeb945	stackeval: Make some provisioners available to stack components ... This makes the built-in "remote-exec" and "file" provisioners available for use in the modules that implement stack components. These are both relatively easy and low-risk to include because they are builtins and don't require anything from outside of Terraform itself. For now this intentionally excludes local-exec because we'll want to think about what constraints we want to put on it, if any, to help ensure we can meet the goal of stack configurations being portable between different execution environments without significant modification, and our current stacks execution environment doesn't guarantee the availability of any external software _at all_. The motivation for adding this now is just to give some better feedback when someone uses a module using one of these provisioners, since otherwise they'll see just a confusing generic error message from the modules runtime about the provisioners not being available. I expect we'll revisit this later and consider expanding it to at least include local-exec, and _maybe_ external provisioner plugins, although that's more questionable because the provisioner plugin mechanism is incredibly legacy and doesn't have any way to work outside of local Terraform CLI usage today. There are no tests here yet because these provisioners are not mockable and would depend on having an SSH or WinRM server to connect to. Later we should ponder how to make this more testable, which might mean making another part of the system responsible for actually providing the provisioner factories and thus our tests here can use fakes. The goal here is just to get this done in a relatively lightweight way for better feedback during preview though, so we're not yet ready to make significant time investments here.	2 years ago