terraform

Commit Graph

Author	SHA1	Message	Date
Radek Simko	0fe906fa8c	make copyrightfix	3 months ago
Daniel Banck	b2b42c0fb4	Store resource identities in state (TF-23255) (#36464 ) * Persist resource identity in Terraform state * make syncdeps * Move identity schema merging closer to the protocol * mock GetResourceIdentitySchemas * Fix identity refresh tests * Add more tests * Change grcpwrap upgrade identity * Review feedback * Remove unnecessary version conversion * Check if GetResourceIdentitySchemas RPC call is implemented * Update function signature docs * Adapt protocol changes * Check unimplemented error for identities in GetSchema	1 year ago
Liam Cervante	bfa320c7b0	stacks: represent ephemeral inputs and outputs as null (#35824 ) * stacks: represent ephemeral inputs and outputs as null * quick fix: remove unnecessary check	2 years ago
James Bardin	384f2d4fab	update collections to use for-range method	2 years ago
Liam Cervante	b38fd17cf9	stacks: emit removal notices for empty components (#35738 ) * stacks: emit removal notices for empty components * fix tests and checks	2 years ago
Liam Cervante	e78294d42b	stacks: stop encoding values into msgpack twice (#35734 )	2 years ago
Liam Cervante	d142486a40	stacks: expand plan and apply outputs for inputs (#35724 )	2 years ago
Liam Cervante	73e3f8096b	stacks: complete stack output implementation for plan and apply (#35723 )	2 years ago
Liam Cervante	f8fe397d88	stacks: add removed block functionality (#35671 ) * stacks: add removed block functionality * fix compile errors	2 years ago
Liam Cervante	7163c4b6d5	stacks: fix destroy ordering of stacks destroy plans (#35665 ) * stacks: remove unneeded required components functions * stacks: refactor the mock provider for realism * deferred actions: fix missing features in deletions * stacks: fix destroy ordering of stacks destroy plans * fix missing import	2 years ago
Liam Cervante	915b174da3	stacks: split the terraform1 RPC package into per-service packages (#35513 ) * stacks: split the terraform1 RPC package into per-service packages * pull latest changes	2 years ago
Liam Cervante	27f26bd1b5	stacks: emit specific moved notification during apply (#35387 )	2 years ago
Martin Atkins	30e2fd6525	Handle marks a little more consistently In the very first implementation of "sensitive values" we were unfortunately not disciplined about separating the idea of "marked value" from the idea of "sensitive value" (where the latter is a subset of the former). The first implementation just assumed that any marking whatsoever meant "sensitive". We later improved that by adding the marks package and the marks.Sensitive value to standardize on the representation of "sensitive value" as being a value marked with _that specific mark_. However, we did not perform a thorough review of all of the mark-handling codepaths to make sure they all agreed on that definition. In particular, the state and plan models were both designed as if they supported arbitrary marks but then in practice marks other than marks.Sensitive would be handled in various inconsistent ways: dropped entirely, or interpreted as if marks.Sensitive, and possibly do so inconsistently when a value is used only in memory vs. round-tripped through a wire/file format. The goal of this commit is to resolve those oddities so that there are now two possible situations: - General mark handling: some codepaths genuinely handle marks generically, by transporting them from input value to output value in a way consistent with how cty itself deals with marks. This is the ideal case because it means we can add new marks in future and assume these codepaths will handle them correctly without any further modifications. - Sensitive-only mark preservation: the codepaths that interact with our wire protocols and file formats typically have only specialized support for sensitive values in particular, and lack support for any other marks. Those codepaths are now subject to a new rule where they must return an error if asked to deal with any other mark, so that if we introduce new marks in future we'll be forced either to define how we'll avoid those markings reaching the file/wire formats or extend the file/wire formats to support the new marks. Some new helper functions in package marks are intended to standardize how we deal with the "sensitive values only" situations, in the hope that this will make it easier to keep things consistent as the codebase evolves in future. In practice the modules runtime only ever uses marks.Sensitive as a mark today, so all of these checks are effectively covering "should never happen" cases. The only other mark Terraform uses is an implementation detail of "terraform console" and does not interact with any of the codepaths that only support sensitive values in particular.	2 years ago
Matej Risek	039cced8ae	Add component address to ComponentInstance under AppliedChanges (#34418 )	2 years ago
Martin Atkins	2aba28eb1a	stacks: Track component instance existence in the state Although we can often use the presence of resource instances in a component instance as an implication of the component instance's existence, it's possible (albeit rare) for a component instance to have no resources in it at all, and thus it would be ambiguous whether it exists or not. Now we'll track the existence of the component instances themselves as extra objects in both the raw state and the state description. Along with their existence we'll also track a snapshot of the output values as they were at the most recent apply, which for now is primarily for external consumption but we'll also include them in the raw state in case it ends up being useful for something down the road.	2 years ago
Matej Risek	8ac8f18636	Feature/add additional fields to resource proto (#34364 ) Add additional fields to the AppliedChange#ResourceInstance We're adding Resource Mode, Resource Type and Provider Address to the AppliedChange's ResourceInstance	2 years ago
CJ Horton	d60a3db261	stackstate: unmark values before serializing apply change descriptions When emitting apply descriptions for changes that contain sensitive values, we need to unmark the value before re-serializing it with MsgPack.	3 years ago
Martin Atkins	9c698a1de6	stackstate: Remove stale comment in AppliedChangeResourceInstanceObject This comment was noting that an earlier stub of this function was only returning an external facing "state description" object and not including the raw state for use when creating future plans. That was subsequently fixed -- this now returns both description and raw forms -- but the comment stuck around.	3 years ago
Martin Atkins	6e7eff9b4b	stackruntime: Report deletion of resource instance objects Previously our strategy for reporting the "applied change" results for resource instance objects was to iterate over the final state and emit an object for each resource instance we found in there. That's not a sufficient solution though, because if any objects are deleted as part of applying the plan then they won't be in the final state and so we won't know to tell the caller that they should drop the relevant objects from the stack-level state. Instead, we'll keep track of all of the resource instance objects that could potentially be affected by applying a plan and then use that set to decide which "applied change" objects to emit. If one of the affected objects is not present in the final state then we'll assume it was deleted and so command the caller to delete its records of that object from both the raw state and the state description. With this in place, it's now possible to create a destroy-mode plan and then apply it to end up with no resource instances at all. The stack runtime itself is still missing handling of destroying its own objects like components and embedded stacks, so this isn't a complete solution but it does at least allow properly cleaning up the records of objects that exist in remote systems, so it's now easier to clean up a development environment with real infrastructure backing it.	3 years ago
Martin Atkins	8fd29e9439	stacks: Preserve prior state from plan to apply We need to retain the prior state (in a form that Terraform Core's modules runtime can accept) between the plan and apply phases because Terraform uses it to verify that the provider's "final plan" is consistent with the initial plan. We previously tried to do this by reusing the "old" value from the planned change, but that's not really possible in practice because of the historical implementation detail that Terraform wants state information in a JSON format and plan information in MessagePack. This also contains the beginnings of handling the "discarding" of unrecognized keys from the state data structures, although we'll probably need to do more in that area later since this is just the bare minimum.	3 years ago
Martin Atkins	5372d5eb98	stacks+rpcapi(stacks): Model deposed objects for resource instances Previously we incorrectly assumed that resource instances only have one "current" object each. However, resource instances can also have deposed objects which we must also keep track of. This is a breaking change to the rpcapi since we're now using a new message type for a resource instance _object_ address in several places. That breakage is intentional here because at the time of writing this commit the rpcapi is not yet in any stable release and we want to force updating our existing internal client to properly handle deposed objects too.	3 years ago
Martin Atkins	49511fe9a8	stackstate: Fuller round-tripping of resource instance object state This gets us ever closer to being able to preserve resource instance objects from one run to the next. In subsequent commits we'll make use of the "LoadFromProto" option to load the prior state during the planning phase and take it into account when we're deciding what actions to propose.	3 years ago
Martin Atkins	54622c68a5	stackstate: Generate "raw state" objects for resource instance objects Previously we just had a stub that only generated basic external description objects. Now we'll also emit "raw state" objects for these, and have some initial support for deposed objects too.	3 years ago
Martin Atkins	60431e738e	stackstate: Use the statekeys package for resource instance object keys We can now replace our previous interim "tmpKey" with a real key in the syntax decided by our separate statekeys package.	3 years ago
Martin Atkins	1cb96ebb32	stackstate: Add HashiCorp copyright comments	3 years ago
Martin Atkins	d91e5dc9f1	stackstate: Include correct object data when announcing applied resource Previously we just had this just stubbed out to always return an empty object because we didn't have the schema information available to transcode the JSON-encoded state data into the form our RPC API wants to produce. An earlier commit added the schema information we need, so we can now transcode just in time to produce the protocol buffers serialization of the applied change description. This is still otherwise just a stub implementation. We'll still need to deal with emitting the "raw" representation of this state for use in future plans, and make sure we're properly handling deposed objects, in future commits.	3 years ago
Martin Atkins	74215a4451	stacks: Include resource type schema in PlannedChange and AppliedChange Unfortunately for historical reasons the "terraform" package produces state and plan artifacts that have dynamic data elements pre-encoded in the formats that Terraform CLI's traditional plan and state formats use. Since Stacks uses different plan and state models, and exclusively uses MessagePack encoding for dynamic values in both, we'll sometimes need to transcode from one format to another when generating and decoding the stacks-oriented representations. Transcoding between these serialization formats requires access to the relevant schema because the formats themselves each have an infoset that is only a subset of Terraform's type system. Therefore we'll now annotate the PlannedChange and AppliedChange objects that include provider-specific data types with the schema required to transcode them. This commit does not yet actually arrange to make use of those schemas. That will follow in later commits.	3 years ago
Martin Atkins	48518b8233	stacks: A very basic stub of applied resource instance change announcements We'd like to start developing some clients for this part of the RPC API concurrently with remaining work here in Terraform Core, and so this is a bare-minimum implementation of emitting "applied change" events for resource instances just so there's something for client developers to test against. There are various things wrong with this, including but not limited to: - It always reports that the new state for a resource instance is an empty object, rather than including the correct updated object. - It only supports "current" resource instance objects, and ignores deposed ones. - It just uses the resource instance address alone as the key for the state description map, which is fine for now when we only have one kind of description object anyway but will not be sufficient for a real implementation that needs to emit various different kinds of object. We'll need to rework most of this in future commits, but this is hopefully sufficient to start implementing and testing API clients, at least to some extend, despite the crudity of the results.	3 years ago
Martin Atkins	fafa36e73a	stackeval: ApplyPlan function and initial supporting internals This is a first pass at actually driving the apply phase through to completion, using the ChangeExec function to schedule the real change operations from the plan and then, just like for planning, a visit to each participating object to ask it to check itself and report the results of any changes it has made. Many of our objects don't have any external side-effects of their own and so just need to check their results are still valid after the apply phase has replaced unknown values with known values. For those we can mostly just share the same logic between plan and apply aside from asking for ApplyPhase instead of PlanPhase. ComponentInstance and OutputValue are the two objects whose treatment differs the most between plan and apply, because both of those need to emit externally-visible "applied change" objects to explain their side-effects to the caller. The apply-walk driver has a lot of behavior in common with the existing plan-walk driver. For this commit we're just accepting that and having two very similar pieces of code that differ only in some leaf details. In a future commit we might try to generalize that so we can share more logic between the two, but only if we can do that without making the code significantly harder to read.	3 years ago

29 Commits (stacks-variable-validations)