40 Commits (main)

Author	SHA1	Message	Date
Radek Simko	56c39007e2	make protobuf	3 days ago
Radek Simko	0fe906fa8c	make copyrightfix	3 days ago
Sarah French	2e5b5dee5d	Upgrade protoc and protoc-gen-go-grpc versions to matching terraform-plugin-go (#37647) ... * Update protoc version in downloader script * go get google.golang.org/grpc/cmd/protoc-gen-go-grpc v1.5.1 This matched terraform-plugin-go * make protobuf * Run `make protobuf` * Update generator to handle generic types from google.golang.org/grpc Looks like this was added in v1.69.3 in https://github.com/grpc/grpc-go/pull/7057 ? * Run `make generate` * Fix "cannot infer Res" compile error - more usage of generics * More fixing compile errors due to switching to use of a generic * Make putting `google.golang.org/grpc` import into generated files conditional * Run `make generate` * Update more places where generics now need to be used * Update generator to handle any types from google.golang.org/grpc in same switch case.	4 months ago
James Bardin	7ed24d208a	make protobuf	6 months ago
James Bardin	a6ec17cb77	rename function results table ... Remove "provider" from the function results fields since it's not going to be exclusively used for providers any longer.	10 months ago
Sarah French	6ef0498f9c	Update to Go 1.24.1, update dependencies (#36790) ... * go get github.com/hashicorp/go-azure-helpers@v0.72.0 * .go-version to 1.24.1 * Run `./scripts/syncdeps.sh` This propagates the Go version upgrade * gcs : go get google.golang.org/api@v0.155.0 * gcs: go mod tidy * Run `./scripts/syncdeps.sh` * make generate && make protobuf	11 months ago
Sarah French	a04664a2e5	Update `hashicorp/go-plugin` dependency (#36561) ... * go get github.com/hashicorp/go-plugin * ./scripts/syncdeps.sh * make protobuf	11 months ago
Liam Cervante	b38fd17cf9	stacks: emit removal notices for empty components (#35738) ... * stacks: emit removal notices for empty components * fix tests and checks	1 year ago
Liam Cervante	e78294d42b	stacks: stop encoding values into msgpack twice (#35734)	1 year ago
Liam Cervante	d142486a40	stacks: expand plan and apply outputs for inputs (#35724)	1 year ago
Liam Cervante	73e3f8096b	stacks: complete stack output implementation for plan and apply (#35723)	1 year ago
James Bardin	9ff727440e	udpate generated files	1 year ago
Liam Cervante	f8fe397d88	stacks: add removed block functionality (#35671) ... * stacks: add removed block functionality * fix compile errors	1 year ago
Liam Cervante	b6ac98122b	stacks: refactor shared functionality in prep for removed blocks (#35670)	1 year ago
Liam Cervante	7163c4b6d5	stacks: fix destroy ordering of stacks destroy plans (#35665) ... * stacks: remove unneeded required components functions * stacks: refactor the mock provider for realism * deferred actions: fix missing features in deletions * stacks: fix destroy ordering of stacks destroy plans * fix missing import	1 year ago
Liam Cervante	483564a0ea	stacks: switch protofile licences to BUSL (#35633)	1 year ago
Liam Cervante	50233d7a78	stacks: support sensitive input values in planned changes (#35640) ... * stacks: support sensitive input values in planned changes * stacks: Fix panic when applying sensitive inputs The stacks-specific check that a given plan's input variable values remain unchanged at apply time would previously panic when given a marked value. Marks are not important for this check, so we simply remove them to ensure that it passes. This seems in line with how AssertObjectCompatible works, which I'm taking as precedent. The new test added in this commit panics without the corresponding code change. --------- Co-authored-by: Alisdair McDiarmid <alisdair@users.noreply.github.com>	2 years ago
Liam Cervante	f1ae07b1af	stacks: add support for provider functions in .tfstack.hcl files (#35610) ... * stacks: fix bug preventing cross-provider move refactorings * also make provider functions work * stacks: add support for provider functions in .tfstack.hcl files	2 years ago
Liam Cervante	915b174da3	stacks: split the terraform1 RPC package into per-service packages (#35513) ... * stacks: split the terraform1 RPC package into per-service packages * pull latest changes	2 years ago
Liam Cervante	87bbc47737	stacks: include plan mode in stacks plan format (#35405) ... * stacks: include plan mode in stacks plan format * fix tests * fix missed files	2 years ago
Martin Atkins	ec2510fc3d	stacks: Track raw stack as separate messages in raw plan ... Previously we had the entire raw prior state serialized as part of the "plan header" message in the raw plan, which meant that the maximum state size was constrained by the maximum allowed gRPC message size. Instead now we'll use a separate raw plan element for each raw prior state element, so that we're limited only in the size of individual items rather than size of the state as a whole. This deals with the last remaining (non-deprecated) case where our RPC protocol tries to pack an entire raw state or plan into a single protobuf message, and so we're now standardized on using a streaming approach in all cases.	2 years ago
Daniel Schmidt	b62a67394b	stacks: save plantimestamp to plan	2 years ago
Martin Atkins	e74896bd24	stackruntime: Handle apply-time-specified input variables ... When the topmost stack configuration declares an ephemeral input variable, its values must be provided separately for each of the plan and apply phases. Therefore here we extend the API to allow specifying input variable values during the apply phase, and add rules to check whether all of the apply-time-required input variables have been specified and whether any non-ephemeral variables are either unspecified or re-specified with equal values during the apply phase. This also extends the FindStackConfigurationComponents response to include more metadata about the input variables and output values so that a caller can know which ones are ephemeral. The name of that RPC function had already become a little too specific with the inclusion of embedded stack information and is even moreso now; we might choose to rename it to something more generic like "AnalyzeStackConfiguration" in future, but that'd be a breaking change and therefore requires more coordination.	2 years ago
Liam Cervante	9b95898f63	stacks: emit events for deferred actions (#35325) ... * stacks: emit events for deferred actions * deferral allowed is always on * Update internal/rpcapi/stacks.go Co-authored-by: Daniel Schmidt <danielmschmidt92@gmail.com> --------- Co-authored-by: Daniel Schmidt <danielmschmidt92@gmail.com>	2 years ago
Daniel Schmidt	59ead5356f	stacks: update RPC APIs with deferral information (#35125) ... * stacks: add deferred resource instance planned change to protobuf * stacks: add deferred resource instance to stack plan sequence * stacks: add planned change for deferred actions * stacks: refactor planned change resource instance planned moving the components out of the main function definition so that we can reuse the implementation for deferred resource instances which wraps the message used for PlannedChangeResourceInstancePlanned * stacks: track deferred changes in stackplan * add simple tests * fix tests * address comments --------- Co-authored-by: Liam Cervante <liam.cervante@hashicorp.com>	2 years ago
Martin Atkins	30e2fd6525	Handle marks a little more consistently ... In the very first implementation of "sensitive values" we were unfortunately not disciplined about separating the idea of "marked value" from the idea of "sensitive value" (where the latter is a subset of the former). The first implementation just assumed that any marking whatsoever meant "sensitive". We later improved that by adding the marks package and the marks.Sensitive value to standardize on the representation of "sensitive value" as being a value marked with _that specific mark_. However, we did not perform a thorough review of all of the mark-handling codepaths to make sure they all agreed on that definition. In particular, the state and plan models were both designed as if they supported arbitrary marks but then in practice marks other than marks.Sensitive would be handled in various inconsistent ways: dropped entirely, or interpreted as if marks.Sensitive, and possibly do so inconsistently when a value is used only in memory vs. round-tripped through a wire/file format. The goal of this commit is to resolve those oddities so that there are now two possible situations: - General mark handling: some codepaths genuinely handle marks generically, by transporting them from input value to output value in a way consistent with how cty itself deals with marks. This is the ideal case because it means we can add new marks in future and assume these codepaths will handle them correctly without any further modifications. - Sensitive-only mark preservation: the codepaths that interact with our wire protocols and file formats typically have only specialized support for sensitive values in particular, and lack support for any other marks. Those codepaths are now subject to a new rule where they must return an error if asked to deal with any other mark, so that if we introduce new marks in future we'll be forced either to define how we'll avoid those markings reaching the file/wire formats or extend the file/wire formats to support the new marks. Some new helper functions in package marks are intended to standardize how we deal with the "sensitive values only" situations, in the hope that this will make it easier to keep things consistent as the codebase evolves in future. In practice the modules runtime only ever uses marks.Sensitive as a mark today, so all of these checks are effectively covering "should never happen" cases. The only other mark Terraform uses is an implementation detail of "terraform console" and does not interact with any of the codepaths that only support sensitive values in particular.	2 years ago
Martin Atkins	d3b8a55781	go.mod: go get google.golang.org/protobuf@v1.33.0 ... Many of the changes between these two versions seem to be in parts of the library that Terraform doesn't use, but there are a few sensitive parts: - Slight changes to details of how the library implements the JSON encoding of protobuf. Terraform CLI/Core don't use this, but some of the backends might use it indirectly when talking to their underlying APIs. - Some details of the "protoreflect" package for working with protobuf descriptors dynamically (rather than using generated stubs) have changed, but it seems pretty marginal and unlikely to cause problems.	2 years ago
CJ Horton	5a750561e6	round-trip check results through stack plans ... Components expect that check results will be round-tripped through the plan in order to update unknown check results during the apply. We hadn't wired this up for stack plans, resulting in panics when the apply tries to update a check status that doesn't exist.	2 years ago
Martin Atkins	884e1fb2a4	terraform: Plans can be "complete" and "applyable" ... These ideas are both already implied by some logic elsewhere in the system, but until now we didn't have the decision logic centralized in a single place that could therefore evolve over time without necessarily always updating every caller together. We'll now have the modules runtime produce its own boolean ruling about each characteristic, which callers can rely on for the mechanical decision-making of whether to offer the user an "approve" prompt, and whether to remind the user after apply that it was an incomplete plan that will probably therefore need at least one more plan/apply round to converge. The "Applyable" flag directly replaces the previous method Plan.CanApply, with equivalent logic. Making this a field instead of a method means that we can freeze it as part of a saved plan, rather than recalculating it when we reload the plan, and we can export the field value in our export formats like JSON while ensuring it'll always be consistent with what Terraform is using internally. Callers can (and should) still use other context in the plan to return more tailored messages for specific situations they already know about that might be useful to users, but with these flags as a baseline callers can now just fall back to a generic presentation when encountering a situation they don't yet understand, rather than making the wrong decision and causing something strange to happen. That is: a lack of awareness of a new rule will now cause just a generic message in the UI, rather than incorrect behavior. This commit mostly just deals with populating the flags, and then all of the direct consequences of that on our various tests. Further changes to actually make use of these flags elsewhere in the system will follow in later commits, both in this repository and in other repositories.	2 years ago
Martin Atkins	6071e06f1a	tfstackdata1: DynamicValueFromTFStackData1 decodes sensitive marks	2 years ago
Martin Atkins	a4d61733f6	stackplan: Include extra info in raw component instance plan ... We'll now track the planned action, the component addresses we depend on, and the planned output values.	2 years ago
Martin Atkins	bcccd67e32	tfstackdata1: A local DynamicValue message type ... Previously we were directly using the planproto.DynamicValue message type, but that's symmetrical with plans.DynamicValue in that it encodes only the value itself and not associated metadata such as the paths that have sensitive value marks. This new tfstackdata1.DynamicValue therefore echoes terraform1.DynamicValue by carrying both the value and the metadata together. This implies a slight change to one of the existing message types that was encoding a map of planned input values for each component instance, but in practice we're not currently making use of that data anyway -- it's there to enable a hypothetical extra correctness check in the stacks runtime so that we can detect inconsistency problems closer to their source -- and so this change doesn't affect anything in practice. Before we make use of it we'll probably want to change the internal API a little so that we can preserve the sensitive value metadata on those values, but that's beyond the scope of this PR. The main point of adding this is to track the output values for a component instance as part of the raw state, and so the new field for that is also added here although nothing will read or write it yet. Actual use of that new field will follow in future commits.	2 years ago
Martin Atkins	972f1c0678	stackplan: Track provider config for every resource instance change ... Previously we were still partially relying on the record of this in the embedded planproto object, but since we're now allowing "planned changes" that don't actually include a change in that traditional sense we need to track the provider configuration address separately as a top-level field. As with some of the addresses before, this means we're now storing the provider config address redundantly in two places when there _is_ a planproto change. Hopefully we'll improve that someday, but we don't need to sweat it too much right now because this only affects the internal raw plan format that is not subject to any compatibility promises between releases, so we can freely change it in any future version.	2 years ago
Martin Atkins	4e34e6ebfa	stacks: Track prior state for all resource instance objects ... Previously our raw plan data structure only included information about resource instances that have planned actions, which meant we were missing those which only get their state updated during refresh and don't actually need any real work done. Now we'll track everything, accepting that some "planned changes" for resource instance objects will not actually include a planned change in the typical sense, and instead only represent a state update.	2 years ago
Martin Atkins	8fd29e9439	stacks: Preserve prior state from plan to apply ... We need to retain the prior state (in a form that Terraform Core's modules runtime can accept) between the plan and apply phases because Terraform uses it to verify that the provider's "final plan" is consistent with the initial plan. We previously tried to do this by reusing the "old" value from the planned change, but that's not really possible in practice because of the historical implementation detail that Terraform wants state information in a JSON format and plan information in MessagePack. This also contains the beginnings of handling the "discarding" of unrecognized keys from the state data structures, although we'll probably need to do more in that area later since this is just the bare minimum.	2 years ago
Martin Atkins	69aa51b9ff	stackruntime: Fix some quirks of data sent from plan to apply ... We previously took a few shortcuts just to more quickly produce a proof of concept. This gets us considerably closer to a properly-working handoff from plan to apply, at least for create, update, and replace actions. Destroying stuff that's been removed from the configuration will take some further work since currently everything we do is driven by objects in the configuration. That will follow in later commits.	2 years ago
Martin Atkins	49511fe9a8	stackstate: Fuller round-tripping of resource instance object state ... This gets us ever closer to being able to preserve resource instance objects from one run to the next. In subsequent commits we'll make use of the "LoadFromProto" option to load the prior state during the planning phase and take it into account when we're deciding what actions to propose.	2 years ago
Martin Atkins	54622c68a5	stackstate: Generate "raw state" objects for resource instance objects ... Previously we just had a stub that only generated basic external description objects. Now we'll also emit "raw state" objects for these, and have some initial support for deposed objects too.	2 years ago
Martin Atkins	d99985720f	stacks: Carry input variable values from plan to apply ... We need to preserve the input variable values provided during the plan phase so we can reuse them during the apply phase to recalculate derived results based on new information learned during the apply process. We also need to recalculate the input values for individual components during the apply phase, so we can react to previously-unknown values that come from other upstream components.	2 years ago
Martin Atkins	f9317731e9	tfstackdata1: Protobuf schema for stack plan and state serialization ... This represents our _internal_ plan and state formats, which are private to Terraform Core so that we can evolve them over time without needing to coordinate with other systems. The plan parts of this are the stack-specific container messages around the existing "tfplan" proto schema that traditional Terraform uses for saved plan files. The state parts of this are really just stubs since we're not really dealing with state just yet, but they're here to illustrate where we intend to expand in future commits when we start implementing the apply phase and the ability to plan based on a prior state.	2 years ago