369 Commits (add-missing-variable-loading)

Author	SHA1	Message	Date
Martin Atkins	2d2baee72d	staticcheck: Broadly ignore unused stuff in all of "stacks" ... We're currently actively developing stuff under the "internal/stacks" directory such that there are some loose ends not yet completed in order to make our PRs at least _slightly_ more digestible. For that reason we'll broadly turn off the "unused symbol" checks from staticcheck under that entire directory for now, instead of making many single-file exceptions that will be harder to clean up later. Once the work in this directory settles we should remove this exception so that the stacks-related packages are subject to the same rules as everything else. Unfortunately staticcheck can only work with configuration files if there are no explicit checks on the command line, so this also makes the top-level checks be a configuration file rather than just a command line option in our staticcheck.sh script. The effect for the rest of the codebase is the same, but having the top-level rules in a configuration file means that subdirectories (like "internal/stacks") can inherit from it to minimize the divergence from the typical rules.	3 years ago
Martin Atkins	ea3b70abf0	stackeval: Component calls are now referenceable ... This means it's valid to write references like "component.foo" in expressions in a stack configuration, where the result is built from the planned output values of the root module of each component instance. For embedded stacks we intentionally designed the language to require explicit types on all output values so that we'd still be able to do type checking even when we can't calculate the output values yet. Unfortunately we can't do the same trick for components because the main Terraform module language treats output values as dynamically-typed and so an output value we can't evaluate yet could have literally any type. Therefore we concede and just return cty.DynamicVal in most failure cases here; perhaps a future edition of the main Terraform language can improve this by requiring explicitly-typed output values there too.	3 years ago
Martin Atkins	4a179ed969	stackeval: Very very basic component planning ... This is the absolute bare-minimum integration with the main Terraform language runtime to plan changes for a component. It's intentionally not yet very robust and limited only to the built-in Terraform provider because we're just trying to get the bones in place for seeing stack planning working, rather than trying to build something complete enough to ship. This currently only deals with the resource instance changes that get planned and doesn't expose the root module's output values for use elsewhere in the stack configuration. Output value handling will follow in subsequent commits.	3 years ago
Martin Atkins	d435cd5a1b	stackconfig: Bring the source bundle with the config ... When we load stackconfig we don't immediately load any of the modules that component blocks refer to, so we need to bring the source bundle along with the configuration so that a downstream piece of code (the stack language runtime, currently) can load those from the same source bundle.	3 years ago
Martin Atkins	d3e93e4a31	stackruntime: Initial stubbing of Plan ... This is just enough implementation of "Plan" for embedded stacks with their own input variables and output values, and exposing the final top-level output values as "planned changes". This still doesn't really do anything end-user-useful but it's helpful to have the expression evaluation and plan walk driver machinery in place before moving on to the main useful language element: component blocks. Support for components will follow in subsequent commits.	3 years ago
Martin Atkins	e76e203c77	stackeval: walkState with custom diagnostics handling ... In an earlier commit we added walkState as something we could concisely pass between all of the functions participating in a config/state walk such as what we do in main_validating.go for validation, dealing with the two cross-cutting concerns of firstly detecting when all of the asynchronous operations have completed and secondly aggregating together diagnostics produced by the various objects into a single set. The diagnostics part of this problem was previously solved by just always appending diagnostics into a hidden syncDiagnostics object. That works okay for validation where we just wait until all the work is complete and return all of the diagnostics together as a normal return value, but for planning we expect to be able to stream out both planned changes and diagnostics gradually as the (potentially-slow) planning process proceeds, so the UI can give gradual feedback and thus feel more responsive than just showing an indeterminite progress indicator for the entire operation. To preserve the idea of just passing a single object through the walk functions while giving a little more flexibility, here we introduce an extra hooking capability where the plan phase can treat diagnostics differently while still preserving the same external walkState API. In future commits the plan walk driver functions will all share the same walkState object and the entry point will populate it with the custom callbacks required to create a bridge to the channel-based API for emitting streaming diagnostics. The old simple API is still available for the validate walk to use. Instead of having the internal syncDiagnostics object directly inside the walkState object, it now lives in the local scope created by calling newWalkState, captured into the closures of the two callback functions. The effect is equivalent.	3 years ago
Martin Atkins	355bae32da	stackruntime: Allow hooking the start and end of the overall plan operation ... This broad pair of hooks allows the caller (i.e. rpcapi) to generate a parent tracing span around the entire plan operation, which all other traced operations will descend from. This is also in part just a way to show that the hook API is workable in practice, since these are the first actual hooks.	3 years ago
Martin Atkins	863a91359a	stackruntime: Plan calls into stackeval ... This nudges the placeholder "this is not a real plan" warning down one more level in the stack, with stackruntime.Plan now being equipped to propagate the individual planned changes and diagnostics up to the rpcapi caller, which will in turn propagate them to its client.	3 years ago
Martin Atkins	0111408ca8	stackruntime: Stub of "Plan" function ... This establishes the overall API shape but implementation-wise just moves the warning that the implementation is fake one level deeper, with stackruntime now being the one to return that warning instead of the rpcapi package directly. Package rpcapi now monitors the two output channels from stackruntime.Plan and translates what it sees into protobuf events to transmit to the client. Of course given the above that's currently always just exactly one diagnostic and no planned changes, but we'll start to make the stackruntime.Plan function real in subsequent commits.	3 years ago
Martin Atkins	8af46fd38b	stackruntime: An extensible "hooks" API ... Following a similar model that we've previously used in other parts of Terraform to expose UI-oriented progress events, here we define stackruntime.Hooks as a collection of optional callbacks a caller can use to subscribe to events, and the supporting utility code to help us concisely and correctly call those hook functions from the main evaluation code without adding too much extra noise to the code. Although the primary focus of this API is driving realtime UI updates, the API design also aims to accommodate distributed tracing use-cases by allowing callers to propagate arbitrary values between related event notifications and to inject additional values into the context.Context used for downstream operations that might not be under the direct supervision of stackruntime/stackeval.	3 years ago
Martin Atkins	2b7fad5bee	stackruntime: Our fake telemetry span must be passed by pointer ... Most of our Span actions don't mutate the object because we just immediately emit messages to the test log, but SetName directly modifies the span name so we need to pass by pointer so we can honor that rename.	3 years ago
Martin Atkins	03c981d9d0	promising,rpcapi,stackruntime: Quiet some staticcheck errors ... This code is still under development and so we have some stuff that's there to support further work despite not actually being used yet. We should reconsider these changes later on once the work here has settled.	3 years ago
Martin Atkins	791a5fb8a6	stackruntime: Top-level "Validate" function ... This is the public entry point for the overall validation of an entire stack configuration. So far it supports validation only of input variables, embedded stack calls, and output values. We'll grow this to support other language constructs in later commits. This is a relatively large commit due to introducing various supporting infrastructure to help make evaluation possible at all. Subsequent commits will hopefully be more focused due to being able to depend on the foundations introduced here.	3 years ago
Martin Atkins	8c7acd5378	stackruntime: Helper for viewing traces in tests ... Since our stackeval package is instrumented for OpenTelemetry tracing, it's helpful to be able to see the trace events as part of the test log when debugging. tracesToTestLog arranges for any spans to be announced into the main test log for the remainder of the current test, and then restores the original trace provider afterwards as a test cleanup action.	3 years ago
Martin Atkins	4754889950	stackruntime: Beginnings of Validate function ... This stubs out the basic structure of stackeval's equivalent of a graph walk, which is really just a recursive tree walk starting async tasks that might depend on each other via the abstractions in package "promising". This initial work is sufficient to validate the inputs to an embedded stack call when referring to input variables in the calling stack, showing that the expression evaluation model seems to be working and the use of promises to deal with dependency ordering seems viable.	3 years ago
Martin Atkins	fd4612b804	stackeval: An OpenTelemetry tracer ... This just introduces an OpenTelemetry tracer in a package-level variable but doesn't actually make use of it yet. We'll use it in subsequent commits to help with debugging how different evaluation tasks interact with one another.	3 years ago
Martin Atkins	c5930cc83d	stackeval: Ignore some unused stuff for now ... This package is currently just a stub for design exploration and discussion, so it intentionally has some unused code in it to help suggest how it might evolve in future. We'll remove these staticcheck exceptions later once we're ready to make this code "real", or possibly remove all this code altogether if we decide to take a different direction.	3 years ago
Martin Atkins	59a044dd6f	stackeval: The naming of promises ... If a stack configuration has a dependency cycle we'll tend to detect it by the promises failing to resolve due to a self-reference. Describing that situation to end-users has been historically hard in existing Terraform's dag-based implementation, and there are some similar challenges for this promise-based design too. To deal with it while minimizing runtime overhead in the happy path we'll teach the various objects to retroactively report names for the promises they've already instantiated and then only when we encounter the ErrSelfDependent error will we walk the tree of objects to find user-friendly names for all of the promises that have contributed to the results so far.	3 years ago
Martin Atkins	3d503f8e71	stackruntime: Stubbing the "interpreter" for stack configurations	3 years ago