33 Commits (3d211eb42ffcdc9d33d56bc2785d64b22f158c39)

Author	SHA1	Message	Date
Daniel Schmidt	a26fd2e81f	stacks: add deferrals to PlanResourceChange ... Co-authored-by: Mutahhir Hayat <mutahhir.hayat@hashicorp.com>	2 years ago
Mark DeCrane	e6f4ffa8a3	stacks: adding deferred action to datasource reads	2 years ago
Daniel Schmidt	b59edb4463	stacks: handle deferred actions in refresh ... Co-authored-by: Matej Risek <matej.risek@hashicorp.com>	2 years ago
Daniel Schmidt	5b8f77d2a4	stacks: add deferred actions to plugin protocol ... We are moving the deferred reason from plans/deferring to providers to have a single representation of deferred reason to be used from providers to the plan. Using the plan's representation causes circular dependencies. Co-authored-by: Matej Risek <matej.risek@hashicorp.com>	2 years ago
James Bardin	300f66b71d	Merge pull request #34603 from hashicorp/jbardin/remove-provider-funtion-warnings ... provider functions can only return an error	2 years ago
Martin Atkins	a77d351ea2	plugin+plugin6: Don't panic if ProviderMeta not populated ... "ProviderMeta" is a niche feature that's used by only two providers, but this code was written under the assumption that the caller would always provide it for any provider that has a schema for it, and unfortunately the Terraform SDK seems to always provide a meta schema even for providers that don't use it, and thus it's empty. The current phase of the "unknown_instances" language experiment is not fully wired in to the main logic as a way to reduce the risk of it impacting behavior for those not participating in the experiment, but that means that right now it doesn't actually have a resolved ProviderMeta value to present, and so was just omitting it on the assumption that it is always optional to provide anyway. As a pragmatic stopgap to resolve that conflict, this makes the gRPC provider dispatching logic tolerate an absent ProviderMeta value and synthesize a null value to use in that case. This makes that logic slightly more robust and also means we can defer doing all of the work to weave ProviderMeta into this experimental codepath. The new TODO comment in that codepath is intended to remind us to consider this again should we decide to stablize the experiment later; when we do that, we'll hopefully do it by refactoring to share more code between the two codepaths, at which point ProviderMeta will be easier to support.	2 years ago
James Bardin	a8701f6ddd	provider functions return an error ... The call site for language functions doesn't currently have a way to handle complex diagnostics, so rather than appear to support them in the protocol we remove the concepts of diagnostics for now. We do however retain the argument index fields, which we can wrap in a function.ArgError and get a little more precise hcl diagnostic from expression.	2 years ago
Brian Flad	c7a44bfc9a	Support Private State with MoveResourceState (#34575) ... This change ensures that providers receive the source private state and can store potentially differing private state as part of a move, rather than always copying the source private state to the target. This functionality has not yet been released and therefore there are no compatibility concerns.	2 years ago
Brian Flad	1298fcd412	Add missing function deprecation_message to JSON output and website documentation (#34520) ... Reference: https://github.com/hashicorp/terraform/pull/34450	2 years ago
Liam Cervante	7d143388f3	Add support for the cross resource type move operation in the proto schema (#34480) ... * Update proto schema and provider interfaces with support for moved across resource type RPCs * address comments * remove unused functions * remove support for flatmap format	2 years ago
James Bardin	2e74fec583	add optional Summary field to function json	2 years ago
James Bardin	e536ae9e07	round trip func decl tests	2 years ago
James Bardin	b0baae5d3e	add file headers	2 years ago
James Bardin	9abaf2f3b0	update provider code to match new rpc methods ... Add the types, methods, and basic framework to use provider defined functions.	2 years ago
James Bardin	9e164b5488	add mocks for new plugin protocol	2 years ago
Martin Atkins	2cc94b4e89	core: terraform.Context with preloaded provider schemas ... Loading schemas from some providers can be particularly expensive, since providers for large remote platforms tend to have very large schemas. Since provider schemas are needed for many operations in Terraform, callers sometimes end up loading schemas themselves anyway. Earlier work tried to mitigate this by introducing a global schema cache for all plugin-based providers, but that's problematic because it forces only a single implementation of each distinct provider source address across the entire lifetime of a process importing package providers. This does not remove that global cache yet, but does add a new capability that will hopefully eventually supplant it: callers of terraform.NewContext can provide a set of preloaded provider schemas which they must ensure would match what Terraform Core would find if it loaded the schemas from an instance of the same provider instantiated through the corresponding factory function given alongside. A caller that wishes to avoid the potential cost of multiple schema lookups can now therefore go look up the schemas itself before calling terraform.NewContext, and provide frozen schemas that we'll use instead of fetching from the associated plugins. As of this commit no callers are actually using this mechanism. The first caller will be the "stackeval" package, which already loads provider schemas in order to evaluate provider configuration blocks anyway and so should always be able to provide a full complement of preloaded schemas to avoid Terraform Core needing to do any further lookups itself.	2 years ago
James Bardin	f9713485a5	add mock tests for GetProviderSchemaOptional	3 years ago
James Bardin	f907df8a6f	correctly check for GetProviderSchemaOptional ... A typo put the check for `GetProviderSchemaOptional` into the wrong `if` expression, and was missed partly because of the named return which we also remove here.	3 years ago
Brian Flad	7094517089	Add GetMetadata RPC to provider protocol (#33739) ... Reference: https://github.com/hashicorp/terraform/pull/33486 This is a followup to the new provider server capability to make the `GetProviderSchema` RPC optional. While this server capability would perform its intended function when directly talking to a single provider server SDK implementation, provider servers using terraform-plugin-mux need a methodology for the mux server to determine the available resource types of each underlying provider server to properly route resource-specific RPCs. Since the only methodology available to the mux server would be calling the `GetProviderSchema` RPC to each of underlying provider servers, any memory optimization of core caching would be lost. The choice of adding a new RPC instead of adjusting the existing `GetProviderSchema` RPC with additional request information, such as "only list the type names and not the schema information in the response", is two-fold: - Prevents the introduction of conditional logic for the existing RPC. - Clearly delineates the purpose of the RPC and can be documented easier. The choice of adding this to the existing provider service is two-fold: - Implementing a separate protocol and/or service only on the provider side of the protocol would be a novel design change. This small of a change does not warrant the potential research and testing effort that would be associated with that implementation. - While the core implementation will not use the new RPC immediately, there is no reason why it should be restricted from doing so in the future if a valid use case surfaces. Other ecosystem tools, beyond terraform-plugin-mux, can also potentially benefit from the lightweight RPC now. This is changing the 5.4 and 6.4 protocol versions following the guidance of this comment in the definition files, since it directly relates to the prior intention of the new minor versions: ```protobuf // Note that only the proto files included in a release tag of Terraform are // official protocol releases. Proto files taken from other commits may include // incomplete changes or features that did not make it into a final release. // In all reasonable cases, plugin developers should take the proto file from // the tag of the most recent release of Terraform, and not from the main // branch or any other development branch. ``` As with any Protocol Buffers definition update, protocol compatibility is guaranteed within a major version, however generated protocol source code compatibility is not guaranteed. In this case, implementing the new RPC method in protocol wrapper types and the moving of the `ServerCapabilities` message to the top namespace are considered acceptable changes.	3 years ago
hashicorp-copywrite[bot]	53c34ff49c	Update copyright file headers to BUSL-1.1	3 years ago
James Bardin	e14b03f7b0	always set schema caches from provider clients ... Allow core to always use the global schema cache, so that providers without GetProviderSchemaOptional are not spun up repeatedly. Rather than conditionally setting the cache, we just conditionally use the cache in the client to work around providers without GetProviderSchemaOptional.	3 years ago
James Bardin	642904204a	enable global schema cache	3 years ago
James Bardin	68d86e81fb	disable cache for now	3 years ago
James Bardin	53901a7e62	Add basic global schema cache ... Add a single global schema cache for providers. This allows multiple provider instances to share a single copy of the schema, and prevents loading the schema multiple times for a given provider type during a single command. This does not currently work with some provider releases, which are using GetProviderSchema to trigger certain initializations. A new server capability will be introduced to trigger reloading their schemas, but not store duplicate results.	3 years ago
hashicorp-copywrite[bot]	325d18262e	[COMPLIANCE] Add Copyright and License Headers	3 years ago
James Bardin	26c569e384	s/Capabilities/ServerCapabilities/	4 years ago
James Bardin	6706d52832	check PlanDestroy capability in plugins ... This is most easily handled in the plugin code, without involving Terraform core. The biggest change here other than checking the PlanDestroy capability, is the removal of the schema helper methods in the plugins. With the addition of the capabilities field, combined with the necessity of checking diagnostics from the schema, the helpers have outlived their usefulness. Perhaps there's a better pattern for these repetitive calls, but for now there isn't too extra verbosity involved.	4 years ago
Brian Flad	0b404f4a95	Return early on GetProviderSchema RPC responses with error diagnostics ... Reference: https://github.com/hashicorp/terraform/issues/31047 Prevent potential panics and immediately return provider-defined errors diagnostics. Previously: ``` --- FAIL: TestGRPCProvider_GetSchema_ResponseErrorDiagnostic (0.00s) panic: runtime error: invalid memory address or nil pointer dereference [recovered] panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x70 pc=0x17fa752] goroutine 13 [running]: testing.tRunner.func1.2({0x191a100, 0x2236330}) /usr/local/Cellar/go/1.18.2/libexec/src/testing/testing.go:1389 +0x24e testing.tRunner.func1() /usr/local/Cellar/go/1.18.2/libexec/src/testing/testing.go:1392 +0x39f panic({0x191a100, 0x2236330}) /usr/local/Cellar/go/1.18.2/libexec/src/runtime/panic.go:838 +0x207 github.com/hashicorp/terraform/internal/plugin6/convert.ProtoToConfigSchema(0x0) /Users/bflad/src/github.com/hashicorp/terraform/internal/plugin6/convert/schema.go:110 +0x52 github.com/hashicorp/terraform/internal/plugin6/convert.ProtoToProviderSchema(...) /Users/bflad/src/github.com/hashicorp/terraform/internal/plugin6/convert/schema.go:98 github.com/hashicorp/terraform/internal/plugin6.(*GRPCProvider).GetProviderSchema(0xc00004a200) /Users/bflad/src/github.com/hashicorp/terraform/internal/plugin6/grpc_provider.go:152 +0x29a github.com/hashicorp/terraform/internal/plugin6.TestGRPCProvider_GetSchema_ResponseErrorDiagnostic(0x0?) /Users/bflad/src/github.com/hashicorp/terraform/internal/plugin6/grpc_provider_test.go:158 +0x265 testing.tRunner(0xc0001031e0, 0x1a733d8) /usr/local/Cellar/go/1.18.2/libexec/src/testing/testing.go:1439 +0x102 created by testing.(*T).Run /usr/local/Cellar/go/1.18.2/libexec/src/testing/testing.go:1486 +0x35f ``` Previously: ``` --- FAIL: TestGRPCProvider_GetSchema_ResponseErrorDiagnostic (0.00s) panic: runtime error: invalid memory address or nil pointer dereference [recovered] panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x70 pc=0x18a2732] goroutine 7 [running]: testing.tRunner.func1.2({0x1a5e720, 0x250be50}) /usr/local/Cellar/go/1.18.2/libexec/src/testing/testing.go:1389 +0x24e testing.tRunner.func1() /usr/local/Cellar/go/1.18.2/libexec/src/testing/testing.go:1392 +0x39f panic({0x1a5e720, 0x250be50}) /usr/local/Cellar/go/1.18.2/libexec/src/runtime/panic.go:838 +0x207 github.com/hashicorp/terraform/internal/plugin/convert.ProtoToConfigSchema(0x0) /Users/bflad/src/github.com/hashicorp/terraform/internal/plugin/convert/schema.go:104 +0x52 github.com/hashicorp/terraform/internal/plugin/convert.ProtoToProviderSchema(...) /Users/bflad/src/github.com/hashicorp/terraform/internal/plugin/convert/schema.go:92 github.com/hashicorp/terraform/internal/plugin.(*GRPCProvider).GetProviderSchema(0xc00004a600) /Users/bflad/src/github.com/hashicorp/terraform/internal/plugin/grpc_provider.go:149 +0x29a github.com/hashicorp/terraform/internal/plugin.TestGRPCProvider_GetSchema_ResponseErrorDiagnostic(0x0?) /Users/bflad/src/github.com/hashicorp/terraform/internal/plugin/grpc_provider_test.go:130 +0x265 testing.tRunner(0xc0001031e0, 0x1be9500) /usr/local/Cellar/go/1.18.2/libexec/src/testing/testing.go:1439 +0x102 created by testing.(*T).Run /usr/local/Cellar/go/1.18.2/libexec/src/testing/testing.go:1486 +0x35f ```	4 years ago
James Bardin	8943c79322	diagnostics must be checked on all schema calls ... We can no longer be assured that the particular instance of a provider we are using has had GetProviderSchema called. Always check the diagnostics even if we're fetching a cached response.	4 years ago
Chris Arcand	369264865e	internal/plugin[6]: Add generated mocks	4 years ago
Martin Atkins	965c0f3f91	build: Run staticcheck with "go run" ... Running the tool this way ensures that we'll always run the version selected by our go.mod file, rather than whatever happened to be available in $GOPATH/bin on the system where we're running this. This change caused some contexts to now be using a newer version of staticcheck with additional checks, and so this commit also includes some changes to quiet the new warnings without any change in overall behavior.	4 years ago
Martin Atkins	36d0a50427	Move terraform/ to internal/terraform/ ... This is part of a general effort to move all of Terraform's non-library package surface under internal in order to reinforce that these are for internal use within Terraform only. If you were previously importing packages under this prefix into an external codebase, you could pin to an earlier release tag as an interim solution until you've make a plan to achieve the same functionality some other way.	5 years ago
Martin Atkins	b40a4fb741	Move plugin/ and plugin6/ to internal/plugin{,6}/ ... This is part of a general effort to move all of Terraform's non-library package surface under internal in order to reinforce that these are for internal use within Terraform only. If you were previously importing packages under this prefix into an external codebase, you could pin to an earlier release tag as an interim solution until you've make a plan to achieve the same functionality some other way.	5 years ago