19 Commits (5c8fd0b147ed36bf8de08047db232ac2d1ac80d1)

Author	SHA1	Message	Date
James Bardin	9e164b5488	add mocks for new plugin protocol	2 years ago
Martin Atkins	2cc94b4e89	core: terraform.Context with preloaded provider schemas ... Loading schemas from some providers can be particularly expensive, since providers for large remote platforms tend to have very large schemas. Since provider schemas are needed for many operations in Terraform, callers sometimes end up loading schemas themselves anyway. Earlier work tried to mitigate this by introducing a global schema cache for all plugin-based providers, but that's problematic because it forces only a single implementation of each distinct provider source address across the entire lifetime of a process importing package providers. This does not remove that global cache yet, but does add a new capability that will hopefully eventually supplant it: callers of terraform.NewContext can provide a set of preloaded provider schemas which they must ensure would match what Terraform Core would find if it loaded the schemas from an instance of the same provider instantiated through the corresponding factory function given alongside. A caller that wishes to avoid the potential cost of multiple schema lookups can now therefore go look up the schemas itself before calling terraform.NewContext, and provide frozen schemas that we'll use instead of fetching from the associated plugins. As of this commit no callers are actually using this mechanism. The first caller will be the "stackeval" package, which already loads provider schemas in order to evaluate provider configuration blocks anyway and so should always be able to provide a full complement of preloaded schemas to avoid Terraform Core needing to do any further lookups itself.	2 years ago
James Bardin	f9713485a5	add mock tests for GetProviderSchemaOptional	3 years ago
James Bardin	f907df8a6f	correctly check for GetProviderSchemaOptional ... A typo put the check for `GetProviderSchemaOptional` into the wrong `if` expression, and was missed partly because of the named return which we also remove here.	3 years ago
Brian Flad	7094517089	Add GetMetadata RPC to provider protocol (#33739) ... Reference: https://github.com/hashicorp/terraform/pull/33486 This is a followup to the new provider server capability to make the `GetProviderSchema` RPC optional. While this server capability would perform its intended function when directly talking to a single provider server SDK implementation, provider servers using terraform-plugin-mux need a methodology for the mux server to determine the available resource types of each underlying provider server to properly route resource-specific RPCs. Since the only methodology available to the mux server would be calling the `GetProviderSchema` RPC to each of underlying provider servers, any memory optimization of core caching would be lost. The choice of adding a new RPC instead of adjusting the existing `GetProviderSchema` RPC with additional request information, such as "only list the type names and not the schema information in the response", is two-fold: - Prevents the introduction of conditional logic for the existing RPC. - Clearly delineates the purpose of the RPC and can be documented easier. The choice of adding this to the existing provider service is two-fold: - Implementing a separate protocol and/or service only on the provider side of the protocol would be a novel design change. This small of a change does not warrant the potential research and testing effort that would be associated with that implementation. - While the core implementation will not use the new RPC immediately, there is no reason why it should be restricted from doing so in the future if a valid use case surfaces. Other ecosystem tools, beyond terraform-plugin-mux, can also potentially benefit from the lightweight RPC now. This is changing the 5.4 and 6.4 protocol versions following the guidance of this comment in the definition files, since it directly relates to the prior intention of the new minor versions: ```protobuf // Note that only the proto files included in a release tag of Terraform are // official protocol releases. Proto files taken from other commits may include // incomplete changes or features that did not make it into a final release. // In all reasonable cases, plugin developers should take the proto file from // the tag of the most recent release of Terraform, and not from the main // branch or any other development branch. ``` As with any Protocol Buffers definition update, protocol compatibility is guaranteed within a major version, however generated protocol source code compatibility is not guaranteed. In this case, implementing the new RPC method in protocol wrapper types and the moving of the `ServerCapabilities` message to the top namespace are considered acceptable changes.	3 years ago
hashicorp-copywrite[bot]	53c34ff49c	Update copyright file headers to BUSL-1.1	3 years ago
James Bardin	e14b03f7b0	always set schema caches from provider clients ... Allow core to always use the global schema cache, so that providers without GetProviderSchemaOptional are not spun up repeatedly. Rather than conditionally setting the cache, we just conditionally use the cache in the client to work around providers without GetProviderSchemaOptional.	3 years ago
James Bardin	642904204a	enable global schema cache	3 years ago
James Bardin	68d86e81fb	disable cache for now	3 years ago
James Bardin	53901a7e62	Add basic global schema cache ... Add a single global schema cache for providers. This allows multiple provider instances to share a single copy of the schema, and prevents loading the schema multiple times for a given provider type during a single command. This does not currently work with some provider releases, which are using GetProviderSchema to trigger certain initializations. A new server capability will be introduced to trigger reloading their schemas, but not store duplicate results.	3 years ago
hashicorp-copywrite[bot]	325d18262e	[COMPLIANCE] Add Copyright and License Headers	3 years ago
James Bardin	26c569e384	s/Capabilities/ServerCapabilities/	4 years ago
James Bardin	6706d52832	check PlanDestroy capability in plugins ... This is most easily handled in the plugin code, without involving Terraform core. The biggest change here other than checking the PlanDestroy capability, is the removal of the schema helper methods in the plugins. With the addition of the capabilities field, combined with the necessity of checking diagnostics from the schema, the helpers have outlived their usefulness. Perhaps there's a better pattern for these repetitive calls, but for now there isn't too extra verbosity involved.	4 years ago
Brian Flad	0b404f4a95	Return early on GetProviderSchema RPC responses with error diagnostics ... Reference: https://github.com/hashicorp/terraform/issues/31047 Prevent potential panics and immediately return provider-defined errors diagnostics. Previously: ``` --- FAIL: TestGRPCProvider_GetSchema_ResponseErrorDiagnostic (0.00s) panic: runtime error: invalid memory address or nil pointer dereference [recovered] panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x70 pc=0x17fa752] goroutine 13 [running]: testing.tRunner.func1.2({0x191a100, 0x2236330}) /usr/local/Cellar/go/1.18.2/libexec/src/testing/testing.go:1389 +0x24e testing.tRunner.func1() /usr/local/Cellar/go/1.18.2/libexec/src/testing/testing.go:1392 +0x39f panic({0x191a100, 0x2236330}) /usr/local/Cellar/go/1.18.2/libexec/src/runtime/panic.go:838 +0x207 github.com/hashicorp/terraform/internal/plugin6/convert.ProtoToConfigSchema(0x0) /Users/bflad/src/github.com/hashicorp/terraform/internal/plugin6/convert/schema.go:110 +0x52 github.com/hashicorp/terraform/internal/plugin6/convert.ProtoToProviderSchema(...) /Users/bflad/src/github.com/hashicorp/terraform/internal/plugin6/convert/schema.go:98 github.com/hashicorp/terraform/internal/plugin6.(*GRPCProvider).GetProviderSchema(0xc00004a200) /Users/bflad/src/github.com/hashicorp/terraform/internal/plugin6/grpc_provider.go:152 +0x29a github.com/hashicorp/terraform/internal/plugin6.TestGRPCProvider_GetSchema_ResponseErrorDiagnostic(0x0?) /Users/bflad/src/github.com/hashicorp/terraform/internal/plugin6/grpc_provider_test.go:158 +0x265 testing.tRunner(0xc0001031e0, 0x1a733d8) /usr/local/Cellar/go/1.18.2/libexec/src/testing/testing.go:1439 +0x102 created by testing.(*T).Run /usr/local/Cellar/go/1.18.2/libexec/src/testing/testing.go:1486 +0x35f ``` Previously: ``` --- FAIL: TestGRPCProvider_GetSchema_ResponseErrorDiagnostic (0.00s) panic: runtime error: invalid memory address or nil pointer dereference [recovered] panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x70 pc=0x18a2732] goroutine 7 [running]: testing.tRunner.func1.2({0x1a5e720, 0x250be50}) /usr/local/Cellar/go/1.18.2/libexec/src/testing/testing.go:1389 +0x24e testing.tRunner.func1() /usr/local/Cellar/go/1.18.2/libexec/src/testing/testing.go:1392 +0x39f panic({0x1a5e720, 0x250be50}) /usr/local/Cellar/go/1.18.2/libexec/src/runtime/panic.go:838 +0x207 github.com/hashicorp/terraform/internal/plugin/convert.ProtoToConfigSchema(0x0) /Users/bflad/src/github.com/hashicorp/terraform/internal/plugin/convert/schema.go:104 +0x52 github.com/hashicorp/terraform/internal/plugin/convert.ProtoToProviderSchema(...) /Users/bflad/src/github.com/hashicorp/terraform/internal/plugin/convert/schema.go:92 github.com/hashicorp/terraform/internal/plugin.(*GRPCProvider).GetProviderSchema(0xc00004a600) /Users/bflad/src/github.com/hashicorp/terraform/internal/plugin/grpc_provider.go:149 +0x29a github.com/hashicorp/terraform/internal/plugin.TestGRPCProvider_GetSchema_ResponseErrorDiagnostic(0x0?) /Users/bflad/src/github.com/hashicorp/terraform/internal/plugin/grpc_provider_test.go:130 +0x265 testing.tRunner(0xc0001031e0, 0x1be9500) /usr/local/Cellar/go/1.18.2/libexec/src/testing/testing.go:1439 +0x102 created by testing.(*T).Run /usr/local/Cellar/go/1.18.2/libexec/src/testing/testing.go:1486 +0x35f ```	4 years ago
James Bardin	8943c79322	diagnostics must be checked on all schema calls ... We can no longer be assured that the particular instance of a provider we are using has had GetProviderSchema called. Always check the diagnostics even if we're fetching a cached response.	4 years ago
Chris Arcand	369264865e	internal/plugin[6]: Add generated mocks	4 years ago
Martin Atkins	965c0f3f91	build: Run staticcheck with "go run" ... Running the tool this way ensures that we'll always run the version selected by our go.mod file, rather than whatever happened to be available in $GOPATH/bin on the system where we're running this. This change caused some contexts to now be using a newer version of staticcheck with additional checks, and so this commit also includes some changes to quiet the new warnings without any change in overall behavior.	4 years ago
Martin Atkins	36d0a50427	Move terraform/ to internal/terraform/ ... This is part of a general effort to move all of Terraform's non-library package surface under internal in order to reinforce that these are for internal use within Terraform only. If you were previously importing packages under this prefix into an external codebase, you could pin to an earlier release tag as an interim solution until you've make a plan to achieve the same functionality some other way.	5 years ago
Martin Atkins	b40a4fb741	Move plugin/ and plugin6/ to internal/plugin{,6}/ ... This is part of a general effort to move all of Terraform's non-library package surface under internal in order to reinforce that these are for internal use within Terraform only. If you were previously importing packages under this prefix into an external codebase, you could pin to an earlier release tag as an interim solution until you've make a plan to achieve the same functionality some other way.	5 years ago