terraform

Commit Graph

Author	SHA1	Message	Date
Austin Valle	a48e873790	PSS: Adjust `StateRange.End` logic in `ReadStateBytes` and `WriteStateBytes` (#38127 ) * fix usage of StateRange.End * add test for last chunk is single byte	3 months ago
Sarah French	a83b7402c2	PSS: Add tests showing `state` subcommands being used with PSS (#37891 ) * test: Add E2E tests for `state list` and `state show` commands * test: Update `mockPluggableStateStorageProvider` to log a warning during tests where the values in `MockStates` aren't compatible with the `ReadStateBytesFn` default function. Make existing test set an appropriate value in `MockStates`. * test: Update `mockPluggableStateStorageProvider` helper to include a resource schema * test: Add command-level test for `state list` showing integration with pluggable state storage code. * test: Add command-level test for `state show` showing integration with pluggable state storage code. * test: Add command-level test for `state pull` showing integration with pluggable state storage code. * test: Add command-level test for `state identities` showing integration with pluggable state storage code. * test: Add command-level test for `state rm` showing integration with pluggable state storage code. * test: Add command-level test for `state mv` showing integration with pluggable state storage code. * test: Add command-level test for `state push` showing integration with pluggable state storage code. * test: Add command-level test for `state replace-provider` showing integration with pluggable state storage code. * test: Change shared test fixture to not be named after a specific command under test. This test fixure is reused across tests that need the config to define a state store but otherwise rely on the mock provider to set up the test scenario. * test: Update test to use shared test fixture * test: Remove redundant test fixture The internal/command/testdata/state-commands-state-store and internal/command/testdata/state-store-unchanged test fixtures are the same. * fix: Re-add logic for setting chunk size in the context of E2E tests using grpcwrap package This was removed, incorrectly, in https://github.com/hashicorp/terraform/pull/37899 * refactor: Let panic happen if there's incompatibility between mock returned from `mockPluggableStateStorageProvider` and the `MockStates` that's been set in the mock * test: Refactor to contain paths in reused variable, remove unnecessary .gitkeep * test: Remove unneeded test code	4 months ago
Sarah French	cf047be4e4	PSS: Use interfaces to configure state stores (#37899 ) * refactor: Move chunk size limit constants to new `chunks` package * refactor: Make `NewPluggable` return a `Pluggable` concrete type, instead of an instance of the `backend.Backend` interface. * refactor: Configure state stores through the backend.Backend interface, instead of directly using methods related to RPCs. This requires changing where we call `SetStateStoreChunkSize`. * docs: Add godocs comment to `StateStoreChunkSizeSetter` interface To summarize, we don't really need the `SetStateStoreChunkSize` method, and instead methods like `(GRPCProvider).ConfigureStateStore` in the `plugin6` package can directly inspect the negotiation process that passes through that code and pull out the chunk size. However that means that that code would also need to implement validation. And that's just `(GRPCProvider).ConfigureStateStore`; what about all the test mocks that are used in different types of test? They would all need to be implemented similarly to GRPCProvider to be good, useful mocks, and then a lot of things that fulfil the provider.Interface interface are coupled to each other. Instead, it's easier to have validation in the `grpcClient` struct's methods in the `remote` package, as that code is common to all scenarios. That code needs a method to 'reach into' the provider.Interface value, so we use the `SetStateStoreChunkSize` method. * chore: Make it clearer that the v6 GRPCProvider implements `SetStateStoreChunkSize` * fix: Remove unnecessary assignment of chunk size I'm surprised that removing this doesn't break E2E tests of PSS that use grpcwrap, but I think there's `plugin6` code that runs in that situation, so maybe chunking is handled elsewhere. * chore: Add panic to try detect unexpected cases when setting chunk size. * feat: Add `providers.StateStoreChunkSizeSetter` implementation to provider-simple-v6 * docs: Update code comments for NewPluggable describing its intended use	5 months ago
Sarah French	f2818db795	PSS : Add `fs` and `inmem` state storage implementations to the builtin `simplev6` provider, update `grpcwrap` package, use PSS implementation in E2E test (#37790 ) * feat: Implement `inmem` state store in provider-simple-v6 * feat: Add filesystem state store `fs` in provider-simple-v6, no locking implemented * refactor: Move PSS chunking-related constants into the `pluggable` package, so they can be reused. * feat: Implement PSS-related methods in grpcwrap package * test: Add E2E test checking an init and apply (no plan) workflow is usable with both PSS implementations * fix: Ensure state stores are configured with a suggested chunk size from Core --------- Co-authored-by: Radek Simko <radeksimko@users.noreply.github.com>	5 months ago
Sarah French	2657a4daec	Set let require_unimplemented_servers default to true when generating code from .proto files (#37737 ) * Remove `require_unimplemented_servers=false` option when generating code from proto files. Run `make protobuf`. This allows require_unimplemented_servers to default to true, and the generated code shows the impacts of that. * Update generator script to embed 'UnimplementedFoobarServer' structs and skip generating code for the `mustEmbedUnimplementedFoobarServer` method that is implemented via that embedding. * Run `make generate` * Embed UnimplementedProviderServer into implementations of ProviderServer * Embed UnimplementedProvisionerServer into implementations of ProvisionerServer	6 months ago
Radek Simko	922fdb2382	Implement LockState & UnlockState provider methods (#37711 )	7 months ago
Radek Simko	51fc4c161f	PSS: Implement `ReadStateBytes` + `WriteStateBytes` (#37440 ) * Implement ReadStateBytes + WriteStateBytes * [WIP] - Testing ReadStateBytes and WriteStateBytes (#37464) * Fix nil pointer error * Add WIP test for ReadStateBytes * Move test mock to separate testing file * Update mock to send unexpected EOF when there's a problem returning data and it's not a true EOF * Add test case for when length != expected length * Add test for when trying to read state from a store type that doesn't exist * Change symbol names to lowercase * Add ability to force a diagnostic to be returned from `mockReadStateBytesClient`'s `Recv` method * Add test showing error diagnostics raised by the ReadStateBytes client are returned * Add missing header * Simplify mock by using an embedded type * Rename `mockOpts` to `mockReadStateBytesOpts` * Update existing tests to assert what arguments are passed to the RPC method call * Add mock WriteStateBytesClient which uses `go.uber.org/mock/gomock` to enable assertions about calls to Send * Add a test for WriteStateBytes that makes assertions about calls to the Send method * Update test case to explicitly test writing data smaller than the chunk size * Implement chunking in WriteStateBytes, add test case to assert expected chunking behaviour * Add generated mock for Provider_WriteStateBytesClient in protocol v6 * Update tests to use new `MockProvider_WriteStateBytesClient`, remove handwritten mock * Update code comments in test * Add tests for diagnostics and errors returned during WriteStateBytes * Add generated mock for Provider_ReadStateBytesClient in protocol v6, replace old mock * Add test case for grpc errors in ReadStateBytes, fix how error is returned * Typo in comment * Add missing warning test, rename some test cases * Update proto file definition of Read/WriteStateBytes RPCs (#37529) * Update Read/WriteStateBytes RPCs to match https://github.com/hashicorp/terraform-plugin-go/pull/531 * Run `make protobuf` * Run `make generate` * Update use of `proto.ReadStateBytes_ResponseChunk` in tests * Fix how diagnostics are handled alongside EOF error, update ReadStateBytes test * More fixes - test setup was incorrect I think? I assume that a response would be returned full of zero-values when EOF is encountered. * WIP - avoid crash if chunk is nil * Sarah's updates to radek/pss-read-write (#37642) * Update code to not expect a chunk when EOF encountered * Return early if any grpc errors are encountered during ReadStateBytes * Close the stream with CloseSend once everything's read without error. Add test case about handling grpc errors from CloseSend. * Fix test case about warnings: We would expect to receive a chunk with data alongside the warning and have a normal closing of the stream after EOF * Add log line, remove unneeded type info * Implement configurable state chunk size * handle metadata in WriteStateBytes correctly * validate chunk sizes received from provider * ReadStateBytes: avoid early return on warnings --------- Co-authored-by: Sarah French <15078782+SarahFrench@users.noreply.github.com>	7 months ago
Daniel Schmidt	871451122f	actions: move schema to single action type and remove linked resources we want to leave the door open to evolve the design in the future based on feedback	7 months ago
James Bardin	3948e19459	grpcwrap	8 months ago
Daniel Schmidt	176263d936	actions: remove future action types removing them for now, this gives us more flexibility in the future since the schema is not yet locked in	8 months ago
Daniel Schmidt	17e7338b0f	add client capabilites across the board	9 months ago
Kristin Laemmert	0374f04424	Add ValidateActionConfig to provider interface, protocol & internal implementations (#37345 ) * Add ValidateActionConfig to provider interface and protocol * Update internal/provider-simple-v6/provider.go Co-authored-by: Daniel Schmidt <danielmschmidt92@gmail.com> --------- Co-authored-by: Daniel Schmidt <danielmschmidt92@gmail.com>	9 months ago
Kristin Laemmert	9256074c43	Actions in plan/changes (#37320 ) * Add actions to the plans and change * jsonplan - ignoring LinkedResources for now, those are not in the MVP * pausing here: we'll work on the plan rendering later	9 months ago
Daniel Schmidt	d0e5c3cc7b	address review feedback	10 months ago
Daniel Schmidt	ac132d1b15	add provider implementation for actions	10 months ago
Samsondeen	870bf99593	List implementation for simple provider and e2e tests (#37297 )	10 months ago
Sarah French	0f8e77a50f	PSS: Add GetStates and DeleteState RPCs to plugin protocol v6 (#37288 ) * Add GetStates RPC method to Plugin Protocol v6 * Add DeleteState RPC method to Plugin Protocol v6 * Add methods to provider-simple * Fix error messages * Apply suggestions from code review Co-authored-by: Radek Simko <radeksimko@users.noreply.github.com> * Change how GetStates RPC returns the list of state names * Change GetStates implementation to use new data type * `make protobuf` to accommodate new code comments --------- Co-authored-by: Radek Simko <radeksimko@users.noreply.github.com>	10 months ago
Samsondeen	3d3439e00e	Testing list blocks with the gRPC Provider (#37236 )	10 months ago
Radek Simko	643266dc90	Add initial state store protocol skeleton (#37197 ) * Add initial state store protocol skeleton Introduce ValidateStateStoreConfig + ConfigureStateStore Update GetProviderSchema + GetMetadata (for mux) * address PR feedback	11 months ago
Daniel Banck	44ae7da18e	Update Plugin Protocol for List and Implement `ValidateListResourceConfig` (#37007 ) * Minor auto-formatting changes * Add list RPC and schema protobuf definitions * make protobuf * make generate * Add ValidateListResourceConfig implementation	11 months ago
Radek Simko	36fb64f297	grpcwrap+provider-simple: Test ephemeral resource schemas in v5 protocol (#36878 ) * grpcwrap+provider-simple: Test ephemeral resource schemas in v5 protocol * Update internal/command/e2etest/providers_schema_test.go Co-authored-by: Daniel Banck <dbanck@users.noreply.github.com> --------- Co-authored-by: Daniel Banck <dbanck@users.noreply.github.com>	1 year ago
Daniel Schmidt	fec6e4b552	send resource identities to provider calls	1 year ago
Daniel Banck	b2b42c0fb4	Store resource identities in state (TF-23255) (#36464 ) * Persist resource identity in Terraform state * make syncdeps * Move identity schema merging closer to the protocol * mock GetResourceIdentitySchemas * Fix identity refresh tests * Add more tests * Change grcpwrap upgrade identity * Review feedback * Remove unnecessary version conversion * Check if GetResourceIdentitySchemas RPC call is implemented * Update function signature docs * Adapt protocol changes * Check unimplemented error for identities in GetSchema	1 year ago
Daniel Banck	10c9b64007	Rename schema.Block to Body (#36629 )	1 year ago
Daniel Banck	857d188308	Add resource identity message to protocol (TF-23178) (#36451 ) * protocol: create new 5.9 and 6.9 protocol versions * protocol: add resource identity messages to 5.9 & 6.9 * make protobuf * make generate * grpcwrap: Add RPC calls skeleton	1 year ago
Daniel Schmidt	0ff1b77afa	standardize client capabilities	1 year ago
Daniel Schmidt	e29708c4d9	ephemeral: add WriteOnlyAttributesAllowed client capability to validate resource	1 year ago
Liam Cervante	eeacced8f0	Add go vet as a check to the Github commit actions (#31461 ) * Add go vet as a check to the Github commit actions * deliberately break to test * fix after successful test * Use directly * fix more errors	1 year ago
James Bardin	0c5fae9e3c	ephemeral e2e test provider	2 years ago
James Bardin	9c3dc8f73a	ephemeral protocol rename Add `Resource` to all the ephemeral protocol symbols to make them consistent with existing naming conventions.	2 years ago
James Bardin	1a72b39d42	fill out remaining ephemeral implementations Complete the remaining provider implementations to satisfy the updated internal interface as well as the new protobuf interfaces.	2 years ago
James Bardin	a8701f6ddd	provider functions return an error The call site for language functions doesn't currently have a way to handle complex diagnostics, so rather than appear to support them in the protocol we remove the concepts of diagnostics for now. We do however retain the argument index fields, which we can wrap in a function.ArgError and get a little more precise hcl diagnostic from expression.	2 years ago
Brian Flad	c7a44bfc9a	Support Private State with MoveResourceState (#34575 ) This change ensures that providers receive the source private state and can store potentially differing private state as part of a move, rather than always copying the source private state to the target. This functionality has not yet been released and therefore there are no compatibility concerns.	2 years ago
Liam Cervante	7d143388f3	Add support for the cross resource type move operation in the proto schema (#34480 ) * Update proto schema and provider interfaces with support for moved across resource type RPCs * address comments * remove unused functions * remove support for flatmap format	2 years ago
James Bardin	f9b93d13a2	update grpcwrap to handle provider functions	2 years ago
James Bardin	5c8fd0b147	add grpcwrap function methods to make it compile	2 years ago
Brian Flad	a7e3339e2d	internal/grpcwrap: Ensure GetProviderSchemaOptional server capability is passed through (#33742 ) Reference: `7094517089/internal/grpcwrap/provider6.go (L72)` The server capability is passed through with version 6, but not version 5.	3 years ago
Brian Flad	7094517089	Add GetMetadata RPC to provider protocol (#33739 ) Reference: https://github.com/hashicorp/terraform/pull/33486 This is a followup to the new provider server capability to make the `GetProviderSchema` RPC optional. While this server capability would perform its intended function when directly talking to a single provider server SDK implementation, provider servers using terraform-plugin-mux need a methodology for the mux server to determine the available resource types of each underlying provider server to properly route resource-specific RPCs. Since the only methodology available to the mux server would be calling the `GetProviderSchema` RPC to each of underlying provider servers, any memory optimization of core caching would be lost. The choice of adding a new RPC instead of adjusting the existing `GetProviderSchema` RPC with additional request information, such as "only list the type names and not the schema information in the response", is two-fold: - Prevents the introduction of conditional logic for the existing RPC. - Clearly delineates the purpose of the RPC and can be documented easier. The choice of adding this to the existing provider service is two-fold: - Implementing a separate protocol and/or service only on the provider side of the protocol would be a novel design change. This small of a change does not warrant the potential research and testing effort that would be associated with that implementation. - While the core implementation will not use the new RPC immediately, there is no reason why it should be restricted from doing so in the future if a valid use case surfaces. Other ecosystem tools, beyond terraform-plugin-mux, can also potentially benefit from the lightweight RPC now. This is changing the 5.4 and 6.4 protocol versions following the guidance of this comment in the definition files, since it directly relates to the prior intention of the new minor versions: ```protobuf // Note that only the proto files included in a release tag of Terraform are // official protocol releases. Proto files taken from other commits may include // incomplete changes or features that did not make it into a final release. // In all reasonable cases, plugin developers should take the proto file from // the tag of the most recent release of Terraform, and not from the main // branch or any other development branch. ``` As with any Protocol Buffers definition update, protocol compatibility is guaranteed within a major version, however generated protocol source code compatibility is not guaranteed. In this case, implementing the new RPC method in protocol wrapper types and the moving of the `ServerCapabilities` message to the top namespace are considered acceptable changes.	3 years ago
hashicorp-copywrite[bot]	53c34ff49c	Update copyright file headers to BUSL-1.1	3 years ago
hashicorp-copywrite[bot]	325d18262e	[COMPLIANCE] Add Copyright and License Headers	3 years ago
James Bardin	26c569e384	s/Capabilities/ServerCapabilities/	4 years ago
James Bardin	fd742cd544	add e2e test with provider schema capabilities enable destroy planning for the simple providers used in the e2e tests	4 years ago
Martin Atkins	b40a4fb741	Move plugin/ and plugin6/ to internal/plugin{,6}/ This is part of a general effort to move all of Terraform's non-library package surface under internal in order to reinforce that these are for internal use within Terraform only. If you were previously importing packages under this prefix into an external codebase, you could pin to an earlier release tag as an interim solution until you've make a plan to achieve the same functionality some other way.	5 years ago
Martin Atkins	31349a9c3a	Move configs/ to internal/configs/ This is part of a general effort to move all of Terraform's non-library package surface under internal in order to reinforce that these are for internal use within Terraform only. If you were previously importing packages under this prefix into an external codebase, you could pin to an earlier release tag as an interim solution until you've make a plan to achieve the same functionality some other way.	5 years ago
Martin Atkins	e81162c4e1	Move provisioners/ to internal/provisioners/ This is part of a general effort to move all of Terraform's non-library package surface under internal in order to reinforce that these are for internal use within Terraform only. If you were previously importing packages under this prefix into an external codebase, you could pin to an earlier release tag as an interim solution until you've make a plan to achieve the same functionality some other way.	5 years ago
Martin Atkins	1409f30f9c	Move providers/ to internal/providers/ This is part of a general effort to move all of Terraform's non-library package surface under internal in order to reinforce that these are for internal use within Terraform only. If you were previously importing packages under this prefix into an external codebase, you could pin to an earlier release tag as an interim solution until you've make a plan to achieve the same functionality some other way.	5 years ago
Martin Atkins	ec85fb1960	Move communicator/ to internal/communicator/ This is part of a general effort to move all of Terraform's non-library package surface under internal in order to reinforce that these are for internal use within Terraform only. If you were previously importing packages under this prefix into an external codebase, you could pin to an earlier release tag as an interim solution until you've make a plan to achieve the same functionality some other way.	5 years ago
Kristin Laemmert	ff05362d51	providers.Interface: rename ValidateDataSourceConfig to ValidateDataResourceConfig (#27874 ) * providers.Interface: rename ValidateDataSourceConfig to ValidateDataResourceConfig This PR came about after renaming ValidateResourceTypeConfig to ValidateResourceConfig: I now understand that we'd called it the former instead of the latter to indicate that the function wasn't necessarily operating on a resource that actually exists. A possibly-more-accurate renaming of both functions might then be ValidateManagedResourceConfig and ValidateDataResourceConfig. The next commit will update the protocol (v6 only) as well; these are in separate commits for reviewers and will get squashed together before merging. * extend renaming to protov6	5 years ago
Kristin Laemmert	7b7273e3aa	Add support for plugin protocol v6 (#27826 ) * Add support for plugin protocol v6 This PR turns on support for plugin protocol v6. A provider can advertise itself as supporting protocol version 6 and terraform will use the correct client. Todo: The "unmanaged" providers functionality does not support protocol version, so at the moment terraform will continue to assume that "unmanaged" providers are on protocol v5. This will require some upstream work on go-plugin (I believe). I would like to convert the builtin providers to use protocol v6 in a future PR; however it is not necessary until we remove protocol v6. * add e2e test for using both plugin protocol versions - copied grpcwrap and made a version that returns protocol v6 provider - copied the test provider, provider-simple, and made a version that's using protocol v6 with the above fun - added an e2etest	5 years ago
Kristin Laemmert	f6505870cc	Mildwonkey/providers interface renaming (#27805 ) * providers.Interface: huge renamification This commit renames a handful of functions in the providers.Interface to match changes made in protocol v6. The following commit implements this change across the rest of the codebase; I put this in a separate commit for ease of reviewing and will squash these together when merging. One noteworthy detail: protocol v6 removes the config from the ValidateProviderConfigResponse, since it's never been used. I chose to leave that in place in the interface until we deprecate support for protocol v5 entirely. Note that none of these changes impact current providers using protocol v5; the protocol is unchanged. Only the translation layer between the proto and terraform have changed.	5 years ago

1 2

52 Commits (27770ee8053da48fbb684fd23d415aae376d2bed)