156 Commits (394cf7f25eddcaeddf45c581ca64d71135ab887b)

Author	SHA1	Message	Date
Martin Atkins	eed605ac05	[WIP] Re-enable the end-to-end tests (#20044) ... * internal/initwd: Allow deprecated relative module paths In Terraform 0.11 we deprecated this form but didn't have any explicit warning for it. Now we'll still accept it but generate a warning. In a future major release we will drop this form altogether, since it is ambiguous with registry module source addresses. This codepath is covered by the command/e2etest suite. * e2e: Skip copying .exists file, if present We use this only in the "empty" test fixture in order to let git know that the directory exists. We need to skip copying it so that we can test "terraform init -from-module=...", which expects to find an empty directory. * command/e2etests: Re-enable and fix up the e2etest "acctests" We disabled all of the tests that accessed remote services like the Terraform Registry while they were being updated to support the new protocols we now expect. With those services now in place, we can re-enable these tests. Some details of exactly what output we print, etc, have intentionally changed since these tests were last updated. * e2e: refactor for modern states and plans * command/e2etest: re-enable e2etests and update for tf 0.12 compatibility plugin/discovery: mkdirAll instead of mkdir when creating cache dir	7 years ago
Radek Simko	b171997b39	plugin/discovery: Fix quoting of provider name	7 years ago
Martin Atkins	88e76fa9ef	configs/configschema: Introduce the NestingGroup mode for blocks ... In study of existing providers we've found a pattern we werent previously accounting for of using a nested block type to represent a group of arguments that relate to a particular feature that is always enabled but where it improves configuration readability to group all of its settings together in a nested block. The existing NestingSingle was not a good fit for this because it is designed under the assumption that the presence or absence of the block has some significance in enabling or disabling the relevant feature, and so for these always-active cases we'd generate a misleading plan where the settings for the feature appear totally absent, rather than showing the default values that will be selected. NestingGroup is, therefore, a slight variation of NestingSingle where presence vs. absence of the block is not distinguishable (it's never null) and instead its contents are treated as unset when the block is absent. This then in turn causes any default values associated with the nested arguments to be honored and displayed in the plan whenever the block is not explicitly configured. The current SDK cannot activate this mode, but that's okay because its "legacy type system" opt-out flag allows it to force a block to be processed in this way anyway. We're adding this now so that we can introduce the feature in a future SDK without causing a breaking change to the protocol, since the set of possible block nesting modes is not extensible.	7 years ago
James Bardin	7df3275120	Merge pull request #20957 from hashicorp/jbardin/shim-schemas ... New methods for obtaining CoreConfigSchema for shimming	7 years ago
James Bardin	a3d58665ad	use LegacyResourceSchema ... rather than the previous .CoreConfigSchemaForShimming	7 years ago
Pam Selle	346e341ff2	Only display status link if public registry used ... Updates to throw a specific error if using the defined public registry vs. another registry.	7 years ago
James Bardin	2ebc2beda0	add 3rd param to mock call	7 years ago
James Bardin	746aac8bda	increase grpc recv limit ... Some providers may generate quite large schemas, and the internal default grpc response size limit is 4MB. 64MB should cover most any use case, and if we get providers nearing that we may want to consider a finer-grained API to fetch individual resource schemas.	7 years ago
Pam Selle	d72456d188	Add friendly error for when registry unresponsive ... If the registry is unresponsive, you will now get an error specific to this, rather than a misleading "provider unavailable" type error. Also adds debug logging for when errors like this may occur	7 years ago
Justin Downing	1e32ae243c	grammatical updates to comments and docs (#20195)	7 years ago
findkim	161fe47b34	plugin/discovery: improve providery discovery verification errors	7 years ago
findkim	2726282307	plugin/discovery: add context for provider discovery verification	7 years ago
findkim	1a32617d5e	plugin/discovery: verify checksum matches Registry response	7 years ago
Justin Campbell	e6316c9de6	plugin/discovery: Parse warnings from TF Registry ... Terraform Registry (and other registry implementations) can now return an array of warnings with the versions response. These warnings are now displayed to the user during a `terraform init`.	7 years ago
Justin Campbell	24e13d8ec1	plugin/discovery: Return tfdiags from Get ... Allows us to surface warnings to the user using the tfdiags interfaces.	7 years ago
Sander van Harmelen	973e2a7cf9	core: add a context to the UIInput interface	7 years ago
Kristin Laemmert	9402b86217	plugin/discovery: create target plugin directory if it does not exist (#20575) ... If the user supplies a plugin cache dir, the provider installer was not checking if target directory exists before linking the cached pluging.	7 years ago
Justin Campbell	0574eadb62	plugin/discovery: Verify sig against HashiCorp key ... https://github.com/hashicorp/terraform/pull/19389 introduced a change to the provider GPG signature verification process, and removed the hardcoded HashiCorp GPG key. While the changes were intended and are still planned for a future release, we should still be verifying all providers in the TF 0.12.0 release against the HashiCorp GPG key until a more robust key verification procedure is in place. Fixes https://github.com/hashicorp/terraform/issues/20527	7 years ago
Martin Atkins	31299e688d	core: Allow legacy SDK to opt out of plan-time safety checks ... Due to the inprecision of our shimming from the legacy SDK type system to the new Terraform Core type system, the legacy SDK produces a number of inconsistencies that produce only minor quirky behavior or broken edge-cases. To retain compatibility with those existing weird behaviors, the legacy SDK opts out of our safety checks. The intent here is to allow existing providers to continue to do their previous unsafe behaviors for now, accepting that this will allow certain quirky bugs from previous releases to persist, and then gradually migrate away from the legacy SDK and remove this opt-out on a per-resource basis over time. As with the apply-time safety check opt-out, this is reserved only for the legacy SDK and must not be used in any new SDK implementations. We still include any inconsistencies as warnings in the logs as an aid to anyone debugging weird behavior, so that they can see situations where blame may be misplaced in the user-visible error messages.	7 years ago
Martin Atkins	1530fe52f7	core: Legacy SDK providers opt out of our new apply result check ... The shim layer for the legacy SDK type system is not precise enough to guarantee it will produce identical results between plan and apply. In particular, values that are null during plan will often become zero-valued during apply. To avoid breaking those existing providers while still allowing us to introduce this check in the future, we'll introduce a rather-hacky new flag that allows the legacy SDK to signal that it is the legacy SDK and thus disable the check. Once we start phasing out the legacy SDK in favor of one that natively understands our new type system, we can stop setting this flag and thus get the additional safety of this check without breaking any previously-released providers. No other SDK is permitted to set this flag, and we will remove it if we ever introduce protocol version 6 in future, assuming that any provider supporting that protocol will always produce consistent results.	7 years ago
Martin Atkins	a9274beaca	build: Run "go generate" in modules mode ... It seems that all of the tools we run here are now sufficiently modules-aware to run without problems in modules mode, and indeed running _not_ in modules mode was causing problems with locating packages in mockgen.	7 years ago
Martin Atkins	bdcac8792d	plugin: Use correct schema when marshaling imported resource objects ... Previously we were using the type name requested in the import to select the schema, but a provider is free to return additional objects of other types as part of an import result, and so it's important that we perform schema selection separately for each returned object. If we don't do this, we get confusing downstream errors where the resulting object decodes to the wrong type and breaks various invariants expected by Terraform Core. The testResourceImportOther test in the test provider didn't catch this previously because it happened to have an identical schema to the other resource type being imported. Now the schema is changed and also there's a computed attribute we can set as part of the refresh phase to make sure we're completing the Read call properly during import. Refresh was working correctly, but we didn't have any tests for it as part of the import flow.	7 years ago
findkim	7816e61614	Bump installer protocol version to 5 and separate client and server protocol references	7 years ago
findkim	abeb86072b	Remove duplicate logging	7 years ago
findkim	6e0de3e3f5	Nest incompatible provider protocol error to include dynamic custom msg	7 years ago
findkim	f75d83c1cd	Prerelease versions are filtered for plugin protocol mismatches	7 years ago
findkim	368ac85a26	Add provider protocol compatibility UI err msg during registry discovery	7 years ago
findkim	43b22d4e14	Fix plugin protocol test	7 years ago
James Bardin	6f54bfaa7c	send config during apply	7 years ago
James Bardin	e706f34794	generate new plugin proto mocks	7 years ago
Martin Atkins	53926ea581	plugin: Fix incorrect trace log message in provider Close ... Was incorrectly printing out "PlanResourceChange" instead of "Close".	7 years ago
James Bardin	1d4c8403a2	enable Auto mTLS in go-plugin	7 years ago
James Bardin	924b97238f	Handle StateFuncs in provider shim ... Any state modifying functions can only be run once during the plan-apply cycle. When regenerating the Diff during ApplyResourceChange, strip out all StateFunc and CustomizeDiff functions from the schema. Thew NewExtra diff field was where config data that was modified by a StateFunc was stored, and needs to be maintained between plan and apply. During PlanResourceChange, store any NewExtra data from the Diff in the PlannedPrivate data, and re-insert the NewExtra data into the Diff generated during ApplyResourceChange.	7 years ago
Martin Atkins	72e279e6b2	providers: Consistently use int64 for schema versions ... The rest of Terraform is still using uint64 for this in various spots, but we'll update that gradually later. We use int64 here because that matches what's used in our protobuf definition, and unsigned integers are not portable across all of the protobuf target languages anyway.	7 years ago
findkim	5e06e39fcc	Use registry alias to fetch providers	7 years ago
Justin Campbell	06825bf46d	plugin/discovery: Add friendly gpg err msg ... When GPG verification fails, display a helpful message to the user instead of the generic openpgp error.	7 years ago
Justin Campbell	9a8a74b9bb	plugin/discovery: Print name before verification ... This is so that any errors output from the checksum/signature verification show up in the expected place in the output.	7 years ago
Justin Campbell	495826444b	plugin/discovery: Use GPG keys from Registry ... When verifying the signature of the SHA256SUMS file, we have been hardcoding HashiCorp's public GPG key and using it as the keyring. Going forward, Terraform will get a list of valid public keys for a provider from the Terraform Registry (registry.terraform.io), and use them as the keyring for the openpgp verification func.	7 years ago
Martin Atkins	4fe9632f09	plugin: Establish our current plugin protocol as version 5 ... The main significant change here is that the package name for the proto definition is "tfplugin5", which is important because this name is part of the wire protocol for references to types defined in our package. Along with that, we also move the generated package into "internal" to make it explicit that importing the generated Go package from elsewhere is not the right approach for externally-implemented SDKs, which should instead vendor the proto definition they are using and generate their own stubs to ensure that the wire protocol is the only hard dependency between Terraform Core and plugins. After this is merged, any provider binaries built against our helper/schema package will need to be rebuilt so that they use the new "tfplugin5" package name instead of "proto". In a future commit we will include more elaborate and organized documentation on how an external codebase might make use of our RPC interface definition to implement an SDK, but the primary concern here is to ensure we have the right wire package name before release.	7 years ago
Martin Atkins	ec57927ea3	build: Take protoc out of the "go generate" path ... Since protoc is not go-gettable, and most development tasks in Terraform won't involve recompiling protoc files anyway, we'll use a separate mechanism for these. This way "go generate" only depends on things we can "go get" in the "make tools" target. In a later commit we should also in some way specify a particular version of protoc to use so that we don't get "flapping" regenerations as developers work with different versions, but the priority here is just to make "make generate" minimally usable again to restore the dev workflow documented in the README. This also includes some updates that resulted from running "make generate" and "make protobuf" after those Makefile changes were in place.	7 years ago
James Bardin	3bdd3b2a6e	update plugin client with PrepareProviderConfig	7 years ago
James Bardin	b173037eb3	update mock for PrepareProviderConfig	7 years ago
James Bardin	ba472c7059	Update proto with new PrepareProviderConfig method	7 years ago
James Bardin	3a0c2f1b67	close grpc TestListener	7 years ago
Martin Atkins	e25f79ed28	plugin/convert: Show approximate location context for all provider errors ... Even if a provider doesn't indicate a specific attribute as the cause of a resource operation error, we know the error relates to some aspect of the resource, so we'll include that approximate information in the result so that we don't produce user-hostile error messages with no context whatsoever. Later we can hopefully refine this to place the source range on the header of the configuration block rather than on an empty part of the body, but that'll require some more complex rework here and so for now we'll just accept this as an interim state so that the user can at least figure out which resource block the error is coming from.	7 years ago
Martin Atkins	3eaf2c308d	plugin/discovery: Use mockos_mockarch when testing installation ... If we don't override these then the tests can only pass on one platform.	7 years ago
James Bardin	0d4d572c39	start work on helper/resource test fixtures ... The helper resource tests won't pass for now, as they use a terraform.MockProvider which can't be used in the schema.Provider shims.	7 years ago
James Bardin	52c0032aed	update provisioners for multiple processes ... The "internal" provisioners are still run in a separate process, and need to be updated to restart on each walk.	7 years ago
James Bardin	a14fe7ae0c	missing diagnostics in pugin protocol ... Some call diagnostics were being missed	7 years ago
James Bardin	795161f615	update to start a new process for each plugin ... Modify the plugin factories to create a new plugin process for each individual plugin.	7 years ago