563 Commits (e024960c745f0a464cb081d04765d93f94deaba0)

Author	SHA1	Message	Date
Martin Atkins	135121562e	helper/plugin: Implement Schema.SkipCoreTypeCheck ... The previous commit added this flag but did not implement it. Here we implement it by adjusting the shape of schema we return to Terraform Core to mark the attribute as untyped and then ensure that gets handled correctly on the SDK side.	7 years ago
Martin Atkins	7f860dc83e	helper/schema: Schema.SkipCoreTypeCheck flag ... When running in v0.12-and-higher mode, this will cause the SDK to report the type of the attribute as "any", effectively skipping type checking on the Core side altogether and checking only in the SDK and provider code. The practical impact of this is to restore the v0.11-style checking behavior of allowing object values to be missing certain attributes as long as they are marked as optional in the schema. The SDK can do this because it uses a unified schema model for both object values and nested blocks, while Terraform Core only supports the idea of "optional" when talking about attributes in nested blocks. This is a continuation of the pile of workarounds that also includes the ConfigMode and AsSingle fields, allowing providers to selectively opt out of new v0.12 behaviors in situations where they conflict with decisions made in the design of the providers in our old world where Terraform Core delegated _all_ validation to providers. This is designed as an opt-in so that we can limit its impact only to specific cases where it's needed and minimize the risk of regressions elsewhere. Providers should use this sparingly only in situations where prevailing usage disagrees with the new expectations of Terraform Core in v0.12. This commit only adds the flag, and does not implement any behavior for it yet. That means this commit can exist in both the v0.11 and v0.12 codebases, allowing for API compatibility. A subsequent commit for v0.12 (not included in v0.11) will then implement this behavior.	7 years ago
Justin Downing	1e32ae243c	grammatical updates to comments and docs (#20195)	7 years ago
Paul Tyng	ea7e922007	oxford comma	7 years ago
Paul Tyng	ec9450a262	Fix limitations on Elem for TypeMap	7 years ago
Martin Atkins	1987a92386	helper/schema: Implementation of the AsSingle mechanism ... The previous commit added a new flag to schema.Schema which is documented to make a list with MaxItems: 1 be presented to Terraform Core as a single value instead, giving a way to switch to non-list nested resources without it being a breaking change for Terraform v0.11 users as long as it's done prior to a provider's first v0.12-compatible release. This is the implementation of that mechanism. It's intentionally implemented as a suite of extra fixups rather than direct modifications to existing shim code because we want to ensure that this has no effect whatsoever on the result of a resource type that _isn't_ using AsSingle. Although there is some small unit test coverage of the fixup steps here, the primary testing for this is in the test provider since the integration of all of these fixup steps in the correct order is the more important result than any of the intermediate fixup steps.	7 years ago
Martin Atkins	1c8150428f	helper/schema: Schema.AsSingle flag ... This setting indicates that an attribute defined as TypeList or TypeSet should be presented to Terraform Core as a single value instead when running in Terraform v0.12 or later. It has no effect for Terraform v0.10 or v0.11. This commit just introduces the setting without any associated behavior, so it can be included in both the v0.12 and v0.11 branches. A subsequent commit only to the v0.12 branch will introduce the behavior as part of the protocol version 5 shims.	7 years ago
Martin Atkins	4de0b33097	helper/schema: Honor ConfigMode when building core schema ... This makes some slight adjustments to the shape of the schema we present to Terraform Core without affecting how it is consumed by the SDK and thus the provider. This mechanism is designed specifically to avoid changing how the schema is interpreted by the SDK itself or by the provider, so that prior behavior can be preserved in Terraform v0.11 mode. This also includes a new rule that Computed-only (i.e. not also Optional) schemas _always_ map to attributes, because that is a better mapping of the intent: they are object values to be used in expressions. Nested blocks conceptually represent nested objects that are in some sense independent of what they are embedded in, and so they cannot themselves be computed.	7 years ago
Martin Atkins	a6d322edec	helper/schema: ConfigMode field in *Schema ... This allows a provider developer slightly more control over how an SDK schema is mapped into the Terraform configuration language, overriding some default assumptions. ConfigMode overrides the default assumption that a schema with an Elem of type *Resource is to be mapped to configuration as a nested block, allowing mapping as an attribute containing an object type instead. These behaviors only apply when a provider is being used with Terraform v0.12 or later. They are ignored altogether in Terraform v0.11 mode, to preserve compatibility. We are adding these primarily to allow the v0.12 version of a resource type schema to be specified to match the prevailing usage of it in existing configurations, in situations where the default mapping to v0.12 concepts is not appropriate. This commit adds only the fields themselves and the InternalValidate rules for them. A subsequent commit for Terraform v0.12 will add the behavior as part of the protocol version 5 shim layer.	7 years ago
Sander van Harmelen	973e2a7cf9	core: add a context to the UIInput interface	7 years ago
James Bardin	2adf5801d9	don't panic of the users aborts backend input ... When the user aborts input, it may end up as an unknown value, which needs to be converted to null for PrepareConfig. Allow PrepareConfig to accept null config values in order to fill in missing defaults.	7 years ago
James Bardin	37f391f1f7	insert defaults during Backend.PrepareConfig ... Lookup any defaults and insert them into the config value before validation.	7 years ago
James Bardin	c814f2da37	Change backend.ValidateConfig to PrepareConfig ... This mirrors the change made for providers, so that default values can be inserted into the config by the backend implementation. This is only the interface and method name changes, it does not yet add any default values.	7 years ago
Brian Flad	3d908f56aa	helper/schema: Add deprecation to ResourceData.UnsafeSetFieldRaw ... This functionality is no longer supported in Terraform 0.12 and above.	7 years ago
James Bardin	8be864c1c7	don't allow computed set elems to be equal ... If set elements are computed, we can't be certain that they are actually equal. Catch identical computed set hashes when they are added to the set, and alter the set key slightly to keep the set counts correct. In previous versions the interpolation string would be included in the set, and different string values would cause the set to hash differently, so this is change is only activated for the new protocol.	7 years ago
James Bardin	55b4307767	add proto5 feature flag ... Add feature flag to allow special proto 5 behavior in helper/schema. This is Meant to be used as a last resort for shim-related bugs.	7 years ago
James Bardin	81a4e705b1	DiffSuppressFunc should noop diffs in sets ... Sets rely on diffs being complete for all elements, even when they are unchanged. When encountering a DiffSuppressFunc inside a set the diffs were being dropped entirely, possible causing set elements to be lost.	7 years ago
James Bardin	9cf8f48239	decode legacy timeouts ... The new decoder is more precise, and unpacks the timeout block into a single map, which ResourceTimeout.ConfigDecode was updated to handle. We however still need to work with legacy versions of terraform, with the old decoder.	7 years ago
Paul Tyng	bb9ae50279	Copy TF version to helper/schema provider	7 years ago
Martin Atkins	ae0be75ae0	helper/schema: TypeMap of Resource is actually of TypeString ... Historically helper/schema did not support non-primitive map attributes because they cannot be represented unambiguously in flatmap. When we initially implemented CoreConfigSchema here we mapped that situation to a nested block of mode NestingMap, even though that'd never worked until now, assuming that it'd be harmless because providers wouldn't be using it. It turns out that some providers are, in fact, incorrectly populating a TypeMap schema with Elem: &schema.Resource, apparently under the false assumption that it would constrain the keys allowed in the map. In practice, helper/schema has just been ignoring this and treating such attributes as map of string. (#20076) In order to preserve the behavior of these existing incorrectly-specified attribute definitions, here we mimic the helper/schema behavior by presenting as an attribute of type map(string). These attributes have also been shown in some documentation as nested blocks (with no equals sign), so that'll need to be fixed in user configurations as they upgrade to Terraform 0.12. However, the existing upgrade tool rules will take care of that as a natural consequence of the name being indicated as an attribute in the schema, rather than as a block type. This fixes #20076.	7 years ago
James Bardin	bc5eecd7f2	make sure id really gets set in SetId ... SetId needs to overwrite the newState as well, since the internal calls to DataSource.Id() will override the set attribute.	7 years ago
Martin Atkins	06acc3f6c8	helper/schema: Skip validation of unknown values ... With the introduction of explicit "null" in 0.12 it's possible for a value that is unknown during plan to become a known null during apply, so we need to slightly weaken our validation rules to accommodate that, in particular skipping the validation of conflicting attributes if the result could potentially be valid after the unknown values become known. This change is in the codepath that is common to both 0.12 and 0.11 callers, but that's safe because 0.11 re-runs validation during the apply step and so will still catch problems here, albeit in the apply step rather than in the plan step, thus matching the 0.12 behavior. This new behavior is a superset of the old in the sense that everything that was valid before is still valid. The implementation here also causes us to skip all other validation for an attribute whose value is unknown. Most of the downstream validation functions handle this directly anyway, but again this doesn't add any new failure cases, and should clean up some of the rough edges we've seen with unknown values in 0.11 once people upgrade to 0.12-compatible providers. Any issues we now short-circuit during planning will still be caught during apply. While working on this I found that the existing "Not a list" test was not actually testing the correct behavior, so this also includes a tweak to that to ensure that it really is checking the "should be a list" path rather than the "cannot be set" codepath it was inadvertently testing before.	7 years ago
James Bardin	8ab5698e2a	Merge pull request #19587 from hashicorp/jbardin/safe-appends ... don't modify argument slices	7 years ago
James Bardin	3d6ec09a83	Merge pull request #19552 from olindata/bugfix/setting-sets-in-list ... helper/schema: Fix setting a set in a list caused error	7 years ago
James Bardin	b5de50c0a2	don't modify argument slices ... There were a couple spots where argument slices weren't being copied before `append` was called, which could possibly modify the caller's slice data.	7 years ago
Farid Neshat	44a45b7332	helper/schema: Fix setting a set in a list ... The added test in this commit, without the fix, will make d.Set return the following error: `Invalid address to set: []string{"ports", "0", "set"}` This was due to the fact that setSet in feild_writer_map tried to convert a slice into a set by creating a temp set schema and calling writeField on that with the address(`[]string{"ports", "0", "set"}"` in this case). However the temp schema was only for the set and not the whole schema as seen in the address so, it should have been `[]string{"set"}"` so it would align with the schema. This commits adds another variable there(tempAddr) which will only contain the last entry of the address that would be the set key, which would match the created schema This commit potentially fixes the problem described in #16331	7 years ago
Brian Flad	1e81a3e7fa	helper/schema: Always propagate NewComputed for previously zero value primative type attributes ... When the following conditions were met: * Schema attribute with a primative type (e.g. Type: TypeString) and Computed: true * Old state of attribute set to zero value for type (e.g. "") * Old state ID of resource set to non-empty (e.g. existing resource) Attempting to use CustomizeDiff with SetNewComputed() would result in the difference previously being discarded. This update ensures that previous zero values or resource existence does not influence the propagation of the computed update. Previously: ``` --- FAIL: TestSetNewComputed (0.00s) --- FAIL: TestSetNewComputed/NewComputed_should_always_propagate (0.00s) resource_diff_test.go:684: Expected (*terraform.InstanceDiff)(0xc00051cea0)({ mu: (sync.Mutex) { state: (int32) 0, sema: (uint32) 0 }, Attributes: (map[string]*terraform.ResourceAttrDiff) (len=1) { (string) (len=3) "foo": (*terraform.ResourceAttrDiff)(0xc0003dcec0)({ Old: (string) "", New: (string) "", NewComputed: (bool) true, NewRemoved: (bool) false, NewExtra: (interface {}) <nil>, RequiresNew: (bool) false, Sensitive: (bool) false, Type: (terraform.DiffAttrType) 0 }) }, Destroy: (bool) false, DestroyDeposed: (bool) false, DestroyTainted: (bool) false, Meta: (map[string]interface {}) <nil> }) , got (*terraform.InstanceDiff)(0xc00051ce80)({ mu: (sync.Mutex) { state: (int32) 0, sema: (uint32) 0 }, Attributes: (map[string]*terraform.ResourceAttrDiff) { }, Destroy: (bool) false, DestroyDeposed: (bool) false, DestroyTainted: (bool) false, Meta: (map[string]interface {}) <nil> }) --- FAIL: TestSchemaMap_Diff (0.01s) --- FAIL: TestSchemaMap_Diff/79-NewComputed_should_always_propagate_with_CustomizeDiff (0.00s) schema_test.go:3289: expected: *terraform.InstanceDiff{mu:sync.Mutex{state:0, sema:0x0}, Attributes:map[string]*terraform.ResourceAttrDiff{"foo":*terraform.ResourceAttrDiff{Old:"", New:"", NewComputed:true, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}}, Destroy:false, DestroyDeposed:false, DestroyTainted:false, Meta:map[string]interface {}(nil)} got: <nil> FAIL FAIL github.com/hashicorp/terraform/helper/schema 0.825s ```	7 years ago
Martin Atkins	72e279e6b2	providers: Consistently use int64 for schema versions ... The rest of Terraform is still using uint64 for this in various spots, but we'll update that gradually later. We use int64 here because that matches what's used in our protobuf definition, and unsigned integers are not portable across all of the protobuf target languages anyway.	7 years ago
Martin Atkins	58fa38b89a	helper/schema: Update docs for PromoteSingle ... This is no longer effective and should not be used in any new schema.	7 years ago
Martin Atkins	3d54af9c09	helper/schema: Better mimic some undocumented behaviors in Core schema ... Since the SDK's schema system conflates attributes and nested blocks, it's possible to state some nonsensical schema situations such as: - A nested block is both optional but has MinItems > 0 - A nested block is entirely computed but has MinItems or MaxItems set Both of these weird situations are handled here in the same way that the existing helper/schema validation code would've handled them: by effectively disabling the MinItems/MaxItems checks where they would've been ignored before. the MinItems/MaxItems	7 years ago
Martin Atkins	37da625ee9	helper/schema: Tell Core attribute is optional if set conditionally ... The SDK has a mechanism that effectively makes it possible to declare an attribute as being _conditionally_ required, which is not a concept that Terraform Core is aware of. Since this mechanism is in practice only used for a small UX improvement in prompting for these values interactively when the environment variable is not set, we avoid here introducing all of this complexity into the plugin protocol by just having the provider selectively modify its schema if it detects that such an attribute might be set dynamically. This then prevents Terraform Core from validating the presence of the argument or prompting for a new value for it, allowing the null value to pass through into the provider so that the default value can be generated again dynamically. This is a kinda-kludgey solution which we're accepting here because the alternative would be a much-more-complex two-pass decode operation within Core itself, and that doesn't seem worth it. This fixes #19139.	7 years ago
James Bardin	21dfa56766	use ShimInstanceStateFromValue in DiffFromValues ... This makes sure the diff is generated with the matching set ids from helper/schema. Update the tests to add ID fields to the state, which will exists in practice, since any state traversing through the shims will have the ID inserted.	7 years ago
James Bardin	df04e2e7a6	move InstanceState shim into schema.Resource ... This was the resource can rebuild the flatmapped state using the schema and ResourceData, providing us the the correct set key values.	7 years ago
Radek Simko	0cbf745e5a	helper/schema: Avoid erroring out on undefined timeouts	7 years ago
Radek Simko	1cb8f1df80	helper/schema: Fix timeout parsing in ResourceTimeout.ConfigDecode	7 years ago
Radek Simko	82a77f9bb5	helper/schema: Add test for invalid timeout value	7 years ago
Radek Simko	2fe3f16cb3	helper/schema: Return error on invalid timeout type	7 years ago
Radek Simko	186a6dcc38	helper/schema: Add test for wrong timeout type	7 years ago
James Bardin	f153720a36	add checks for timeouts attributes and blocks ... Don't overwrite anything the provider defined, in order to maintain existing behavior. Change strings to pre-defined constants	7 years ago
James Bardin	6dad121e70	insert resource timeouts into the config schema ... Resource timeouts were a separate config block, but did not exist in the resource schema. Insert any defined timeouts when generating the configshema.Block so that the fields can be accepted and validated by core.	7 years ago
James Bardin	d50a152f8b	check for a nil diff in simpleDiff	8 years ago
James Bardin	ff4e81cc2b	add old values when computing the new InstanceDiff ... This was previously done in the RequiresNew code, which is skipped in new style provider.	8 years ago
James Bardin	38163f2b37	use SimpleDiff and set "id" as RequiresReplace ... Use the new SimpleDiff method of the provider so that the diff isn't altered by ForceNew attributes. Always set an "id" as RequiresReplace so core knows an instance will be replaced, even if all ForceNew attributes are filtered out due to ignore_changes.	8 years ago
James Bardin	46b4c27dbe	create a SimpleDiff for the new provider shims ... Terraform now handles any actual "diffing" of resource, and the existing Diff functions are only used to shim the schema.Provider to the new methods. Since terraform is handling what used to be the Diff, the provider now should not modify the diff based on RequiresNew due to it interfering with the ignore_changes handling.	8 years ago
James Bardin	0d4d572c39	start work on helper/resource test fixtures ... The helper resource tests won't pass for now, as they use a terraform.MockProvider which can't be used in the schema.Provider shims.	8 years ago
Martin Atkins	33151f5011	core: Move StateValueFromInstanceState shim from helper/schema ... This one doesn't depend on any helper/schema specific bits and it'll also be useful for the shims in our mock provider in core.	8 years ago
Martin Atkins	76d11f44cc	core: Move some of the helper/schema shims so provider mock can use them ... The old names are now wrappers around these new functions.	8 years ago
James Bardin	a87470cc15	resource ids must always have a value ... The "id" field is assumed to always exist, and must have a valid value. Set "id" to unknown when planning new resource changes to indicate that it will be computed.	8 years ago
James Bardin	b88410984b	legacy provider needs to handle StateUpgraders ... In order to not require state migrations to be supported in both MigrateState and StateUpgraders, the legacy provider codepath needs to handle the StateUpgraders transparently during Refresh.	8 years ago
James Bardin	dcaf5aa262	add some of the shims used for the grpc provider ... This adds some of the required shim functions to the schema package. While this further bloats the already huge package, adding the helpers here was significantly less disruptive than refactoring types into separate packages to prevent import cycles. The majority of tests here are directly adapted from existing schema tests to provide as many known good values to the shims as possible.	8 years ago