130 Commits (e2b6ccad8a755fb9ea1b32fb71eec02216438111)

Author	SHA1	Message	Date
James Nugent	e0226c9039	core: Check for attrV being nil before dereference ... This can be an issue with unset computed fields. Fixes #8815.	10 years ago
James Nugent	85ec09111b	helper/schema: Add diff suppression callback ... This commit adds a new callback, DiffSuppressFunc, to the schema.Schema structure. If set for a given schema, a callback to the user-supplied function will be made for each attribute for which the default type-based diff mechanism produces an attribute diff. Returning `true` from the callback will suppress the diff (i.e. pretend there was no diff), and returning false will retain it as part of the plan. There are a number of motivating examples for this - one of which is included as an example: 1. On SSH public keys, trailing whitespace does not matter in many cases - and in some cases it is added by provider APIs. For digitalocean_ssh_key resources we previously had a StateFunc that trimmed the whitespace - we now have a DiffSuppressFunc which verifies whether the trimmed strings are equivalent. 2. IAM policy equivalence for AWS. A good proportion of AWS issues relate to IAM policies which have been "normalized" (used loosely) by the IAM API endpoints. This can make the JSON strings differ from those generated by iam_policy_document resources or template files, even though the semantics are the same (for example, reordering of `bucket-prefix/` and `bucket-prefix/*` in an S3 bucket policy. DiffSupressFunc can be used to test for semantic equivalence rather than pure text equivalence, but without having to deal with the complexity associated with a full "provider-land" diff implementation without helper/schema.	10 years ago
James Nugent	074545e536	core: Use .% instead of .# for maps in state ... The flatmapped representation of state prior to this commit encoded maps and lists (and therefore by extension, sets) with a key corresponding to the number of elements, or the unknown variable indicator under a .# key and then individual items. For example, the list ["a", "b", "c"] would have been encoded as: listname.# = 3 listname.0 = "a" listname.1 = "b" listname.2 = "c" And the map {"key1": "value1", "key2", "value2"} would have been encoded as: mapname.# = 2 mapname.key1 = "value1" mapname.key2 = "value2" Sets use the hash code as the key - for example a set with a (fictional) hashcode calculation may look like: setname.# = 2 setname.12312512 = "value1" setname.56345233 = "value2" Prior to the work done to extend the type system, this was sufficient since the internal representation of these was effectively the same. However, following the separation of maps and lists into distinct first-class types, this encoding presents a problem: given a state file, it is impossible to tell the encoding of an empty list and an empty map apart. This presents problems for the type checker during interpolation, as many interpolation functions will operate on only one of these two structures. This commit therefore changes the representation in state of maps to use a "%" as the key for the number of elements. Consequently the map above will now be encoded as: mapname.% = 2 mapname.key1 = "value1" mapname.key2 = "value2" This has the effect of an empty list (or set) now being encoded as: listname.# = 0 And an empty map now being encoded as: mapname.% = 0 Therefore we can eliminate some nasty guessing logic from the resource variable supplier for interpolation, at the cost of having to migrate state up front (to follow in a subsequent commit). In order to reduce the number of potential situations in which resources would be "forced new", we continue to accept "#" as the count key when reading maps via helper/schema. There is no situation under which we can allow "#" as an actual map key in any case, as it would not be distinguishable from a list or set in state.	10 years ago
James Nugent	91587a49f3	core: Remove unnecessary debug logging ... Some unnecessary debug logging was introduced in 7b6df27e4, this commit removes it so as not to clutter logs.	10 years ago
Chris Marchesi	9d7fb89114	core: Adding Sensitive attribute to resource schema ... This an effort to address hashicorp/terraform#516. Adding the Sensitive attribute to the resource schema, opening up the ability for resource maintainers to mark some fields as sensitive. Sensitive fields are hidden in the output, and, possibly in the future, could be encrypted.	10 years ago
Sander van Harmelen	8560f50cbc	Change taint behaviour to act as a normal resource ... This means it’s shown correctly in a plan and takes into account any actions that are dependant on the tainted resource and, vice verse, any actions that the tainted resource depends on. So this changes the behaviour from saying this resource is tainted so just forget about it and make sure it gets deleted in the background, to saying I want that resource to be recreated (taking into account the existing resource and it’s place in the graph).	10 years ago
Martin Atkins	6a468dcd83	helper/schema: Resource can be writable or not ... In the "schema" layer a Resource is just any "thing" that has a schema and supports some or all of the CRUD operations. Data sources introduce a new use of Resource to represent read-only resources, which require some different InternalValidate logic.	10 years ago
James Nugent	7b6df27e4a	helper/schema: Read native maps from configuration ... This adds a test and the support necessary to read from native maps passed as variables via interpolation - for example: ``` resource ...... { mapValue = "${var.map}" } ``` We also add support for interpolating maps from the flat-mapped resource config, which is necessary to support assignment of computed maps, which is now valid. Unfortunately there is no good way to distinguish between a list and a map in the flatmap. In lieu of changing that representation (which is risky), we assume that if all the keys are numeric, this is intended to be a list, and if not it is intended to be a map. This does preclude maps which have purely numeric keys, which should be noted as a backwards compatibility concern.	10 years ago
Paul Hinze	b4df304b47	helper/schema: Normalize bools to "true"/"false" in diffs ... For a long time now, the diff logic has relied on the behavior of `mapstructure.WeakDecode` to determine how various primitives are converted into strings. The `schema.DiffString` function is used for all primitive field types: TypeBool, TypeInt, TypeFloat, and TypeString. The `mapstructure` library's string representation of booleans is "0" and "1", which differs from `strconv.FormatBool`'s "false" and "true" (which is used in writing out boolean fields to the state). Because of this difference, diffs have long had the potential for cosmetically odd but semantically neutral output like: "true" => "1" "false" => "0" So long as `mapstructure.Decode` or `strconv.ParseBool` are used to interpret these strings, there's no functional problem. We had our first clear functional problem with #6005 and friends, where users noticed diffs like the above showing up unexpectedly and causing troubles when `ignore_changes` was in play. This particular bug occurs down in Terraform core's EvalIgnoreChanges. There, the diff is modified to account for ignored attributes, and special logic attempts to handle properly the situation where the ignored attribute was going to trigger a resource replacement. That logic relies on the string representations of the Old and New fields in the diff to be the same so that it filters properly. So therefore, we now get a bug when a diff includes `Old: "0", New: "false"` since the strings do not match, and `ignore_changes` is not properly handled. Here, we introduce `TypeBool`-specific normalizing into `finalizeDiff`. I spiked out a full `diffBool` function, but figuring out which pieces of `diffString` to duplicate there got hairy. This seemed like a simpler and more direct solution. Fixes #6005 (and potentially others!)	10 years ago
Chris Marchesi	8c5354b7dc	Add MaxItems attribute to Schema ... * MaxItems defines a maximum amount of items that can exist within a TypeSet or TypeList. Specific use cases would be if a TypeSet is being used to wrap a complex structure, however more than one instance would cause instability.	10 years ago
Paul Hinze	99244c5597	helper/schema: skip provider input for deprecated fields ... There's no reason that a field that's been deprecated should ever prompt. fixes #4033	11 years ago
Paul Hinze	f1e7cec566	Merge pull request #3992 from svanharmelen/f-change-sets ... core: change set internals and make (extreme) performance improvements	11 years ago
Sander van Harmelen	ef4726bd50	Change Set internals and make (extreme) performance improvements ... Changing the Set internals makes a lot of sense as it saves doing conversions in multiple places and gives a central place to alter the key when a item is computed. This will have no side effects other then that the ordering is now based on strings instead on integers, so the order will be different. This will however have no effect on existing configs as these will use the individual codes/keys and not the ordering to determine if there is a diff or not. Lastly (but I think also most importantly) there is a fix in this PR that makes diffing sets extremely more performand. Before a full diff required reading the complete Set for every single parameter/attribute you wanted to diff, while now it only gets that specific parameter. We have a use case where we have a Set that has 18 parameters and the set consist of about 600 items (don't ask 😉). So when doing a diff it would take 100% CPU of all cores and stay that way for almost an hour before being able to complete the diff. Debugging this we learned that for retrieving every single parameter it made over 52.000 calls to `func (c *ResourceConfig) get(..)`. In this function a slice is created and used only for the duration of the call, so the time needed to create all needed slices and on the other hand the time the garbage collector needed to clean them up again caused the system to cripple itself. Next to that there are also some expensive reflect calls in this function which also claimed a fair amount of CPU time. After this fix the number of calls needed to get a single parameter dropped from 52.000+ to only 2! 😃	11 years ago
Paul Hinze	c7dc1c10a3	helper/schema: skip StateFunc when value is nil ... This takes the nil checking burden off of StateFunc. fixes #3586, see that issue for further discussion	11 years ago
Martin Atkins	a67182543c	Nicer error when list/map assigned to string argument. ... Previous this would return the following sort of error: expected type 'string', got unconvertible type '[]interface {}' This is the raw error returned by the underlying mapstructure library. This is not a helpful error message for anyone who doesn't know Go's type system, and it exposes Terraform's internals to the UI. Instead we'll catch these cases before we try to use mapstructure and return a more straightforward message. By checking the type before the IsComputed exception this also avoids a crash caused when the assigned value is a computed list. Otherwise the list of interpolations is allowed through here and then crashes later during Diff when the value is not a primitive as expected.	11 years ago
Paul Hinze	2a179d1065	helper/schema: ValidateFunc support for maps	11 years ago
Panagiotis Moustafellos	e4845f75cc	removed extra parentheses	11 years ago
Martin Atkins	cc8e8a55de	helper/schema: Default hashing function for sets ... A common issue with new resource implementations is not considering parts of a complex structure that's used inside a set, which causes quirky behavior. The schema helper has enough information to provide a default reasonable implementation of a set function that includes all non-computed attributes in a deterministic way. Here we implement such a function and use it when no explicit hashing function is provided. In order to achieve this we encapsulate the construction of the zero value for a schema in a new method schema.ZeroValue, which allows us to put the fallback logic to the new default function in a single spot. It is no longer valid to use &Set{F: schema.Set} and all uses of that construct should be replaced with schema.ZeroValue().(*Set) .	11 years ago
Anthony Scalisi	198e1a5186	remove various typos	11 years ago
Mitchell Hashimoto	0100d4139b	helper/schema: clean up style	11 years ago
Mitchell Hashimoto	6e509aedcb	helper/schema: diff should include removed set items [GH-1823]	11 years ago
Radek Simko	6fdbca8e58	Merge pull request #2466 from TimeIncOSS/f-schema-field-name-validate ... schema: Add field name to ValidateFunc	11 years ago
Radek Simko	92db4802b6	schema: Add field name to ValidateFunc	11 years ago
Mitchell Hashimoto	4f391902a0	helper/schema: validate objects are objects [GH-2166]	11 years ago
Paul Hinze	a4912cc51f	helper/schema: limit ValidateFunc to primitives for now ... I couldn't see a simple path get this working for Maps, Sets, and Lists, so lets land it as a primitive-only schema feature. I think validation on primitives comprises 80% of the use cases anyways.	11 years ago
Paul Hinze	49352db26f	helper/schema: skip ValidateFunc on other errors ... Guarantees that the `interface{}` arg to ValidateFunc is the proper type, allowing implementations to be simpler. Finish the docstring on `ValidateFunc` to call this out. /cc @mitchellh	11 years ago
Paul Hinze	61fee6735d	helper/schema: ValidateFunc ... Allows provider authors to implement arbitrary per-field validation warnings or errors.	11 years ago
Sam Boyer	b82bd0c280	Condense switch fallthroughs into expr lists	11 years ago
Paul Hinze	1e3d1b07e6	helper/schema: validate ConflictsWith against top-level ... The runtime impl of ConfictsWith uses Resource.Get(), which makes it work with any other attribute of the resource - the InternalValidate was only checking against the local schemaMap though, preventing subResource from using ConflictsWith properly. It's a lot of wiring and it's a bit ugly, but it's not runtime code, so I'm a bit less concerned about that aspect. This should take care of the problem mentioned in #1909	11 years ago
Mitchell Hashimoto	707aa36aec	helper/schema: only use ~ on first char of code	11 years ago
Mitchell Hashimoto	1f46bc1926	helper/schema: validate unknown fields with computed values [GH-1507]	11 years ago
Mitchell Hashimoto	3176e5b44a	Merge pull request #1595 from TimeIncOSS/aws-account-protection ... aws: Allow defining blacklist/whitelist of account IDs	11 years ago
Mitchell Hashimoto	54e3e6104e	Merge pull request #1594 from TimeIncOSS/schema-conflicts-with ... helper/schema: add schema.ConflictsWith[]	11 years ago
Mitchell Hashimoto	51951d68f4	helper/schema: change diff logic around maps to fix case #57 and #44	11 years ago
Radek Simko	150fd00932	AWS account ID protection added	11 years ago
Radek Simko	34f48b3e06	Add schema.ConflictsWith[] ... - this will allow defining logically conflicting attributes	11 years ago
Radek Simko	e0df74c863	Add schema.ConflictsWith[] ... - this will allow defining logically conflicting attributes	11 years ago
Paul Hinze	ef70c8cae5	helper/schema: allow Schema attrs to be Removed ... Removed fields show a customizable error message to the user when they are used in a Terraform config. This is a tool that provider authors can use for user feedback as they evolve their Schemas. refs #957	11 years ago
Paul Hinze	888f16d2d3	helper/schema: allow Schema attrs to be Deprecated ... Deprecated fields show a customizable warning message to the user when they are used in a Terraform config. This is a tool that provider authors can use for user feedback as they evolve their Schemas. fixes #957	11 years ago
Mitchell Hashimoto	17680bb7ff	helper/schema: some more test cases, revert some weird behavior from ... dbfb95fcd5 I don't know why that behavior was in there, but it was breaking a lot of existing Terraform states. Let's circle back on it.	11 years ago
Mitchell Hashimoto	659a77c6ae	helper/schema: validate subresources more effectively	11 years ago
Mitchell Hashimoto	fa7f496bef	helper/schema: zero value of a set should be empty	11 years ago
Mitchell Hashimoto	5c06cc386a	helper/schema: empty map values should show up in diff [GH-968]	11 years ago
Mitchell Hashimoto	dbfb95fcd5	helper/schema: show in diff when no config is going to empty set	11 years ago
Mitchell Hashimoto	bcdec738d4	helper/schema: default the new value to zero only for the decode	11 years ago
Mitchell Hashimoto	b778a65a83	helper/schema: diff of zero value in state with lack of value should not ... diff	11 years ago
Clint Shryock	4c7b732dad	typo	11 years ago
INADA Naoki	33aa9d3ee8	Fix stringer error on helper/schema/schema.go	11 years ago
Dave Cunningham	18c26cb2eb	Add some missing Float cases	11 years ago
Paul Hinze	5d4e69cc80	helper/schema: apply schema defaults at the field level when reading from config ... We were waiting until the higher-level (m schemaMap) diffString method to apply defaults, which was messing with set hashcode evaluation for cases when a field with a default is included in the hash function. fixes #824	11 years ago