139 Commits (25cebfbd8be2c43a25dfafdcf882e79b9ccc3d3d)

Author	SHA1	Message	Date
Alisdair McDiarmid	dbe5272931	functions: Improve marks support for lookup ... Several changes to lookup to improve how we handle marked values: - If the entire collection is marked, preserve the marks on any result (whether successful or fallback) - If a returned value from the collection is marked, preserve the marks from only that value, combined with any overall collection marks - Retain marks on the fallback value when it is returned, combined with any overall collection marks - Include marks on the key in the result, as otherwise the result it ends up selecting could imply what the sensitive value was - Retain collection marks when returning an unknown value for a not wholly-known collection See also https://github.com/zclconf/go-cty/pull/98	5 years ago
Alisdair McDiarmid	e0c6b3fcda	functions: Improve marks support for length ... Similar to cty's implementation, we only need to preserve marks from the value itself, not any nested values it may contain. This means that taking the length of an umarked list with marked elements results in an unmarked number.	5 years ago
Kristin Laemmert	f6af7b4f7a	lang/funcs: add (console-only) TypeFunction (#28501) ... * lang/funcs: add (console-only) TypeFunction The type() function, which is only available for terraform console, prints out the type of a given value. This is mainly intended for debugging - it's nice to be able to print out terraform's understanding of a complex variable. This introduces a new field for Scope: ConsoleMode. When ConsoleMode is true, any additional functions intended for use in the console (only) may be added.	5 years ago
Kristin Laemmert	5ae4c2f92b	lang/funcs: Defaults handling of marked arguments ... Defaults will now preserve marks from non-null inputs and apply marks from any default values used. I've added tests for various structural types with marks, as well as some basic unknown cases.	5 years ago
Kristin Laemmert	b34588ffca	lang/funcs: TransposeFunc Tests ... This commit adds test cases to TestTranspose to document how this function handles marks. The short version is that any marks anywhere will be applied to the return value, be they marks on the input map or marks on elements (either the entire list of strings, or individual elemnets of those lists).	5 years ago
Martin Atkins	14336ae6f8	lang/funcs: Conversion functions can handle sensitive values ... In order to avoid updating every one of our existing functions with explicit support for sensitive values, there's a default rule in the functions system which makes the result of a function sensitive if any of its arguments contain sensitive values. We were applying that default to the various type conversion functions, like tomap and tolist, which meant that converting a complex-typed value with a sensitive value anywhere inside it would result in a wholly-sensitive result. That's unnecessarily conservative because the cty conversion layer (which these functions are wrapping) already knows how to handle sensitivity in a more precise way. Therefore we can opt in to handling marked values (which Terraform uses for sensitivity) here and the only special thing we need to do is handle errors related to sensitive values differently, so we won't print their values out literally in case of an error (and so that the attempt to print them out literally won't panic trying to extract the marked values).	5 years ago
Martin Atkins	140c613ae8	lang/funcs: "one" function ... In the Terraform language we typically use lists of zero or one values in some sense interchangably with single values that might be null, because various Terraform language constructs are designed to work with collections rather than with nullable values. In Terraform v0.12 we made the splat operator [*] have a "special power" of concisely converting from a possibly-null single value into a zero-or-one list as a way to make that common operation more concise. In a sense this "one" function is the opposite operation to that special power: it goes from a zero-or-one collection (list, set, or tuple) to a possibly-null single value. This is a concise alternative to the following clunky conditional expression, with the additional benefit that the following expression is also not viable for set values, and it also properly handles the case where there's unexpectedly more than one value: length(var.foo) != 0 ? var.foo[0] : null Instead, we can write: one(var.foo) As with the splat operator, this is a tricky tradeoff because it could be argued that it's not something that'd be immediately intuitive to someone unfamiliar with Terraform. However, I think that's justified given how often zero-or-one collections arise in typical Terraform configurations. Unlike the splat operator, it should at least be easier to search for its name and find its documentation the first time you see it in a configuration. My expectation that this will become a common pattern is also my justification for giving it a short, concise name. Arguably it could be better named something like "oneornull", but that's a pretty clunky name and I'm not convinced it really adds any clarity for someone who isn't already familiar with it.	5 years ago
Alisdair McDiarmid	c1f7193454	lang/funcs: Make nonsensitive more permissive ... Calling the nonsensitive function with values which are not sensitive will result in an error. This restriction was added with the goal of preventing confusingly redundant use of this function. Unfortunately, this breaks when using nonsensitive to reveal the value of sensitive resource attributes. This is because the validate walk does not (and cannot) mark attributes as sensitive based on the schema, because the resource value itself is unknown. This commit therefore alters this restriction such that it permits nonsensitive unknown values, and adds a test case to cover this specific scenario.	5 years ago
Martin Atkins	89b2405080	lang/funcs: "sensitive" and "nonsensitive" functions ... These aim to allow hinting to Terraform about situations where it's not able to automatically infer value sensitivity. "nonsensitive" is for situations where Terraform's behavior is too conservative, such as when a new value is derived from a sensitive value in such a way that all of the sensitive content is removed. "sensitive", on the other hand, is for situations where Terraform can't otherwise infer that a value is sensitive. These situations should be pretty rare in a module that's making effective use of sensitive input variables and output values, but the documentation shows one example of an uncommon situation where a more direct hint via this function would be needed. Both of these functions are aimed at only occasional use in unusual situations. They are here for reasons of pragmatism, not because we expect them to be used routinely or recommend their use.	5 years ago
Alisdair McDiarmid	7f97bd4489	functions: Fix missing defaults for objects/tuples ... If no default is specified for a nested optional structural typed attribute, the defaults function should just pass through its input. Before this commit the function assumed that the fallback value was always of the correct type, which would panic.	5 years ago
Alisdair McDiarmid	0bbe583eb8	functions: Fix defaults for null objects/tuples ... When using defaults with a value which contains null objects or tuples, we cannot continue to traverse the value and apply defaults. Instead, when we find an attribute which is null, we return early and stop processing this branch.	5 years ago
Alisdair McDiarmid	66f8d1c1c2	functions: Fix defaults mismatched types fallback ... We allow primitive fallback values which have mismatched types, but only if there is a conversion to the target type. Previously we would allow unsafe conversions (e.g. string to bool), but later had no capacity to return an error if the conversion failed due to the value of the fallback being unable to convert to the target type. This commit makes the more conservative requirement that default fallback values must have a safe conversion.	5 years ago
Alisdair McDiarmid	178a9b32d7	functions: Fix defaults null collections panic ... When applying default values to collection types, null collections in the input should result in empty collections in the output.	5 years ago
Alisdair McDiarmid	f27dae2ab7	lang: Improved robustness of sum function ... Fixes error when calling sum with values not known until apply time. Also allows sum to cope with numbers too large to represent in float64, along with correctly handling errors when trying to sum opposing infinities.	5 years ago
Alisdair McDiarmid	df626e898c	lang/funcs: Fix alltrue/anytrue with unknowns ... The alltrue/anytrue functions did not correctly handle unknown values. This commit changes these functions so that the result is unknown if: - The list argument is unknown - For alltrue: any elements are unknown - For anytrue: any elements are unknown and no known elements are true The last change is a little subtle, so there are test cases to cover it specifically. Examples: - anytrue(unknown) -> unknown - anytrue(false, unknown) -> unknown - anytrue(false, unknown, true) -> true	5 years ago
James Bardin	e08422511e	lang/funcs: staticcheck	5 years ago
Martin Atkins	cec4578005	lang/funcs: Experimental "defaults" function ... This is a new part of the existing module_variable_optional_attrs experiment, because it's intended to complement the ability to declare an input variable whose type constraint is an object type with optional attributes. Module authors can use this to replace null values (that were either explicitly set or implied by attribute omission) with other non-null values of the same type. This function is a bit more type-fussy than our functions typically are because it's intended for use primarily with input variables that have fully-specified type constraints, and thus it uses that type information to help inform how the defaults data structure should be interpreted. Other uses of this function will probably be harder today because it takes a lot of extra annotation to build a value of a specific type if it isn't passing through a variable type constraint. Perhaps later language features for more general type conversion will make this more applicable, but for now the more general form of this problem is better solved other ways.	6 years ago
Martin Atkins	ae3c0c6a4a	lang/funcs: Remove the deprecated "list" and "map" functions ... Prior to Terraform 0.12 these two functions were the only way to construct literal lists and maps (respectively) in HIL expressions. Terraform 0.12, by switching to HCL 2, introduced first-class syntax for constructing tuple and object values, which can then be converted into list and map values using the tolist and tomap type conversion functions. We marked both of these functions as deprecated in the Terraform v0.12 release and have since then mentioned in the docs that they will be removed in a future Terraform version. The "terraform 0.12upgrade" tool from Terraform v0.12 also included a rule to automatically rewrite uses of these functions into equivalent new syntax. The main motivation for removing these now is just to get this change made prior to Terraform 1.0. as we'll be doing with various other deprecations. However, a specific reason for these two functions in particular is that their existence is what caused us to invent the idea of a "type expression" as a distinct kind of expression in Terraform v0.12, and so removing them now would allow potentially unifying type expressions with value expressions in a future release. We do not have any current specific plans to make that change, but one potential motivation for doing so would be to take another attempt at a generalized "convert" function which takes a type as one of its arguments. Our previous attempt to implement such a function was foiled by the fact that Terraform's expression validator doesn't have any way to know to treat one argument of a particular function as special, and so it was generating incorrect error messages. We won't necessarily do that, but having these "list" and "map" functions out of the way leaves the option open.	6 years ago
Arthur Burkart	d4716a69e1	lang/funcs: "anytrue" function ... This is an analog to the "alltrue" function, using OR as the reduce operator rather than AND. This also includes some simplification of the "alltrue" implementation to implement it similarly as a sort of reduce operation with AND as the reduce operator, but with the same effective behavior.	6 years ago
Martin Atkins	1dc4950bfa	lang/funcs: Rename the base64 character encoding functions ... These were initially introduced as functions with "encode" and "decode" prefixes, but that doesn't match with our existing convention of putting the encoding format first so that the encode and decode functions will group together in a alphabetically-ordered function list. "text" is not really a defined serialization format, but it's a short word that hopefully represents well enough what these functions are aiming to encode and decode, while being consistent with existing functions like jsonencode/jsondecode, yamlencode/yamldecode, etc. The "base64" at the end here is less convincing because there is precedent for that modifier to appear both at the beginning and the end in our existing function names. I chose to put it at the end here because that seems to be our emergent convention for situations where the base64 encoding is a sort of secondary modifier alongside the primary purpose of the function, as we see with "filebase64". (base64gzip is an exception here, but it seems outvoted by the others.)	6 years ago
r0bnet	877399c631	lang/funcs: Functions for encoding text in specific character encodings	6 years ago
OwenTuz	bb39fafbe5	lang/funcs: lookup() will only treat map as unknown if it is wholly unknown (#26427) ... Fix for issue #26320 - this allows us to derive known values from partially known maps where we can, and may prevent unnecessary destroy/rebuild cycles during apply in some cases.	6 years ago
Arthur Burkart	6ed47c7241	lang/funcs: Add "alltrue" function (#25656) ... This commit adds an `alltrue` function to Terraform configuration. A reason we might want this function is because it will enable more powerful custom variable validations. For example: ```hcl variable "amis" { type = list(object({ id = string })) validation { condition = (alltrue([ for a in var.amis : length(a.id) > 4 && substr(a.id, 0, 4) == "ami-" ])) error_message = "The ID of at least one AMI was invalid." } } ```	6 years ago
Kristin Laemmert	e639d813ee	add test cases and remove no-longer-needed validation	6 years ago
Kristin Laemmert	384cfaacf9	lang/funcs: refactor Subnet and Host functions to support 64-bit systems	6 years ago
Martin Atkins	8f77bd344e	lang/funcs: Filesystem functions hint about dynamically-generated files ... The functions that interact with the filesystem are, by design, restricted to reading files that are distributed as a static part of the configuration, and cannot be used to interact with files that are generated dynamically by resources in the configuration. However, new users have often yet developed a correct mental model of how Terraform execution works and are confused by the terse error messages these functions return. As an interim step to help some of those users, this just adds some more commentary to the error message which gives a vague, generic directive to look to attributes of the resource that is generating the file, which should (if it's designed well) export attributes that allow the resulting file to be used effectively with common patterns, such as checksums of the content of the generated file. The error message here is not particularly attractive due to the limitations of the context where it's being returned from, but I'm accepting that here in the interest of keeping this change simple, so we can give a hint about a case that seems to frequently generate new-user questions. We may iterate further on the presentation of this message later.	6 years ago
Alisdair McDiarmid	a4f5e04066	lang/funcs: Add support for OpenSSH RSA key format ... Previously this function only supported the x509 RSA private key format. More recent versions of OpenSSH default to generating a new PEM key format, which this commit now supports using the x/crypto/ssh package. Also improve the returned error messages for various invalid ciphertext or invalid private key errors.	6 years ago
Daniel Dreier	0749e419de	Merge pull request #24484 from ctjhoa/fix_typo ... fix typo in Base64DecodeFunc log	6 years ago
Noah Mercado	d4d8812afa	Feature: Sum Function (#24666) ... The sum function takes a list or set of numbers and returns the sum of those numbers.	6 years ago
Camille TJHOA	d7f711f40e	fix typo in Base64DecodeFunc log	6 years ago
Alisdair McDiarmid	37006c5841	lang: Fix non-string key panics in map function ... The map function assumed that the key arguments were strings, and would panic if they were not. After this commit, calling `map(1, 2)` will result in a map `{"1" = 1}`, and calling `map(null, 1)` will result in a syntax error. Fixes #23346, fixes #23043	6 years ago
James Bardin	0bd40fd496	Merge pull request #24265 from hashicorp/jbardin/cty-update ... Update cty to v1.3.1 and migrate to cty stdlib function where applicable	6 years ago
James Bardin	d999d43483	remove old funcs code	6 years ago
Martin Atkins	67d95b97ce	lang/funcs: templatefile requires valid variable names ... Previously the templatefile function would permit any arbitrary string as a variable name, but due to the HCL template syntax it would be impossible to refer to one that isn't a valid HCL identifier without causing an HCL syntax error. The HCL syntax errors are correct, but don't really point to the root cause of the problem. Instead, we'll pre-verify that the variable names are valid before we even try to render the template, and given a specialized error message that refers to the vars argument expression as the problematic part, which will hopefully make the resolution path clearer for a user encountering this situation. The syntax error still remains for situations where all of the variable names are correct but e.g. the user made a typo referring to one, which makes sense because in that case the problem _is_ inside the template.	6 years ago
Martin Atkins	ec9f950b3f	lang/funcs: Test actual error messages from templatefile ... This function has a number of different error cases with hopefully-helpful error messages for each, so it's good to test we're getting the error message we were actually expecting in each case.	6 years ago
James Bardin	529271e0be	update merge docs to match behavior	6 years ago
James Bardin	f5bf9aa55d	make the merge function more precise ... This PR implements 2 changes to the merge function. - Rather than always defining the merge return type as dynamic, return a precise type when all argument types match, or all possible object attributes are known. - Always return a value containing all keys when the keys are known. This allows the use of merge output in for_each, even when keys are yet to be determined.	6 years ago
George Christou	91100c003c	lang/funcs: Add more `trim*` functions (#23016) ... * lang/funcs: Add `trim*` functions	7 years ago
Parviz	6eaf1c23f9	lang/funcs: don't panic when transpose produces empty result	7 years ago
Martin Atkins	39e609d5fd	vendor: switch to HCL 2.0 in the HCL repository ... Previously we were using the experimental HCL 2 repository, but now we'll shift over to the v2 import path within the main HCL repository as part of actually releasing HCL 2.0 as stable. This is a mechanical search/replace to the new import paths. It also switches to the v2.0.0 release of HCL, which includes some new code that Terraform didn't previously have but should not change any behavior that matters for Terraform's purposes. For the moment the experimental HCL2 repository is still an indirect dependency via terraform-config-inspect, so it remains in our go.sum and vendor directories for the moment. Because terraform-config-inspect uses a much smaller subset of the HCL2 functionality, this does still manage to prune the vendor directory a little. A subsequent release of terraform-config-inspect should allow us to completely remove that old repository in a future commit.	7 years ago
Martin Atkins	f84ab99b7d	lang/funcs: cidrsubnets function ... This is a companion to cidrsubnet that allows bulk-allocation of multiple subnet addresses at once, with automatic numbering. Unlike cidrsubnet, cidrsubnets allows each of the allocations to have a different prefix length, and will pack the networks consecutively into the given address space. cidrsubnets can potentially create more complicated addressing schemes than cidrsubnet alone can, because it's able to take into account the full set of requested prefix lengths rather than just one at a time.	7 years ago
Jeet Parekh	bcc69c05bb	lang/funcs: parseint function	7 years ago
Pam Selle	f9ebae749c	Merge pull request #22707 from vsimon/spell ... docs: Minor spelling and typo fixes	7 years ago
Vicken Simonian	853a0e0677	docs: Minor spelling and typo fixes	7 years ago
Brian Flad	19cf34114f	lang/funcs: Switch fileset() function glob implementation to github.com/bmatcuk/doublestar to support additional glob patterns ... This allows the usage of the glob patterns `**` and `{alternative1,...}` to simplify Terraform configuration logic for more complex file matching.	7 years ago
Brian Flad	769f626a0e	lang/funcs: Remove homedir.Expand() and refactor path trimming with filepath.Rel() in fileset() function ... Reference: https://github.com/hashicorp/terraform/pull/22621#pullrequestreview-282259385	7 years ago
Brian Flad	820b79d91d	lang/funcs: Clarification update to MakeFileSetFunc comment for filepath.ToSlash usage	7 years ago
Brian Flad	af7f6ef441	lang/funcs: Update fileset() function to include path as separate first argument, automatically trim the path argument from results, and ensure results are always canonical with forward slash path separators ... Reference: https://github.com/hashicorp/terraform/pull/22523#pullrequestreview-279694703 These changes center around better function usability and consistency with other functions. The function has not yet been released, so these breaking changes can be applied safely.	7 years ago
Brian Flad	d48d9ed766	lang/funcs: Add fileset function ... Reference: https://github.com/hashicorp/terraform/issues/16697 Enumerates a set of regular file names from a given glob pattern. Implemented via the Go stdlib `path/filepath.Glob()` functionality. Notably, stdlib does not support `**` or `{}` extended patterns. See also: https://github.com/golang/go/issues/11862 To support the extended glob patterns, it will require adding a dependency on a third party library or adding our own matching code.	7 years ago
Alex Pilon	e2eb70b737	remove satori/go.uuid in favor of google/uuid	7 years ago