60 Commits (add-missing-variable-loading)

Author	SHA1	Message	Date
James Bardin	7960f60d21	Merge pull request #38160 from hashicorp/jbardin/inline-type-conversion ... inline type conversions	1 month ago
Radek Simko	0fe906fa8c	make copyrightfix	2 months ago
James Bardin	9ce50aa8a6	add convert function ... The convert function allows for inline type conversions within the Terraform language. A custom decoder for the function which allows for type expression literals as the second parameter.	2 months ago
Daniel Schmidt	2714cfde69	standardize on marks.Has	6 months ago
Liam Cervante	2a1b4c3ed4	Update go-cty and HCL for 1.13 release (#37298)	9 months ago
James Bardin	f78adc889e	add wrappers to verify file function results ... Create wrappers to inject function results verification into all the filesystem function implementations.	11 months ago
Samsondeen	08fa94c2ed	Return error when the template's single interpolation results in null value (#36658)	1 year ago
Radek Simko	473e5931ac	lang/funcs/transpose: Avoid crash due to map with `null` values (#36611) ... * lang/funcs/transpose: Avoid crash on nulls in input * add changelog entry	1 year ago
Radek Simko	289a3568da	Address `go vet` 1.24 failures (#36485) ... * address go vet failures * avoid unnecessary .Error()	1 year ago
Daniel Schmidt	6b49e80b50	ephemeral: add helper to remove ephemeral values from cty.Values	1 year ago
James Bardin	5ccb3c7c7a	templatefile need to be unmarked for TypeFunc ... The templatefile function actually calls the full renderer within the TypeFunc to determine what the resulting might be, but that also requires unmarking the top-level of the vars map to strip marks even though the value is unused.	1 year ago
James Bardin	833b07436f	template variables may be entirely unknown ... While the evaluator can deal with unknown template variables, if the entire map of variables is unknown, we can't create the map and need to short-circuit the call.	1 year ago
James Bardin	b4c24c22f8	issensitive must return unknown for unknown args ... Terraform attempts to track marks as accurately as possible, but unknown values may not always have the same marks as they will when they become known. This is most easily seen with functions, which are allowed to return an unknown value when faced with any unknown arguments, while they are also allowed to manipulate the marks on the values as they see fit. This results in situations where the marks simply cannot be known. Terraform generally takes the stance that if an unknown has a mark, it will remain in the final value, but the absence of a mark is not indicative of the absence of any marks in the final value.	1 year ago
James Bardin	ac54255d68	unknown+ephem in length+lookup	1 year ago
James Bardin	2ae01741cd	handle marked-unknowns in number funcs	1 year ago
James Bardin	651a80fb57	deal with unknown marked vals in fs funcs ... Filesystem functions accepts marked values, and must also deal with unknowns.	1 year ago
James Bardin	a51be5e930	lang functions must deal with unknown and marks ... You can't deal with marks in a function without also dealing with unknown. If AllowMarked is true, but AllowKnown is not, the caller will short-circuit the evaluation with an unknown, without being able to deal with marks from all arguments.	1 year ago
James Bardin	c4741dcd4c	there are other marks besides sensitive ... The sensitive function must not strip off other marks	1 year ago
James Bardin	1e011248ff	Sensitive is not the only possible mark ... This has not applied within Terraform for a long time, but the sensitive function was never tested with extra marks.	1 year ago
James Bardin	ca7b3c0c05	plantimestamp can be unknown during validation ... Before a timestamp is assigned during plan, the `time.Time` value will be the zero value for that type. This represents a case where the timestamp is not yet known, but the handling of that timestamp isn't using our usual cty value system for representing unknowns. Rather than refactor the use of the timestamp itself throughout the code, we can simply have the function return an unknown if it encounters a zero value.	1 year ago
Martin Atkins	71d14e78fd	lang: Experimental "ephemeralasnull" function ... This is another part of the existing ephemeral_values experiment, taking a value of any type that might have ephemeral values in it and returning a value of the same type which has any ephemeral value replaced with a null value. The primary purpose of this is to allow a module to conveniently return an object that would normally contain nested ephemeral values -- such as an instance of a managed resource type that has a write-only attribute -- through an output value that isn't declared as ephemeral. This would then expose all of the non-ephemeral parts of the object but withhold the ephemeral parts. In the case of write-only attributes, it exposes the normal attributes while withholding the write-only ones. The name of this function could potentially change before stabilization, because it's quite long and clunky. I did originally consider "nonephemeral" to match with the existing "nonsensitive", but that didn't feel right because "nonsensitive" removes the sensitive mark while preserving the underlying value while this function removes the mark and the real value at the same time. (It would not be appropriate to have a function that just removes the ephemeral mark while preserving the value, because correct handling of ephemerality is important for correctness while sensitivity is primarily a UI concern so we don't need to be quite so picky about it.)	2 years ago
Martin Atkins	e23e6b84c7	lang/funcs: Allow some more expression types in templatestring ... The templatestring function has some special constraints on its first argument that are included to add some intentional friction for those who are new to Terraform, want to do some simple template rendering, but have only found the templatestring function so far. We know from previous experience with the hashicorp/template provider that this sort of functionality tends to attract those who haven't yet learned that the Terraform language has built-in support for string templates (without calling any function), who would then get confused by the need for an extra level of escaping to render a template string only indirectly through this function. However, this rule is not intended to be onerous and require writing the rest of the containing module in an unnatural way to work around it, so here we loosen the rule to allow some additional forms: - An index expression whose collection operand meets these rules. - A relative traversal whose source operand meets these rules. In particular this makes it possible to write an expression like: data.example.example[each.key].result ...which is a relative traversal from an index from a scope traversal, and is a very reasonable thing to write if you've retrieved multiple templates using a data resource that uses for_each. This also treats splat expressions in the same way as index expressions at the static check stage, but that's only to allow us to reach the dynamic type check that will ultimately report that a string is required, because the result of a splat expression is a tuple. The type-related error message is (subjectively) more helpful/relevant than the syntax-related one for this case. Finally, this includes some revisions to the documentation for this function to correct some editing errors from the first pass and to slightly loosen the language about what's allowed. It's still a little vague about what exactly is allowed, but I'm doubtful that a precise definition in terms of HCL's expression types would be very enlightening for a typical reader anyway. We can tweak the specificity of the language here if we start to see repeated questions about what is and is not valid.	2 years ago
kmoe	1e2d4a2ecc	lang: stabilise templatestring func experiment (#35224) ... * lang: stabilise templatestring func experiment * command/jsonfunction: marshal templatestring * docs: add templatestring	2 years ago
Martin Atkins	9dd28fc2e5	funcs: Don't panic if templatefile path is sensitive ... Previously we were partially propagating any marks from the path, but not going all the way so we still ran into trouble when trying to use the string containing the file contents. Now we'll have loadTmpl also return the marks it had to read through to actually parse the template, and then we'll use those (instead of the original path marks) to mark the result. In practice the pathMarks and the tmplMarks should always match today, but this is intentionally structured to make the data flow clearer -- the marks always travel along with whatever they related to -- so we're less likely to break this accidentally under future maintenence.	2 years ago
Martin Atkins	edf335e7eb	lang/funcs: Experimental "templatestring" function ... This function complements the existing "templatefile" to deal with the unusual situation of rendering a template that comes from somewhere outside of the current module's source code, such as from a data resource result. We have some historical experience with the now-deprecated hashicorp/template provider and its template_file data source, where we found that new authors would find it via web search and assume it was "the way" to render templates in Terraform, and then get frustrated dealing with the confusing situation of writing a string template that generates another string template for a second round of template rendering. To try to support those who have this unusual need without creating another attractive nuisance that would derail new authors, this function imposes the artificial extra rule that its template argument may only be populated using a single reference to a symbol defined elsewhere in the same module. This is intended to entice folks trying to use this function for something other than its intended purpose to refer to its documentation (once written) and then hopefully learn what other Terraform language feature they ought to have used instead. The syntax restriction only goes one level deep, so particularly-determined authors can still intentionally misuse this function by adding one level of indirection, such as by building template source code in a local value and then passing that local value as the template argument. The restriction is in place only to reduce the chances of someone _misunderstanding_ the purpose of this function; we don't intend to prevent someone from actively deciding to misuse it, if they have a good reason to do so. This new function inherits the same restriction as templatefile where it does not allow recursively calling other template-rendering functions. This is to dissuade from trying to use Terraform templates "at large", since Terraform's template language is not designed for such uses. It would be better to build a Terraform provider that wraps a more featureful template system like Gonja if someone really does need advanced templating, beyond Terraform's basic goals of being able to build small configuration files, etc. Because this function's intended purpose is rendering templates obtained from elsewhere, this function also blocks calls to any of Terraform's functions that would read from the filesystem of the computer where Terraform is running. This is a small additional measure of isolation to reduce the risk of an attacker somehow modifying a dynamically-fetched template to inspire Terraform to write sensitive data from the host computer into a location accessible to the same attacker, or similar. This is currently only a language experiment and so will not yet be available in stable releases of Terraform. Before stabilizing this and committing to supporting it indefinitely we'll want to gather feedback on whether this function actually meets the intended narrow set of use-cases around dynamic template rendering.	2 years ago
Ansgar Mertens	cd1306430b	fix(docs): Include product in relative link (#34510) ... CDKTF uses these definitions to generate docs which end up on developer.hashicorp.com There are content checks that fail for this link as the product (terraform) is required to be explicitly stated now	2 years ago
Daniel Schmidt	3a77065e69	add `issensitive` function ... This function can be used to check if a value is marked as sensitive or not within terraform. Closes #34572	2 years ago
James Bardin	1644c2f63b	check function descriptions directly ... Now that we dynamically add scoped versions to the global functions we can't check the static Descriptions list. This should be OK since the function descriptions are taken directly from Descriptions list, and if any other scoped functions were added in the future, they may not be coming from the current global set of functions anyway.	2 years ago
Martin Atkins	d1df979115	lang: Expose built-in functions under core:: namespace ... This essentially doubles up the registrations for all of the built-in functions to have both non-namespaced and core::-namespaced versions of each function. This is in preparation for later commits introducing other namespaces, such as a provider:: namespace which could hold functions that were contributed by providers that are currently in scope.	2 years ago
Joao C Costa	47beda901e	nonsensitive: no longer produces error when applied to nonsensitive values (#33856) ... * Proposing fix to #31693 (not always possible to apply nonsensitive) https://github.com/hashicorp/terraform/issues/31693 * hashicorp#31693 not always possible to apply nonsensitive updated documentation * Update sensitive_test.go Fixing tests * Corrections as per https://github.com/hashicorp/terraform/pull/33856#discussion_r1330042125	2 years ago
CJ Horton	98b81d3c50	add missing copyright headers	3 years ago
hashicorp-copywrite[bot]	53c34ff49c	Update copyright file headers to BUSL-1.1	3 years ago
Martin Atkins	81c15f987e	lang/funcs: startswith considers string prefix refinement ... If the string to be tested is an unknown value that's been refined with a prefix and the prefix we're being asked to test is in turn a prefix of that known prefix then we can return a known answer despite the inputs not being fully known. There are also some other similar deductions we can make about other combinations of inputs. This extra analysis could be useful in a custom condition check that requires a string with a particular prefix, since it can allow the condition to fail even on partially-unknown input, thereby giving earlier feedback about a problem.	3 years ago
Martin Atkins	c912970153	lang/funcs: Non-null refinements for various functions ... cty's new "refinements" concept allows us to reduce the range of unknown values from our functions. This initial changeset focuses only on declaring which functions are guaranteed to return a non-null result, which is a helpful baseline refinement because it allows "== null" and "!= null" tests to produce known results even when the given value is otherwise unknown. This commit also includes some updates to test results that are now refined based on cty's own built-in refinement behaviors, just as a result of us having updated cty in the previous commit.	3 years ago
hashicorp-copywrite[bot]	325d18262e	[COMPLIANCE] Add Copyright and License Headers	3 years ago
Zac Clifton	3bb6a5896b	Feat: Add strcontains function and documentation (#33069) ... * add strcontains function and documentation	3 years ago
Liam Cervante	aae6990d38	Add native `plantimestamp` function to Terraform (#32980) ... * add plantimestamp function * documentation * add missing links * fix typo	3 years ago
Martin Atkins	9f827f57ae	go.mod: Revert cty minor version v1.13 upgrade ... We inadvertently incorporated the new minor release of cty into the 1.4 branch, and that's introduced some more refined handling of unknown values that is too much of a change to introduce in a patch release. Therefore this reverts back to the previous minor release for the v1.4 series, and then we'll separately get the main branch ready to work correctly with the new cty before Terraform v1.5. This reverts just the upgrade and the corresponding test changes from #32775, while retaining the HCL upgrade and the new test case it introduced for that bug it was trying to fix. That new test is still passing so it seems that the cty upgrade is not crucial to that fix.	3 years ago
Liam Cervante	32f151f20b	Update HCL to handle type mismatches when applying defaults (#32775) ... * add test reproducing terraform crash * pull latest hcl * add missing go.sum file * also fix tests broken by go-cty update	3 years ago
Daniel Banck	f29156cdef	Replace HTML entities in function descriptions (#32710)	3 years ago
Daniel Banck	4fd8322802	Add function descriptions (#32453) ... * Add consolidated function description list * Add function parameter descriptions * Add descriptions to all functions * Add sanity test for function descriptions * Apply suggestions from code review Co-authored-by: kmoe <5575356+kmoe@users.noreply.github.com> Co-authored-by: kmoe <5575356+kmoe@users.noreply.github.com>	3 years ago
Martin Atkins	2ee9589650	lang/funcs: "timecmp" function ... This is a complement to "timestamp" and "timeadd" which allows establishing the ordering of two different timestamps while taking into account their timezone offsets, which isn't otherwise possible using the existing primitives in the Terraform language.	4 years ago
Martin Atkins	783a07d9e8	build: Use Go 1.19 ... Go 1.19's "fmt" has some awareness of the new doc comment formatting conventions and adjusts the presentation of the source comments to make it clearer how godoc would interpret them. Therefore this commit includes various updates made by "go fmt" to acheve that. In line with our usual convention that we make stylistic/grammar/spelling tweaks typically only when we're "in the area" changing something else anyway, I also took this opportunity to review most of the comments that this updated to see if there were any other opportunities to improve them.	4 years ago
Joshua Barton	c73c0ad63c	feat: add startswith and endswith funcs	4 years ago
Alisdair McDiarmid	718b0875ef	lang: Remove defaults function ... Now that we are able to specify optional object attribute defaults inline in a type constraint, the separate `defaults` function is no longer needed.	4 years ago
Martin Atkins	d4776e8ef1	lang/funcs: type conversion functions can convert null values ... We had intended these functions to attempt to convert any given value, but there is a special behavior in the function system where functions must opt in to being able to handle dynamically-typed arguments so that we don't need to repeat the special case for that inside every function implementation. In this case we _do_ want to specially handle dynamically-typed values, because the keyword "null" in HCL produces cty.NullVal(cty.DynamicPseudoType) and we want the conversion function to convert it to a null of a more specific type. These conversion functions are already just a thin wrapper around the underlying type conversion functionality anyway, and that already supports converting dynamic-typed values in the expected way, so we can just opt in to allowing dynamically-typed values and let the conversion functionality do the expected work. Fixing this allows module authors to use type conversion functions to give additional type information to Terraform in situations that are too ambiguous to be handled automatically by the type inference/unification process. Previously tostring(null) was effectively a no-op, totally ignoring the author's request to treat the null as a string.	4 years ago
shinbunbun	a1ead44c71	cidrnetmask() produce an error with IPv6 (#30703) ... * Add error handling for IPv6 Co-authored-by: kmoe <5575356+kmoe@users.noreply.github.com>	4 years ago
Alisdair McDiarmid	0764726e3e	functions: Fix sum() of all strings ... The sum() function accepts a collection of values which must all convert to numbers. It is valid for this to be a collection of string values representing numbers. Previously the function would panic if the first element of a collection was a non-number type, as we didn't attempt to convert it to a number before calling the cty `Add` method.	4 years ago
Alisdair McDiarmid	843c50e8ce	lang: Further limit the console-only type function ... This commit introduces a capsule type, `TypeType`, which is used to extricate type information from the console-only `type` function. In combination with the `TypeType` mark, this allows us to restrict the use of this function to top-level display of a value's type. Any other use of `type()` will result in an error diagnostic.	4 years ago
Alisdair McDiarmid	903d6f1055	lang: Remove use of marks.Raw in tests ... These instances of marks.Raw usage were semantically only testing the properties of combining multiple marks. Testing this with an arbitrary value for the mark is just as valid and clearer.	4 years ago