603 Commits (3d211eb42ffcdc9d33d56bc2785d64b22f158c39)

Author	SHA1	Message	Date
Liam Cervante	924958bce0	plan renderer: ensure JSON strings are completely JSON (#34959)	2 years ago
Martin Atkins	c1342f4f6a	terraform test: Include test case execution time in JUnit XML ... Our currently-experimental JUnit XML output didn't previously include the "time" attribute for test runs at all, but from testing with common JUnit XML consumers we've learned that populating this is effectively mandatory because otherwise software just assumes that it's zero, producing a misleading result. We'll now populate it based on the test runner's measure of the execution time, assuming that the test runner actually measured it. If not, we'll leave it omitted and thus get the same behavior as before, which in practice means that most software will treat it as zero anyway. Because JUnit XML output is still experimental with the goal of getting feedback on exactly how we map the Terraform testing model onto that Java-oriented format, this is intentionally not supported for remote test runs in Terraform Cloud yet. It isn't really practical for Terraform Cloud to participate in short-lived Terraform CLI experiments, because Terraform Cloud has a different release schedule than Terraform CLI does. However, if this experiment is successful then we will eventually need to find a way to plumb this additional information through the JSON logs and ensure that it arrives in the locally-generated JUnit XML output.	2 years ago
Martin Atkins	c8611b51d7	terraform test: JUnit XML output sets testcase classname ... JUnit XML output from "terraform test" is currently experimental, because we're still trying to decide how best to map Terraform's testing concepts onto the Java-oriented JUnit XML format. The first round of feedback showed that some JUnit consumers completely ignore test suite names, and instead expect each test case to specify which "class" it belongs to. Those tools then might allow grouping or filtering the tests by "class". Despite the Java-oriented terminology, it seems that "classname" is really just interpreted something like "test group name", where the classname and the name together represent the full test case identifier. With that in mind then, we'll now duplicate the test suite name (which in our case is the filename of the test scenario file) into the classname attribute of each testcase element. For now we'll retain the per-suite names, always containing the same value, because we also have evidence of some consumers preferring to find this information in that location.	2 years ago
Martin Atkins	14b08c3745	terraform: Move variable validation into its separate node type ... The parent commit added a new node to the graph for each existing input variable node, but that node didn't do anything yet. Now those new nodes are actually responsible for checking the variable's validation rules, instead of that being included in the same graph node that decides the variable's final value. This also refines the validation node graph transformer to only insert a validation node if there's at least one rule to check, which should avoid bloating the graph for anyone who isn't actually using input variable validation. Therefore the plan graph builder tests revert back to their old expected result because that input configuration does not include any validation rules. For the moment this just moves around functionality that already existed and doesn't add anything new, but having the validation handled in a separate graph node means that we can evaluate the validation rules in the EvalContext of the module where the variable was declared instead of where it was defined, and so a future commit will allow -- initially just as an experiment -- validation rules that can refer to other objects in the module where the variable is declared, including the values of other input variables. As a nice bonus, this fixed a small bug in "terraform test" where problems with variable values defined in the test run were reported as problems with the variable declaration, rather than with the expression assigning the value to it. Since it's now NodeRootVariable that decides the definition source range, rather than the validation-rule-checking helper, it has access to the source range provided in a terraform.InputValue object, which was already populated correctly for per-test-run definitions.	2 years ago
Nick Fagerlund	dc9a13a975	Catch a missing err check in apply command ... Plan and refresh already had this, but apply would deref a nil opReq and panic.	2 years ago
Nick Fagerlund	1781c8e8e9	Experimental `-allow-deferral` flag for enabling deferred actions on CLI ... Now that deferred actions is a top-level per-plan option, here's a lil command-line option for turning it on! - It's called `-allow-deferral`. - It is ONLY legal to use if this is an experimental build, i.e. compiled with `go build -ldflags "-w -s -X 'main.experimentsAllowed=yes'"` or the like. Implementation notes: The design constraint here was, "please avoid having to change the build infrastructure or alter the function of every args parsing function". So, since `extendedFlagSet()` doesn't have access to the var that says we're in an experimental build, I had to move the validation for that particular flag to later in the process, when we're building the operation request.	2 years ago
Craig Sloggett	f20b055618	Prevent duplicate error messages for unset environment variables ... Track and report each unset environment variable only once, even if it's referenced multiple times in the Terraform configuration.	2 years ago
Craig Sloggett	b53db4ef14	Ensure environment variables are set when interpolating rawPath	2 years ago
Craig Sloggett	302c6deddc	Interpolate environment variables when configuring dev_overrides	2 years ago
Martin Atkins	58b48f4bab	provider/terraform: Use new convention for function names ... When we added these functions we hadn't yet settled on a single convention for how provider-contributed functions ought to be named, and so these followed the convention previously used for Terraform's own built-in functions. We've now decided to use the new provider namespace as an opportunity to partially fix the historical design error of naming functions as all lowercase without any separation between words. This makes the provider functions inconsistent with the built-in functions, but should at least hopefully encourage the provider functions to be consistent with one another, so that the rule for which to use is relatively easy to remember. This also switches the word order because the provider functions convention prefers the verb to come first: encode_tfvars instead of tfvarsencode.	2 years ago
Martin Atkins	dbde17b535	terraform console: Multi-line entry support ... The console command, when running in interactive mode, will now detect if the input seems to be an incomplete (but valid enough so far) expression, and if so will produce another prompt to accept another line of expression input. This is primarily to make it easier to paste in multi-line expressions taken from elsewhere, but it could also be used for manual input. The support for multi-line _editing_ is limited by the fact that the readline dependency we use doesn't support multiline input and so we're currently doing this in spite of that library. Hopefully we'll be able to improve on that in future either by contributing multi-line editing support upstream or by switching to a different readline dependency. The delimiter-counting heuristic employed here is similar to the one used by HCL itself to decide whether a newline should end the definition of an attribute, but this implementation is simpler because it doesn't need to produce error messages or perform any recovery. Instead, it just bails out if it encounters something strange so that the console session can return a parse error. Because some invalid input may cause a user to become "stuck" in a multi- line sequence, we consider a blank line as intent to immediately try to evaluate what was entered, and also interpret SIGINT (e.g. Ctrl+C) as cancellation of multi-line input, assuming that at least one line was already entered, extending the previous precedent that SIGINT cancels when at least one character was already entered at the prompt.	2 years ago
Liam Cervante	c6ff573180	Include provider functions in scope used to evaluate test assertions (#34825)	2 years ago
Martin Atkins	88cb4aeea3	backendrun: Separate the types/etc for backends that support operations ... We previously had all of the types and helpers for all kinds of backends together in package backend. That kept things relatively simple, but it also meant that the majority of backends that only deal with remote state storage ended up still indirectly depending on the entire Terraform modules runtime, configuration loader, etc, etc, which brings into scope a bunch of external dependencies that the remote state backends don't really need. Since backends that support operations are a rare exception, we'll move the types and helpers for those into a separate package "backendrun", and then the main package backend can have a much more modest set of types and, more importantly, a modest set of dependencies on other packages in this codebase. This is part of an ongoing effort to reduce the exposure of Terraform Core and CLI code to the remote backends and vice-versa, so that in the long run we can more often treat them as separate for dependency maintenance purposes.	2 years ago
Martin Atkins	545980d664	langrefs: Separate package for expression reference analysis ... Finding references in an expression is a static analysis operation, but we previously had it grouped in with "package lang" that primarily deals with dynamic evaluation of expressions. We need to do static analysis in far more locations than we do full expression evaluation, the static-analysis-only callers don't need anything else from package lang, and the reference-analysis functions are self-contained little wrappers around the address parsing logic in package addrs anyway. Splitting these into a separate package therefore minimizes which packages depend indirectly on package lang, and thus which packages depend indirectly on the implementations of the built-in functions. Some of the built-in functions rely on dependencies we don't use anywhere else, so cutting these out shrinks how many packages indirectly depend on those external dependencies. In particular, they are no longer in the indirect import chain for the "legacy" module.	2 years ago
Martin Atkins	5f2eab6944	providerreqs: Provider requirement/version types in their own package ... As we previously found with "package addrs", any package that contains types used as identifiers or addresses tends to get imported from all over, which means that if we mix real logic with those types in the same package then a lot more packages end up depending indirectly on our external dependencies. In this case, the getproviders package depends on various things we need only during provider installation, such as the library we use for GPG signature verification. Lots of different packages in Terraform need to talk about provider requirements, but only the CLI layer actually needs to do provider installation, and so this commit splits the types we use to talk about requirements into a new package "providerreqs", which package getproviders then depends on. To keep the size of this diff relatively small to start I've left forwarding aliases in package getproviders, which means that not all of the callers needed to be updated all at once. This commit does update all of the packages that the remote state backends depend on though, because that then allows us to reduce the set of indirect dependencies for each of those backends. The nested module go.mod files show that this has disconnected the indirect dependencies on github.com/ProtonMail/go-crypto and on github.com/cloudflare/circl , both of which Terraform CLI uses only for provider installation.	2 years ago
Martin Atkins	66bec6c3c7	command: TestWorkspace_deleteWithState uses modern state format ... This test is intending to test behavior in the presence of a state snapshot but was achieving that using the legacy state snapshot format that we preserved only for its previous (now eliminated, in an earlier commit) purpose of implementing the CLI backend state format. Instead, we'll construct a state snapshot format using the modern API for doing that, which therefore constructs a modern state snapshot file as would be created by today's Terraform. This eliminates the last use of the "legacy/terraform" package aside from the calls from other packages under "legacy".	2 years ago
Martin Atkins	06f51dbe04	command/workdir: Model the "backend state" file format ... Historically (before there were "backends") Terraform had a single state file format used both for real state snapshots and for tracking where remote state was stored. Terraform v0.12 caused these two to be split because we adopted state snapshot version 4 for real snapshots but retained a subset of version 3 for tracking the remote backend configuration in the local working directory. Unfortunately we previously kept that working by retaining a snapshot of the entire Terraform v0.11 "terraform" package as legacy/terraform, which happened to still be around because we also needed to retain a copy of the entire legacy SDK to keep the remote state backends working. This now hoists just the tiny slice of legacy terraform package functionality needed to implement the "backend state" file format into package workdir. This package is a good home for it because it's part of the working directory state. Ideally it would be accessed through methods of the workdir.Dir type, but that's too disruptive a refactor to combine into this and so that'll need to wait for another day; for now we'll keep the existing callers doing their access through our "clistate" package that is itself a forked snapshot of what statemgr.Filesystem used to be in Terraform v0.11. This removes all but one of the uses of "legacy/terraform" aside from calls in the other packages under "legacy". We'll clean up the last one in a later commit, because it's not related to the backend state file format.	2 years ago
Martin Atkins	1358245617	dependencies: Remove all non-legacy uses of hashicorp/go-multierror ... This functionality is now provided by the "errors" package in the standard library, so the HashiCorp-specific library is obsolete. This also removed our one direct use of github.com/pkg/errors, which we were using in a way that was totally redundant with the stdlib package errors. However, it lives on as an indirect dependency through the Consul backend's dependency on the Consul SDK, which uses that library.	2 years ago
thirdkeyword	8284cde302	remove repetitive words ... Signed-off-by: thirdkeyword <fliterdashen@gmail.com>	2 years ago
Mark DeCrane	57d2802053	Merge pull request #34746 from hashicorp/modify-dev-override-warn-with-backend ... Modify provider dev override warning to include a remote backend specific line	2 years ago
Mark DeCrane	b83275f05d	chore: remove extra newline char	2 years ago
Mark DeCrane	51ae7d32a7	fix: check for local operations, separate warning for remote execution	2 years ago
Mark DeCrane	9580ee27f9	chore: add remote backend specfic warnings to dev override warning	2 years ago
Nick Fagerlund	c9c24706aa	Merge pull request #34741 from hashicorp/nf/feb24-provider-addrs-in-statuses ... Stacks, Hooks: Include full provider addresses	2 years ago
James Bardin	dada9d683e	Merge pull request #34740 from hashicorp/jbardin/render-string-null-changes ... try to render some changes between `""` and `null`	2 years ago
Nick Fagerlund	ccab5f1e55	Rename some UiHook tests for easier ad-hoc runs	2 years ago
Nick Fagerlund	94ea534e40	Refactor terraform.Hook to use a resource-identifying wrapper struct ... The terraform.Hook interface lets other areas of code perform streaming reactions to various events, generally in the service of some UI somewhere. Nearly all of the methods on this interface take an `addrs.AbsResourceInstance` as their first argument, to identify the resource that's being operated on. However, that addrs struct doesn't necessarily contain everything you might want in order to uniquely and usefully identify a resource. It has the module instance and resource instance addresses, but it lacks the provider source address, which can affect how the consuming UI should display the resource's events. (For example, Terraform Cloud wants reliable info about who maintains a given provider, what cloud provider it operates on, and where to find its documentation.) Instead of polluting `addrs.AbsResourceInstance` with extra information that isn't relevant to other call sites, let's change the first argument of each Hook method to be a wrapper struct defined in the package that owns the Hook interface, and add the provider address to that wrapper as a sibling of the resource address. This causes a big noisy commit today, but should streamline future updates to the UI-facing "identity" of a resource; existing callers can ignore any new fields they're uninterested in, or exploit new info as needed. Other than making new information available for future edits to Hook implementing types, this commit should have no effect on existing behavior.	2 years ago
Nick Fagerlund	d81a546e55	Lints: hook_count_test, "redundant type from slice composite literal" ... I was in the file for other reasons, and wanted to reduce noise in my editor.	2 years ago
Martin Atkins	573c2db658	provider/terraform: Terraform-specific encoding functions ... Using the new possibility of provider-contributed functions, this introduces three new functions which live in the terraform.io/builtin/terraform provider, rather than being language builtins, due to their Terraform-domain-specific nature. The three new functions are: - tfvarsencode: takes a mapping value and tries to transform it into Terraform CLI's "tfvars" syntax, which is a small subset of HCL that only supports key/value pairs with constant values. - tfvarsdecode: takes a string containing content that could potentially appear in a "tfvars" file and returns an object representing the raw variable values defined inside. - exprencode: takes an arbitrary Terraform value and produces a string that would yield a similar value if parsed as a Terraform expression. All three of these are very specialized, of use only in unusual situations where someone is "gluing together" different Terraform configurations etc when the usual strategies such as data sources are not suitable. There's more information on the motivations for (and limitations of) each function in the included documentation.	2 years ago
James Bardin	bdd5a2cd33	try to render some empty string -> null changes ... Changes between empty strings and `null` were hidden in the CLI output, because the SDK could not reliably detect the difference and may return either value depending on the situation. This legacy behavior can be confusing for authors of new provider which can correctly handle `null`, and it would be preferable to be able to render those changes in the CLI. While we don't have enough information to detect when the legacy behavior is required, we can detect a number of cases where it's certain that we are not dealing with a legacy schema and should output the full diff.	2 years ago
Daniel Schmidt	8564e934aa	Merge pull request #34699 from hashicorp/terraform-test-use-vars-within-blocks ... test: allow using global variables in suite-level variable definitions	2 years ago
Daniel Schmidt	0529299cc3	run terraform fmt on test	2 years ago
Liam Cervante	c46515c5da	stacks: add support for external providers to validate context (#34701)	2 years ago
Daniel Schmidt	7ae7a203d8	test: allow using global variables in suite-level variable definitions ... Closes #34534	2 years ago
Liam Cervante	82361afd73	plan rendering: fix truncation of very large numbers (#34702)	2 years ago
Daniel Schmidt	adf008d489	adjust tests to new implementation	2 years ago
Daniel Schmidt	19f9bdac14	handle multi-line string diffs correctly	2 years ago
Daniel Schmidt	a64df71152	fix comment	2 years ago
Daniel Schmidt	1531fee472	clarify comment	2 years ago
Daniel Schmidt	2a625b75a4	handle reordering explicitly	2 years ago
Daniel Schmidt	31bd2b5be0	plan: compare individual elements if list is same length ... Closes #33779	2 years ago
Liam Cervante	f0b61090a2	Move the testing MockProvider into a dedicated shared package (#34681) ... * move the testing MockProvider into a dedicated shared package * address comments * fix new changes	2 years ago
Martin Atkins	884e1fb2a4	terraform: Plans can be "complete" and "applyable" ... These ideas are both already implied by some logic elsewhere in the system, but until now we didn't have the decision logic centralized in a single place that could therefore evolve over time without necessarily always updating every caller together. We'll now have the modules runtime produce its own boolean ruling about each characteristic, which callers can rely on for the mechanical decision-making of whether to offer the user an "approve" prompt, and whether to remind the user after apply that it was an incomplete plan that will probably therefore need at least one more plan/apply round to converge. The "Applyable" flag directly replaces the previous method Plan.CanApply, with equivalent logic. Making this a field instead of a method means that we can freeze it as part of a saved plan, rather than recalculating it when we reload the plan, and we can export the field value in our export formats like JSON while ensuring it'll always be consistent with what Terraform is using internally. Callers can (and should) still use other context in the plan to return more tailored messages for specific situations they already know about that might be useful to users, but with these flags as a baseline callers can now just fall back to a generic presentation when encountering a situation they don't yet understand, rather than making the wrong decision and causing something strange to happen. That is: a lack of awareness of a new rule will now cause just a generic message in the UI, rather than incorrect behavior. This commit mostly just deals with populating the flags, and then all of the direct consequences of that on our various tests. Further changes to actually make use of these flags elsewhere in the system will follow in later commits, both in this repository and in other repositories.	2 years ago
Daniel Schmidt	91ea23aee5	remove outdated comment	2 years ago
Daniel Schmidt	14feb738a7	update comment	2 years ago
Daniel Schmidt	472a07ad6e	fix typo ... Co-authored-by: Liam Cervante <liam.cervante@hashicorp.com>	2 years ago
Daniel Schmidt	be262b52d7	make using the plugin cache opt-in	2 years ago
Daniel Schmidt	f720a060ff	use plugin cache for provider lock across for all platforms ... The cache architecture already supports multi-platform, the only downside is that this might grow the cache. This seems expected.	2 years ago
Daniel Schmidt	2c98e6a347	improve provider lock speed by using cache ... Closes #33837	2 years ago
Martin Atkins	89a53e9105	command/test: Fix invalid XML processing instruction ... Annoyingly, I seem to have flubbed this slightly while I was adapting to use enc.EncodeToken instead of (as originally written) just writing the literal processing instruction bytes onto the front of the buffer directly. This made the output invalid for any XML processor that interprets the special "xml" processing instruction. It'll now be valid again.	2 years ago