391 Commits (9d815ea7271d56a43fc45fdff186e57bfb032a06)

Author	SHA1	Message	Date
Martin Atkins	fda0579537	Experiments supported only in alpha/dev builds ... We originally introduced the idea of language experiments as a way to get early feedback on not-yet-proven feature ideas, ideally as part of the initial exploration of the solution space rather than only after a solution has become relatively clear. Unfortunately, our tradeoff of making them available in normal releases behind an explicit opt-in in order to make it easier to participate in the feedback process had the unintended side-effect of making it feel okay to use experiments in production and endure the warnings they generate. This in turn has made us reluctant to make use of the experiments feature lest experiments become de-facto production features which we then feel compelled to preserve even though we aren't yet ready to graduate them to stable features. In an attempt to tweak that compromise, here we make the availability of experiments _at all_ a build-time flag which will not be set by default, and therefore experiments will not be available in most release builds. The intent (not yet implemented in this PR) is for our release process to set this flag only when it knows it's building an alpha release or a development snapshot not destined for release at all, which will therefore allow us to still use the alpha releases as a vehicle for giving feedback participants access to a feature (without needing to install a Go toolchain) but will not encourage pretending that these features are production-ready before they graduate from experimental. Only language experiments have an explicit framework for dealing with them which outlives any particular experiment, so most of the changes here are to that generalized mechanism. However, the intent is that non-language experiments, such as experimental CLI commands, would also in future check Meta.AllowExperimentalFeatures and gate the use of those experiments too, so that we can be consistent that experimental features will never be available unless you explicitly choose to use an alpha release or a custom build from source code. Since there are already some experiments active at the time of this commit which were not previously subject to this restriction, we'll pragmatically leave those as exceptions that will remain generally available for now, and so this new approach will apply only to new experiments started in the future. Once those experiments have all concluded, we will be left with no more exceptions unless we explicitly choose to make an exception for some reason we've not imagined yet. It's important that we be able to write tests that rely on experiments either being available or not being available, so here we're using our typical approach of making "package main" deal with the global setting that applies to Terraform CLI executables while making the layers below all support fine-grain selection of this behavior so that tests with different needs can run concurrently without trampling on one another. As a compromise, the integration tests in the terraform package will run with experiments enabled _by default_ since we commonly need to exercise experiments in those tests, but they can selectively opt-out if they need to by overriding the loader setting back to false again.	4 years ago
Alisdair McDiarmid	c7bc82bd87	Merge pull request #31235 from hashicorp/alisdair/json-plan-unknown-outputs ... json-output: Extended detail for unknown outputs	4 years ago
Alisdair McDiarmid	9497b2cd6f	json-output: Fix unknowns for tuples and sets ... The JSON output for sequences previously omitted unknown values for tuples and sets, which made it impossible to interpret the corresponding unknown marks. For example, consider this resource: resource "example_resource" "example" { tags = toset(["alpha", timestamp(), "charlie"]) } This would previously be encoded in JSON as: "after": { "tags": ["alpha", "charlie"] }, "after_unknown": { "id": true, "tags": [false, true, false] }, That is, the timestamp value would be omitted from the output altogether, while the corresponding unknown marks would include a value for each of the set members. This commit changes the behaviour to: "after": { "tags": ["alpha", null, "charlie"] }, "after_unknown": { "id": true, "tags": [false, true, false] }, This aligns tuples and sets with the prior behaviour for lists, and makes it clear which elements are known and which are unknown.	4 years ago
Alisdair McDiarmid	48d64eabb2	json-output: Extended detail for unknown outputs ... Planned output changes are represented in the JSON output format using the same change object as planned resource changes. This structure includes an `after` value and a parallel `after_unknown` value, which can be combined to determine which specific parts of a value are known only at apply time. Previously, structured output values would be marked in the JSON plan as coarsely known or unknown, even if only some subset of the structure will be known only at apply time. This simplification was unnecessary, and this commit reuses the same logic for resource changes to give more information to consumers of this format. For example, consider this output: output "bar" { value = tolist([ "hello", timestamp(), "world", ]) } The plan output for this output would be: + bar = [ + "hello", + (known after apply), + "world", ] For the same plan, the JSON output was previously: "bar": { "actions": [ "create" ], "before": null, "after_unknown": true, "before_sensitive": false, "after_sensitive": false } After this commit, the output is instead: "bar": { "actions": [ "create" ], "before": null, "after": [ "hello", null, "world" ], "after_unknown": [ false, true, false ], "before_sensitive": false, "after_sensitive": false }	4 years ago
James Bardin	47f9850f02	add XTerraformGetLimit to prevent redirect loops	4 years ago
kmoe	54b837416c	command: mention plan options in refresh help text (#30892) ... * command: add parallelism option to refresh help text	4 years ago
Alisdair McDiarmid	06f6a901ea	Merge pull request #30985 from hashicorp/alisdair/var-flag-spaces-error ... cli: Improved error for invalid -var "foo = bar"	4 years ago
Martin Atkins	096f0dc0da	build: Write the detected version number into the generated executable ... This also sets an additional variable if it detects that this is an alpha or development build, which currently does nothing but might eventually turn on the ability to use experimental features, if we make that something available only in prereleases.	4 years ago
Craig Wright	be7f46a1ad	Merge pull request #29354 from srdecny/main ... Fix CLI help text for output. Reviewed with @apparentlymart and @kmoe.	4 years ago
Martin Atkins	4cffff24b1	core: Report reason for deferring data read until apply ... We have two different reasons why a data resource might be read only during apply, rather than during planning as usual: the configuration contains unknown values, or the data resource as a whole depends on a managed resource which itself has a change pending. However, we didn't previously distinguish these two in a way that allowed the UI to describe the difference, and so we confusingly reported both as "config refers to values not yet known", which in turn led to a number of reasonable questions about why Terraform was claiming that but then immediately below showing the configuration entirely known. Now we'll use our existing "ActionReason" mechanism to tell the UI layer which of the two reasons applies to a particular data resource instance. The "dependency pending" situation tends to happen in conjunction with "config unknown", so we'll prefer to refer that the configuration is unknown if both are true.	4 years ago
Alisdair McDiarmid	91d75baba1	cli: Improved error for invalid -var "foo = bar" ... When specifying variable values on the command line, name-value pairs must be joined with an equals sign, without surrounding spaces. Previously Terraform would interpret "foo = bar" as assigning the value " bar" to the variable named "foo ". This is never valid, as variable names may not include whitespace. This commit looks for this specific error and returns a diagnostic with a suggestion for correcting it. We cannot simply trim whitespace, because it is valid to write "foo= bar" to assign the value " bar" to the variable "foo", as unlikely as it seems.	4 years ago
Alisdair McDiarmid	12c8f9498c	json-output: Add output type to JSON format ... Previously the supported JSON plan and state formats included only serialized output values, which was a lossy serialization of the Terraform type system. This commit adds a type field in the usual cty JSON format, which allows reconstitution of the original value. For example, previously a list(string) and a set(string) containing the same values were indistinguishable. This change serializes these as follows: { "value": ["a","b","c"], "type": ["list","string"] } and: { "value": ["a","b","c"], "type": ["set","string"] }	4 years ago
James Bardin	7da52d94f2	Merge pull request #30900 from hashicorp/jbardin/replace-triggered-by ... Configurable instance replacement via lifecycle `replace_triggered_by`	4 years ago
James Bardin	c52e3ed37b	test fixture race	4 years ago
James Bardin	3a0a019521	round-trip replace triggers	4 years ago
James Bardin	868052c9e3	set replace_trigered_by reason in diff output	4 years ago
Brandon Croft	1943af51a2	fix(creds): allow periods in TF_TOKEN_... credentials vars	4 years ago
Brandon Croft	f04202d222	Update credentials_test.go	4 years ago
Brandon Croft	fda05f3d44	feat(credentials): allow hyphens to be encoded as __ in variables ... Hyphen characters are allowed in environment variable names, but are not valid POSIX variable names. Usually, it's still possible to set variable names with hyphens using utilities like env or docker. But, as a fallback, host names may encode their hyphens as double underscores in the variable name. For the example "café.fr", the variable name "TF_TOKEN_xn____caf__dma_fr" or "TF_TOKEN_xn--caf-dma_fr" may be used.	4 years ago
Brandon Croft	307326fa3a	allow remote service creds to be configured using env ... Introduces a new method of configuring token service credentials using a host-specific environment variable. This configuration was previously possible using the [terraform-credentials-env](https://github.com/apparentlymart/terraform-credentials-env) credentials helper. This new method is now consulted first, as it is seen to be the most proximate source of credentials before CLI configuration while still falling back to the credentials helper.	4 years ago
Alisdair McDiarmid	fadcaaaad2	cli: Fix double-quoted map keys in diff UI ... A previous change added missing quoting around object keys which do not parse as barewords. At the same time we introduced a bug where map keys could be double-quoted, due to calling the `displayAttributeName` helper function (to quote non-bareword keys) then using the `writeValue` method (which quotes all strings). This commit fixes this and adds test coverage for map keys which require quoting.	4 years ago
Alisdair McDiarmid	d23f0998fb	cli: Fix plan diff for sensitive nested attributes ... When rendering diffs for resources which use nested attribute types, we must cope with collections backing those attributes which are entirely sensitive. The most common way this will be seen is through sensitive values being present in sets, which will result in the entire set being marked sensitive.	4 years ago
Eng Zer Jun	fedd315275	test: use `T.TempDir` to create temporary test directory (#30803) ... This commit replaces `ioutil.TempDir` with `t.TempDir` in tests. The directory created by `t.TempDir` is automatically removed when the test and all its subtests complete. Prior to this commit, temporary directory created using `ioutil.TempDir` needs to be removed manually by calling `os.RemoveAll`, which is omitted in some tests. The error handling boilerplate e.g. defer func() { if err := os.RemoveAll(dir); err != nil { t.Fatal(err) } } is also tedious, but `t.TempDir` handles this for us nicely. Reference: https://pkg.go.dev/testing#T.TempDir Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>	4 years ago
Alisdair McDiarmid	bb35f02c95	Conclude preconditions/postconditions experiment	4 years ago
Alisdair McDiarmid	c5d10bdef1	core: Store condition block results in plan ... In order to include condition block results in the JSON plan output, we must store them in the plan and its serialization. Terraform can evaluate condition blocks multiple times, so we must be able to update the result. Accordingly, the plan.Conditions object is a map with keys representing the condition block's address. Condition blocks are not referenceable in any other context, so this address form cannot be used anywhere in the configuration. The commit includes a new test case for the JSON output of a refresh-only plan, which is currently the only way for a failing condition result to be rendered through this path.	4 years ago
Martin Atkins	49d7c879ac	Fix problems caught by staticcheck v0.3.0 ... This will allow us to upgrade to this version in a later commit without causing the our build checks to fail.	4 years ago
kmoe	5907a86301	command/format: Correctly quote diff object keys (#30766) ... When rendering a diff, we should quote object attribute names if the string representation is not a valid identifier. While this is not strictly necessary, it makes the diff output more closely resemble the configuration language, which is less confusing. This commit applies to both top-level schema attributes and any object value attributes. We use a simplistic "%q" Go format string to quote the strings, which is not strictly identical to HCL's quoting requirements, but is the pattern used elsewhere in HCL and Terraform. Co-Authored-By: Katy Moe <katy@katy.moe> Co-authored-by: Alisdair McDiarmid <alisdair@users.noreply.github.com>	4 years ago
James Bardin	0bd59238d6	Merge pull request #30765 from hashicorp/jbardin/contributing-attributes ... improve the contributing attributes filter	4 years ago
UKEME BASSEY	1c9929a378	Merge pull request #30773 from hashicorp/uk1288/fix-for-cloud-integration-panic ... command/meta_backend: fix for cloud integration panic	4 years ago
gabriel376	f5a8608989	cli: Make state commands check required version (#30511)	4 years ago
uk1288	9093b487fd	fix for cloud integration panic	4 years ago
James Bardin	09dde8d5ed	improve the contributing attributes filter ... The initial rough implementation contained a bug where it would incorrectly return a NilVal in some cases. Improve the heuristics here to insert null values more precisely when parent objects change to or from null. We also check for dynamic types changing, in which case the entire object must be taken when we can't match the individual attribute values.	4 years ago
Matthias Baur	971c899c12	Clarify multiple times usage (#30724) ... * Clarify multiple times usage Co-authored-by: kmoe <5575356+kmoe@users.noreply.github.com>	4 years ago
Markus Bösling	275ff5b223	double word removed (#30679)	4 years ago
Alisdair McDiarmid	b386f76b65	Merge pull request #30685 from hashicorp/alisdair/fix-30641 ... cli: Fix missing identifying attributes in diff	4 years ago
James Bardin	8c5e11d41a	add relevant_attributes to the json plan format ... Add the resource instances and individual attributes which may have contributed to the planned changes to the json format of the plan. We use the existing path encoding for individual attributes, which is already used in the replace_paths change field.	4 years ago
James Bardin	f0cd8be66f	add whole resource references	4 years ago
James Bardin	e2b74247f2	track contributing instances ... Track individual instance drift rather than whole resources which contributed to the plan. This will allow the output to be more precise, and we can still use NoKey instances as a proxy for containing resources when needed.	4 years ago
James Bardin	25f4c0d3dd	filter attribute refresh changes from plan UI ... Filter the refresh changes from the normal plan UI at the attribute level. We do this by constructing fake plans.Change records for diff generation, reverting all attribute changes that do not match any of the plan's ContributingResourceReferences.	4 years ago
James Bardin	0e7cec83db	decode change before creating diff ... This is functionally equivalent, but will allow us to filter the change values directly for reduced drift output.	4 years ago
James Bardin	c02e8bc5b3	change plan to store individual relevant attrs ... Storing individual contributing attributes will allow finer tuning of the plan rendering. add contributing to outputs	4 years ago
Alisdair McDiarmid	ad9c89fc19	cli: Fix missing identifying attributes in diff ... When rendering a diff for an object value within a resource, Terraform should always display the value of attributes which may be identifying. At present, this is a simple rule: render attributes named "id", "name", or "tags". Prior to this commit, Terraform would only apply this rule to top-level resource attributes and those inside nested blocks. Here we extend the implementation to include object values in other contexts as well.	4 years ago
James Bardin	f1e8aed48d	output drift footer in the right place	4 years ago
James Bardin	c5c7045a89	filter out non-relevant drift changes ... Only show drift changes which may have affected the plan output.	4 years ago
James Bardin	a151aaef05	only show drift when there are changes	4 years ago
Alisdair McDiarmid	70db495d00	Merge pull request #30525 from nozaq/provider-config-default ... jsonconfig: fix keys for default providers	4 years ago
Alisdair McDiarmid	3e4d6b252f	jsonplan: Improve performance for deep objects ... When calculating the unknown values for JSON plan output, we would previously recursively call the `unknownAsBool` function on the current sub-tree twice, if any values were unknown. This was wasteful, but not noticeable for normal Terraform resource shapes. However for deeper nested object values, such as Kubernetes manifests, this was a severe performance problem, causing `terraform show -json` to take several hours to render a plan. This commit reuses the already calculated unknown value for the subtree, and adds benchmark coverage to demonstrate the improvement.	4 years ago
nozaq	3c32f7a56c	jsonconfig: add implicitly created provider configs	4 years ago
nozaq	0ce040405c	jsonconfig: fix provider mappings with same names	4 years ago
nozaq	7f4019e1f6	jsonconfig: fix keys for default providers ... Fixes provider config keys to reflect implicit provider inheritance.	4 years ago
Pedro Belém	aed7162e9a	cli: Fix state mv exit code for missing resource (#29839)	4 years ago
Alisdair McDiarmid	fe8183c4af	json: Increment JSON plan format version ... The JSON plan configuration data now includes a `full_name` field for providers. This addition warrants a backwards compatible increment to the version number.	4 years ago
Alisdair McDiarmid	ddc81a204f	json: Disregard format version in tests ... Instead of manually updating every JSON output test fixture when we change the format version, disregard any differences when testing.	4 years ago
Alisdair McDiarmid	f5b90f84a8	jsonconfig: Improve provider configuration output ... When rendering configuration as JSON, we have a single map of provider configurations at the top level, since these are globally applicable. Each resource has an opaque key into this map which points at the configuration data for the provider. This commit fixes two bugs in this implementation: - Resources in non-root modules had an invalid provider config key, which meant that there was never a valid reference to the provider config block. These keys were prefixed with the local module name instead of the path to the module. This is now corrected. - Modules with passed provider configs would point to either an empty provider config block or one which is not present at all. This has been fixed so that these resources point to the provider config block from the calling module (or wherever up the module tree it was originally defined). We also add a "full_name" key-value pair to the provider config block, with the entire fully-qualified provider name including hostname and namespace.	4 years ago
Alisdair McDiarmid	5a9bc76d1a	Merge pull request #30316 from superkooks/main ... Fix autocomplete for workspace subcommands	4 years ago
Krista LaFentres (she/her)	6dcf00aefc	Merge pull request #30344 from hashicorp/lafentres/refactor-show-command ... cli: Refactor show command & migrate to command arguments and views	4 years ago
Krista LaFentres	fea8f6cfa2	cli: Migrate show command to use command arguments and views	4 years ago
Krista LaFentres	8d1bced812	cli: Refactor show command to remove dependence on local run and only load the backend when we need it ... See https://github.com/hashicorp/terraform/pull/30205#issuecomment-997113175 for more context	4 years ago
Martin Atkins	36c4d4c241	core and backend: remove redundant handling of default variable values ... Previously we had three different layers all thinking they were responsible for substituting a default value for an unset root module variable: - the local backend, via logic in backend.ParseVariableValues - the context.Plan function (and other similar functions) trying to preprocess the input variables using terraform.mergeDefaultInputVariableValues . - the newer prepareFinalInputVariableValue, which aims to centralize all of the variable preparation logic so it can be common to both root and child module variables. The second of these was also trying to handle type constraint checking, which is also the responsibility of the central function and not something we need to handle so early. Only the last of these consistently handles both root and child module variables, and so is the one we ought to keep. The others are now redundant and are causing prepareFinalInputVariableValue to get a slightly corrupted view of the caller's chosen variable values. To rectify that, here we remove the two redundant layers altogether and have unset root variables pass through as cty.NilVal all the way to the central prepareFinalInputVariableValue function, which will then handle them in a suitable way which properly respects the "nullable" setting. This commit includes some test changes in the terraform package to make those tests no longer rely on the mergeDefaultInputVariableValues logic we've removed, and to instead explicitly set cty.NilVal for all unset variables to comply with our intended contract for PlanOpts.SetVariables, and similar. (This is so that we can more easily catch bugs in callers where they _don't_ correctly handle input variables; it allows us to distinguish between the caller explicitly marking a variable as unset vs. not describing it at all, where the latter is a bug in the caller.)	4 years ago
SuperKooks	5171c89e1e	Fix autocomplete for workspace subcommands	4 years ago
Martin Atkins	b802db75d7	build: Build and run e2etest as part of the release build pipeline ... This uses the decoupled build and run strategy to run the e2etests so that we can arrange to run the tests against the real release packages produced elsewhere in this workflow, rather than ones generated just in time by the test harness. The modifications to make-archive.sh here make it more consistent with its originally-intended purpose of producing a harness for testing "real" release executables. Our earlier compromise of making it include its own terraform executable came from a desire to use that script as part of manual cross-platform testing when we weren't yet set up to support automation of those tests as we're doing here. That does mean, however, that the terraform-e2etest package content must be combined with content from a terraform release package in order to produce a valid contest for running the tests. We use a single job to cross-compile the test harness for all of the supported platforms, because that build is relatively fast and so not worth the overhead of matrix build, but then use a matrix build to actually run the tests so that we can run them in a worker matching the target platform. We currently have access only to amd64 (x64) runners in GitHub Actions and so for the moment this process is limited only to the subset of our supported platforms which use that architecture.	4 years ago
Alisdair McDiarmid	535da4ebc7	Merge pull request #30205 from hashicorp/alisdair/fix-show-plan-against-non-default-state ... command/show: Disable plan state lineage checks	4 years ago
Martin Atkins	23395a1022	providercache: Discard lock entries for unused providers ... Previously we would only ever add new lock entries or update existing ones. However, it's possible that over time a module may _cease_ using a particular provider, at which point we ought to remove it from the lock file so that operations won't fail when seeing that the provider cache directory is inconsistent with the lock file. Now the provider installer (EnsureProviderVersions) will remove any lock file entries that relate to providers not included in the given requirements, which therefore makes the resulting lock file properly match the set of packages the installer wrote into the cache. This does potentially mean that someone could inadvertently defeat the lock by removing a provider dependency, running "terraform init", then undoing that removal, and finally running "terraform init" again. However, that seems relatively unlikely compared to the likelihood of removing a provider and keeping it removed, and in the event it _did_ happen the changes to the lock entry for that provider would be visible in the diff of the provider lock file as usual, and so could be noticed in code review just as for any other change to dependencies.	4 years ago
Alisdair McDiarmid	768741c0f7	command/show: Disable plan state lineage checks ... When showing a saved plan, we do not need to check the state lineage against current state, because the plan cannot be applied. This is relevant when plan and apply specify a `-state` argument to choose a non-default state file. In this case, the stored prior state in the plan will not match the default state file, so a lineage check will always error.	4 years ago
Chris Arcand	8b8fe2771f	Merge pull request #30142 from hashicorp/chrisarcand/remote-backend-no-workspaces-regression ... command/meta_backend: Allow the remote backend to have no workspaces [again]	4 years ago
Chris Arcand	98978b3853	command/meta_backend: Allow the remote backend to have no workspaces [again] ... A regression introduced in d72a413ef8 The comment explains, but TLDR: The remote backend actually *depended* on being able to write it's backend state even though an 'error' occurred (no workspaces).	4 years ago
Martin Atkins	096cddb4b7	command/format: Limitation of plans.ResourceInstanceDeleteBecauseNoModule ... This is an explicit technical debt note that our plan renderer isn't able to give a fully-specific hint in this particular case of deletion reason. This reason code means that at least one of the module instance keys in the resource's module path doesn't match an instance declared in the configuration, but the plan data structure doesn't retain enough information to know which is the first step in the path which refers to a missing instance, and so we just always return the whole thing. This would be confusing if we return module.foo[0].module.bar not being in the configuration as a result of module.foo not using "count"; it would be better to say "module.foo[0] is not in the configuration" instead. It would be most ideal to handle all of the different situations that ResourceInstanceDeleteBecauseWrongRepetition's rendering does, so that we can go further and explain exactly _why_ that module instance isn't declared anymore. We can do neither of those things today because only the Terraform Core "expander" component knows that information, and we've discarded that by the time we get to rendering a plan. To fix this one day would require preserving in the plan information about which module instances are declared, as a separate sidecar data structure from which resource instances we're taking actions on, and then using that to identify which step in addr.Module here first selects an invalid instance.	4 years ago
Chris Arcand	f4f5b7c968	Merge pull request #30059 from hashicorp/barrettclark/explicit-local-empty-migrate-messaging ... Cloud: Do not ask to migrate empty default workspace	4 years ago
Barrett Clark	c706c8f92b	Do not ask to migrate empty default workspace ... When migrating from an explicit local backend to Terraform Cloud, we ask if you want to migrate the state. If there is no state to migrate we should not ask if they want to migrate the emptiness.	4 years ago
Barrett Clark	e08a02e7bf	Fixes Issue #29959, Apply w/o init error message ... When going from a local backend to Terraform Cloud, if you skip the `terraform init` and run `terraform apply` this will give the user more clear instructions.	4 years ago
Luces Huayhuaca	d72a413ef8	command/meta_backend: Prompt to select workspace before saving backend config (#29945) ... When terraform detects that a user has no workspaces that map to their current configuration, it will prompt the user to create a new workspace and enter a value name. If the user ignores the prompt and exits it, the legacy backend (terraform.tfstate) will be left in a awkward state: 1. This saved backend config will show a diff for the JSON attributes "serial", "tags" and "hash" 2. "Terraform workspace list" will show an empty list 3. "Terraform apply" will run successfully using the previous workspace, from the previous config, not the one from the current saved backend config 4. The cloud config is not reflective of the current working directory Solution: If the user exits the prompt, the saved backend config should not be updated because they did not select a new workspace. They are back at the beginning where they are force to re run the init cmd again before proceeding with new changes.	4 years ago
Chris Arcand	b5af7b6c92	Merge pull request #29987 from hashicorp/chrisarcand/backend-flag-with-tfc ... command/init: Add -cloud alias to -backend, adjust `init` help output	4 years ago
Barrett Clark	a6b56ad76f	Merge pull request #29997 from hashicorp/barrettclark/cloud-input-false ... Cloud integration requires input for migrations	4 years ago
Barrett Clark	419676cb69	Cloud integration requires input for migrations ... We cannot programmatically migrate workspaces to Terraform Cloud without prompts, so `-input=false` should not be allowed in those cases. There are 4 scenarios where we need input from a user to complete migrating workspaces to Terraform Cloud. 1.) Migrate from a single local workspace to Terraform Cloud * Terraform config for a local backend. Implicit local (no backend specified) is fine. * `terraform init` and `terraform apply` * Change the Terraform config to use the cloud block * `terraform init -input=false` * You should now see an error message 2.) Migrate from a remote backend with a prefix to Terraform Cloud with tags * Create a workspace in Terraform Cloud manually. The name should include a prefix, like "app-one" * Have the terraform config use `backend "remote"` with a prefix set to "app-" * `terraform init` and `terraform apply` * Update the Terraform config to use a cloud block with `tags = ["app"]`. There should not be a prefix defined in the config now. * `terraform init -input=false` * You should now see an error message 3.) Migrate from multiple local workspaces to a single Terraform Cloud workspace * Create one or many local workspaces * `terraform init` and `terraform apply` in each * Change the Terraform config to use the cloud block * `terraform init -input=false` * You should now see an error message 4.) Migrate to Terraform Cloud and ask for a workspace name * Create several local workspaces * `terraform init` and `terraform apply` in each * Change the Terraform config to use the cloud block with tags * `terraform init -input=false` * You should now see an error message	4 years ago
Luces Huayhuaca	f63b6198ca	Create a function for logic that assigns value to initReason var after changing backend configuration (#29967) ... * Create a function for logic that assigns value to initReason var after changing backend configuration Create func determineInitReason() for logic block that assigns value to initReason var after changing backend/cloud configuration block or migrating to a different type of backend configuration. Also clarify 'cloud' configuration block message to say 'Terraform Cloud configuration block has changed' instead of 'Terraform Cloud configuration has changed'.	4 years ago
Alisdair McDiarmid	a8b9d086b2	cli: Fix nested single and map diff rendering ... Indentation was out by two spaces. This is now consistent with the other displays of object/map values: four spaces for each nesting level.	4 years ago
Chris Arcand	a0c7c1dc8d	command/init: Adjust init help output ... Some of the wording here needed adjusting with the change that backends largely reflect state snapshot storage (removing 'enhanced' designation), and that a 'backend' is not necessarily always present.	4 years ago
Chris Arcand	33d7c1e6d6	command/init: Add -cloud alias (of -backend) ... This fixes an issue where a user could not disable initialization of the 'cloud' configuration block (As is possible with -backend=false), as well as add some syntactic sugar around -backend by adding a mutually exclusive -cloud alias.	4 years ago
Alisdair McDiarmid	b562c4fb84	cli: Fix nested set diff rendering indentation	4 years ago
Alisdair McDiarmid	2d0349d9a4	cli: Fix diff for nested set unchanged elements ... Unchanged elements in nested attributes backed by sets were previously misrendered as empty objects. This commit removes the additional brackets and adds a count of unchanged elements.	4 years ago
Martin Atkins	7d6d31eff8	command/init: Skip redundant state migration prompt in Cloud mode ... The specialized Terraform Cloud migration process asks right up top whether the user wants to migrate state, because there are various other questions contingent on that answer. Therefore we ought to just honor their earlier answer when we get to the point of actually doing the state migration, rather than prompting again. This is tricky because we're otherwise just reusing a codepath that's common to both modes. Hopefully we can find a better way to do this in a later commit, but for the moment our main motivation is minimizing risk to the very next release.	4 years ago
Martin Atkins	bac59d2480	command/init: Be explicit that some options are not relevant for Cloud ... There are a few command line options for "terraform init" which are only relevant when working with traditional backends, with the Cloud integration previously just mostly ignoring them, or sometimes misbehaving slightly due to them creating an unreasonable situation. Now we'll catch these and return explicit errors, in order to be clear that these options are not needed nor supported in Cloud mode.	4 years ago
Barrett Clark	a146a2746e	Add clarifying commend and e2e test ... This pull request focuses on removing the prompt to rename the default workspace when it is empty. Functionality already exists to not migrate an empty workspace. This commit adds some clarifying language in the comment where we do the evaluation to know whether to ask for a new name or not. I also added an end to end test, which I should have added to begin with.	4 years ago
Barrett Clark	a9eb62d692	Cloud: Migrate empty default workspace prompt ... Given: You have multiple explicit local workspaces, and the `default` workspace is empty. When: You migrate the workspaces to Terraform Cloud. Then: Terraform should _not_ ask for a workspace to migrate the `default` workspace to in Terraform Cloud.	4 years ago
Chris Arcand	bf76cc98ef	command: Suggestions on migration copy from user feedback ... Some small edits to the TFC migration copy.	4 years ago
kmoe	64635edfb9	Merge pull request #29885 from hashicorp/kmoe/more-init-interrupts ... command: make module installation interruptible	5 years ago
Katy Moe	6da61bf07b	revert change to installModules diag handling ... Error diags from c.installModules() no longer cause getModules() to exit early. Whether installModules completed successfully, errored, or was cancelled, we try to update the manifest as best we can, preferring incomplete information to none.	5 years ago
kmoe	40ec62c139	command: make module installation interruptible ... Earlier work to make "terraform init" interruptible made the getproviders package context-aware in order to allow provider installation to be cancelled. Here we make a similar change for module installation, which is now also cancellable with SIGINT. This involves plumbing context through initwd and getmodules. Functions which can make network requests now include a context parameter whose cancellation cancels those requests. Since the module installation code is shared, "terraform get" is now also interruptible during module installation.	5 years ago
Krista LaFentres	c44c589f0a	Error if backup or backup-out options are used without the state option on non-local backends for the state mv command	5 years ago
Chris Arcand	779c958fbf	cloud: Add streamlined 'remote' backend state migration path ... For Terraform Cloud users using the 'remote' backend, the existing 'pattern' prompt should work just fine - but because their workspaces are already present in TFC, the 'migration' here is really just realigning their local workspaces with Terraform Cloud. Instead of forcing users to do the mental gymnastics of what it means to migrate from 'prefix' - and because their remote workspaces probably already exist and already conform to Terraform Cloud's naming concerns - streamline the process for them and calculate the necessary pattern to migrate as-is, without any user intervention necessary.	5 years ago
Chris Arcand	6f25ad3e17	cloud: Display the newly renamed workspaces after a migration	5 years ago
Chris Arcand	c3bf5d4512	cloud: Automatically select renamed current workspace ... After migrating to TFC with renamed workspaces, automatically select what was the previous current workspace on behalf of the user. We don't need to make the user reselect.	5 years ago
Chris Arcand	f80b2177c8	cloud: Tweak multi-to-multi migration UX ... Note these change do break the internal/cloud/e2e tests; they are in a sad state that needs adjusting anyway, so I'm not updating them for these changes at this time.	5 years ago
Chris Arcand	f04ea2bd3d	Merge pull request #29860 from hashicorp/fix-init-after-error-take-two ... command: Update backend hash value only after a successful migration	5 years ago
Chris Arcand	7c0c2e952f	command: Don't always update backend hash when fetching the saved backend ... The Meta.backend_C_r_S_unchanged() method was sadly a bit of a mess. It seems to have originally been used as a method to be called when the backend is not changing, with an extra assumption that if the configured backend's hash doesn't match the one in state, surely the hash should just be updated as an option might have been moved to command line flags. However, this function was used throughout this file as 'the method to load the initialized (but not necessarily configured) backend', regardless of whether or not it is the same (unchanged). This is in addition to Meta.backendFromState(), which is used to load the same thing except in the main codepath of 'init -backend=false'. These changes separate the concerns of backend_C_r_S_unchanged() by 1) Fetching the saved backend (savedBackend()) 2) Updating the hash value in the backend cache when appropriate (either by leaving it to the caller to do themselves or by calling updateSavedupdateSavedBackendHash()) This allows migration codepaths to *not* update the hash value until after a migration has successfully taken place.	5 years ago
Łukasz Sierant	19cce931a8	Test case for changing backend hash during aborted state migration ... Pulled and updated from https://github.com/hashicorp/terraform/pull/26260	5 years ago
Billy Keyes	52ca30273f	command/format: improve list nested attribute rendering (#29827) ... * command/format: fix list nested attr diff rendering Previously, diffs only rendered correctly if all changed elements appeared before all unchanged elements. Once an unchanged element was found, all remaining elements were skipped. This usually led to the output being an empty list with a weird amount of space between the brackets. * command/format: improve list nested attr rendering This makes several changes that make diffs for lists clearer and more consistent: * Separate items with brackets instead of only new lines. This better matches the input syntax and avoids confusion from the first and last brackets implying there is a single item. * Render an action symbol for each element of the list * Use the correct action symbol for new and deleted items * Fix the alignment of opening and closing brackets I also refactored the structure so it is similar to the set and map cases to minimize duplication of the new prints. * Fix re-use of blockBodyDiffResult struct	5 years ago
Nick Fagerlund	02e62c9851	Cloud: Init without erroring when no workspaces match the `tags` (#29835) ... Previously, `terraform init` was throwing an error if you configured the cloud block with `tags` and there weren't any tagged workspaces yet. Confusing and alienating, since that that's a fairly normal situation! Basically TFC was handling an empty list of workspaces worse than other backends, because it doesn't support an unnamed default workspace. This commit catches that condition during `Meta.selectBackend()` and asks the user to pick a name for their first tagged workspace. If they cancel out, we still error, but if we know what name they want, we can handle it the same way as a nonexistent workspace specified in `name` -- just pass it to `Meta.SetWorkspace()`, and let the workspace get implicitly created when `InitCommand.Run()` eventually calls `StateMgr()`.	5 years ago
Martin Atkins	94cbc8fb5d	experiments: config_driven_move has concluded ... Based on feedback during earlier alpha releases, we've decided to move forward with the current design for the first phase of config-driven refactoring. Therefore here we've marked the experiment as concluded with no changes to the most recent incarnation of the functionality. The other changes here are all just updating test fixtures to no longer declare that they are using experimental features.	5 years ago
Chris Arcand	511afcd43a	command: Adjust skipping of resource counts for any remote implementation ... When using the Terraform Cloud integration - like the 'remote' backend - resource count output should be suppressed if those counts are being rendered remotely. This generalizes this to the shared BackendWithRemoteTerraformVersion interface.	5 years ago
James Bardin	81e709d185	Merge pull request #29825 from hashicorp/jbardin/no-panicwrap ... Remove the use of panicwrap in the Terraform CLI	5 years ago
Martin Atkins	f266d1ee82	"Add cloud integration option" ... A more native integration for Terraform Cloud and its CLI-driven run workflow. Instead of a backend, users declare a special block in the top-level terraform settings block to configure Terraform Cloud, then run terraform init. Full documentation will follow in later commits.	5 years ago
Chris Arcand	6dc1fed6d5	Clarify legacy Ui comments	5 years ago
Chris Arcand	14260a5b4c	Adjust default workspace naming prompt ... The previous version is a little overdramatic and somewhat accusatory. Just lay it out how it is: TFC needs workspaces to be named.	5 years ago
Omar Ismail	fc5863844c	Cloud migration: ignore backend version check when empty worksapces	5 years ago
Barrett Clark	f5366468b4	Cloud Backend reference migrating away from TFC	5 years ago
Chris Arcand	2c0294c7e3	Tweak configuration copy for TFC ... This aligns more with the existing copy for backends	5 years ago
Omar Ismail	1ff7827416	Multi-state migration prompt fix	5 years ago
Barrett Clark	cc6de251d8	Update init reconfigure error message ... If you move from the remote backend to the cloud block you will see this error message.	5 years ago
Barrett Clark	e16c53b561	Simplify/Consolidate the cloud conditionals	5 years ago
Barrett Clark	ab304d831f	Found another path to backend init error ... There are actually a few different ways to get to this message. 1. Blank state — no previous terraform applied. Start with a cloud block. 1. Implicit local — start with no backend specified. This actually goes through the same code execution path as the first scenario. 1. Explicit local — start with a backend local block that has been applied, then change from the local backend to a cloud block. This will recognize the state, and is a different path through the code in the meta backend. This commit handles the last case. The messaging has also been tweaked. End to end test included as well.	5 years ago
Barrett Clark	d29532cfeb	tfc apply before init ... https://app.asana.com/0/1199201948575144/1201019450474843/f	5 years ago
Barrett Clark	e49f271eb5	Update copy for terraform init with cloud block	5 years ago
Chris Arcand	09d5b70b7a	command/meta_backend_migrate:Tweak multi-to-single To TFC copy	5 years ago
Omar Ismail	dfb4609be2	Backend State Migration from cloud to cloud. ... * Add test for tfc to tfc mgiration * Fix old tests, and remove unused code.	5 years ago
Barrett Clark	93bfcff61a	More Terraform Cloud UX changes ... * Update e2e tests to specify Terraform Cloud * Update more of the terraform command flow to specify Terraform Cloud	5 years ago
Barrett Clark	261a2b49d3	UX for terraform init that copies state to TFC	5 years ago
Barrett Clark	0cd9490838	PR Feedback: cleanup variable setter	5 years ago
Chris Arcand	f8256f6634	Update ux for "terraform init [-reconfigure]"	5 years ago
Omar Ismail	dc76bbee73	Backend State Migration: Add remote backend test	5 years ago
Omar Ismail	3fedd6898c	Backend State Migration to `cloud`: Multiple Workspaces ... * Handle when there are multiple workspaces migrating to cloud, using both the cloud name strategy and cloud tags strategy. * Add e2e tests	5 years ago
Barrett Clark	edbc84420c	Run apply -refresh-state instead of refresh ... When a user runs `terraform refresh` we give them an error message that tells them to run `terraform apply -refresh-state`. We could just run that command for them, though. That is what this PR does.	5 years ago
Omar Ismail	55fc590904	Teraform Cloud Backend State Migration ... * determining source or destination to cloud * handling single to single state migrations to cloud, using a name strategy or a tags strategy * Add end-to-end tests for state migration.	5 years ago
Chris Arcand	fccc873a3d	Remove ability to declare a 'cloud' backend ... With the alternative block introduced in 7bf9b2c7b, this removes the ability to explicitly declare the 'cloud' backend. The literal backend interface is an implementation detail and no longer a user-level concept when using Terraform Cloud.	5 years ago
Chris Arcand	a4c24e3147	Add cloud {} configuration block for Terraform Cloud ... This is a replacement declaration for using Terraform Cloud as a remote backend, leaving the literal backend as an implementation detail and not a user-level concept.	5 years ago
James Bardin	9b62dd6901	Merge pull request #29648 from hashicorp/jbardin/tfproto6.1 ... tfproto version 6.1	5 years ago
James Bardin	d03a037567	insert panic handlers	5 years ago
James Bardin	42742c173d	remove wrapped streams and readline	5 years ago
James Bardin	622c4df14c	remove the use of panicwrap ... Stop using panicwrap, and execute terraform in the main process.	5 years ago
Alisdair McDiarmid	ded4f1a0fd	Merge pull request #29805 from hashicorp/alisdair/fix-init-workspace-select-input-false ... command/init: Fail if -input=false but required	5 years ago
Alisdair McDiarmid	ecb98e1c43	command/init: Fail if -input=false but required ... When running `terraform init` against a backend with multiple workspaces, none of which are the currently indicated local workspace, Terraform prompts the user to choose a workspace from the list. In automation, using the `-input=false` argument should disable asking for input, but previously would hang instead.	5 years ago
Alisdair McDiarmid	6d0521e6be	Merge pull request #29803 from hashicorp/alisdair/init-backend-provider-errors ... cli: Fix backend init failure with deleted cache	5 years ago
Alisdair McDiarmid	22923f9873	Merge pull request #29793 from hashicorp/alisdair/fix-remote-backend-migrate-version-check ... backend/remote: Fix version check when migrating	5 years ago
Alisdair McDiarmid	d42d83572b	cli: Fix backend init failure with deleted cache ... When an explicit backend is configured with a configuration which has not yet been initialized, running `terraform init` performs a state migration to fetch the remotely stored state in order to operate on it. Like the previous bug introduced by the recent provider diagnostics change, this code path was not correctly configured to enable init mode for the backend, which resulted in a fatal error during init when the cache dir is deleted. Setting the `Init` backend option allows this code path to continue without error when first initializing the backend for state migration. The new e2e test fails without this change.	5 years ago
Alisdair McDiarmid	1729431520	backend/remote: Fix version check when migrating ... When migrating state to an existing Terraform Cloud workspace using the remote backend, we check the remote version is compatible with the local one by default. This commit fixes two bugs in this code: - If using the "name" strategy for the remote backend, the list of destination workspaces is empty. This resulted in no version checking of the remote workspace, and we fell back to the string equality check. - The user-specified CLI flag `-ignore-remote-version` was not being applied for the state migration version checking.	5 years ago
Alisdair McDiarmid	39de3ebec7	cli: Fix init failure with deleted cache ... The init command needs to initialize a backend, in order to access state, in turn to derive provider requirements from state. The backend initialization step requires building provider factories, which previously would fail if a lockfile was present without a corresponding local provider cache. This commit ensures that in this situation only, errors with the provider factories are temporarily ignored. This allows us to continue to initialize the backend, fetch providers, and then report any errors as necessary.	5 years ago
Alisdair McDiarmid	1190b95fe2	command/e2etest: Ensure init fixes missing cache ... We test that a deleted provider cache results in an error when running terraform plan, but previously did not test that running init (as instructed) would resolve the issue. This (failing) e2e test adds that step.	5 years ago
Martin Atkins	5b266dd5ca	command: Remove the experimental "terraform add" command ... We introduced this experiment to gather feedback, and the feedback we saw led to us deciding to do another round of design work before we move forward with something to meet this use-case. In addition to being experimental, this has only been included in alpha releases so far, and so on both counts it is not protected by the Terraform v1.0 Compatibility Promises.	5 years ago
Alisdair McDiarmid	cdd5ee6fb3	Merge pull request #29773 from hashicorp/alisdair/init-lock-flags ... cli: Restore -lock and -lock-timeout init flags	5 years ago
Alisdair McDiarmid	c587384dff	cli: Restore -lock and -lock-timeout init flags ... The -lock and -lock-timeout flags were removed prior to the release of 1.0 as they were thought to have no effect. This is not true in the case of state migrations when changing backends. This commit restores these flags, and adds test coverage for locking during backend state migration. Also update the help output describing other boolean flags, showing the argument as the user would type it rather than the default behavior.	5 years ago
Alisdair McDiarmid	fb58f9e6d2	cli: Fix flaky init cancel test ... There is a race between the MockSource and ShutdownCh which sometimes causes this test to fail. Add a HangingSource implementation of Source which hangs until the context is cancelled, so that there is always time for a user-initiated shutdown to trigger the cancellation code path under test.	5 years ago
James Bardin	ef3c98466d	Merge pull request #29755 from hashicorp/jbardin/first-plan-lineage ... Check for stale plan with no state metadata	5 years ago
James Bardin	9c80574417	test planfile may need to have a specific lineage ... In order to test applying a plan from an existing state, we need to be able to inject the state meta into the planfile.	5 years ago
Martin Atkins	bee7403f3e	command/workspace_delete: Allow deleting a workspace with empty husks ... Previously we would reject attempts to delete a workspace if its state contained any resources at all, even if none of the resources had any resource instance objects associated with it. Nowadays there isn't any situation where the normal Terraform workflow will leave behind resource husks, and so this isn't as problematic as it might've been in the v0.12 era, but nonetheless what we actually care about for this check is whether there might be any remote objects that this state is tracking, and for that it's more precise to look for non-nil resource instance objects, rather than whole resources. This also includes some adjustments to our error messaging to give more information about the problem and to use terminology more consistent with how we currently talk about this situation in our documentation and elsewhere in the UI. We were also using the old State.HasResources method as part of some of our tests. I considered preserving it to avoid changing the behavior of those tests, but the new check seemed close enough to the intent of those tests that it wasn't worth maintaining this method that wouldn't be used in any main code anymore. I've therefore updated those tests to use the new HasResourceInstanceObjects method instead.	5 years ago
Alisdair McDiarmid	b9f3dab035	json-output: Release format version 1.0	5 years ago
Omar Ismail	bea7e3ebce	Backend State Migration: change variable names from one/two to source/destination (#29699)	5 years ago
Martin Atkins	01b22f4b76	command/e2etest: TestProviderTampering ... We have various mechanisms that aim to ensure that the installed provider plugins are consistent with the lock file and that the lock file is consistent with the provider requirements, and we do have existing unit tests for them, but all of those cases mock our fake out at least part of the process and in the past that's caused us to miss usability regressions, where we still catch the error but do so at the wrong layer and thus generate error message lacking useful additional context. Here we'll add some new end-to-end tests to supplement the existing unit tests, making sure things work as expected when we assemble the system together as we would in a release. These tests cover a number of different ways in which the plugin selections can grow inconsistent. These new tests all run only when we're in a context where we're allowed to access the network, because they exercise the real plugin installer codepath. We could technically build this to use a local filesystem mirror or other such override to avoid that, but the point here is to make sure we see the expected behavior in the main case, and so it's worth the small additional cost of downloading the null provider from the real registry.	5 years ago
Martin Atkins	d09510a8fb	command: Early error message for missing cache entries of locked providers ... In the original incarnation of Meta.providerFactories we were returning into a Meta.contextOpts whose signature didn't allow it to return an error directly, and so we had compromised by making the provider factory functions themselves return errors once called. Subsequent work made Meta.contextOpts need to return an error anyway, but at the time we neglected to update our handling of the providerFactories result, having it still defer the error handling until we finally instantiate a provider. Although that did ultimately get the expected result anyway, the error ended up being reported from deep in the guts of a Terraform Core graph walk, in whichever concurrently-visited graph node happened to try to instantiate the plugin first. This meant that the exact phrasing of the error message would vary between runs and the reporting codepath didn't have enough context to given an actionable suggestion on how to proceed. In this commit we make Meta.contextOpts pass through directly any error that Meta.providerFactories produces, and then make Meta.providerFactories produce a special error type so that Meta.Backend can ultimately return a user-friendly diagnostic message containing a specific suggestion to run "terraform init", along with a short explanation of what a provider plugin is. The reliance here on an implied contract between two functions that are not directly connected in the callstack is non-ideal, and so hopefully we'll revisit this further in future work on the overall architecture of the CLI layer. To try to make this robust in the meantime though, I wrote it to use the errors.As function to potentially unwrap a wrapped version of our special error type, in case one of the intervening layers is changed at some point to wrap the downstream error before returning it.	5 years ago
Martin Atkins	df578afd7e	backend/local: Check dependency lock consistency before any operations ... In historical versions of Terraform the responsibility to check this was inside the terraform.NewContext function, along with various other assorted concerns that made that function particularly complicated. More recently, we reduced the responsibility of the "terraform" package only to instantiating particular named plugins, assuming that its caller is responsible for selecting appropriate versions of any providers that _are_ external. However, until this commit we were just assuming that "terraform init" had correctly selected appropriate plugins and recorded them in the lock file, and so nothing was dealing with the problem of ensuring that there haven't been any changes to the lock file or config since the most recent "terraform init" which would cause us to need to re-evaluate those decisions. Part of the game here is to slightly extend the role of the dependency locks object to also carry information about a subset of provider addresses whose lock entries we're intentionally disregarding as part of the various little edge-case features we have for overridding providers: dev_overrides, "unmanaged providers", and the testing overrides in our own unit tests. This is an in-memory-only annotation, never included in the serialized plan files on disk. I had originally intended to create a new package to encapsulate all of this plugin-selection logic, including both the version constraint checking here and also the handling of the provider factory functions, but as an interim step I've just made version constraint consistency checks the responsibility of the backend/local package, which means that we'll always catch problems as part of preparing for local operations, while not imposing these additional checks on commands that _don't_ run local operations, such as "terraform apply" when in remote operations mode.	5 years ago
Martin Atkins	6a98e4720c	plans/planfile: Create takes most arguments via a struct type ... Previously the planfile.Create function had accumulated probably already too many positional arguments, and I'm intending to add another one in a subsequent commit and so this is preparation to make the callsites more readable (subjectively) and make it clearer how we can extend this function's arguments to include further components in a plan file. There's no difference in observable functionality here. This is just passing the same set of arguments in a slightly different way.	5 years ago
Martin Atkins	8d193ad268	core: Simplify and centralize plugin availability checks ... Historically the responsibility for making sure that all of the available providers are of suitable versions and match the appropriate checksums has been split rather inexplicably over multiple different layers, with some of the checks happening as late as creating a terraform.Context. We're gradually iterating towards making that all be handled in one place, but in this step we're just cleaning up some old remnants from the main "terraform" package, which is now no longer responsible for any version or checksum verification and instead just assumes it's been provided with suitable factory functions by its caller. We do still have a pre-check here to make sure that we at least have a factory function for each plugin the configuration seems to depend on, because if we don't do that up front then it ends up getting caught instead deep inside the Terraform runtime, often inside a concurrent graph walk and thus it's not deterministic which codepath will happen to catch it on a particular run. As of this commit, this actually does leave some holes in our checks: the command package is using the dependency lock file to make sure we have exactly the provider packages we expect (exact versions and checksums), which is the most crucial part, but we don't yet have any spot where we make sure that the lock file is consistent with the current configuration, and we are no longer preserving the provider checksums as part of a saved plan. Both of those will come in subsequent commits. While it's unusual to have a series of commits that briefly subtracts functionality and then adds back in equivalent functionality later, the lock file checking is the only part that's crucial for security reasons, with everything else mainly just being to give better feedback when folks seem to be using Terraform incorrectly. The other bits are therefore mostly cosmetic and okay to be absent briefly as we work towards a better design that is clearer about where that responsibility belongs.	5 years ago
Zach Whaley	c9a5fdb366	cliconfig: Fix error message about invalid credentials helper type	5 years ago
James Bardin	ab0322e406	remove debugging println	5 years ago
James Bardin	c2e0d265cf	LoadModule now always returns the module ... We don't need to load the configuration twice, since configload can return the module for us.	5 years ago
James Bardin	a53faf43f6	return partial config from LoadConfig with errors ... LoadConfig should return any parsed configuration in order for the caller to verify `required_version`.	5 years ago
James Bardin	625e768678	make sure required_version is checked before diags ... We must ensure that the terraform required_version is checked as early as possible, so that new configuration constructs don't cause init to fail without indicating the version is incompatible. The loadConfig call before the earlyconfig parsing seems to be unneeded, and we can delay that to de-tangle it from installing the modules which may have their own constraints. TODO: it seems that loadConfig should be able to handle returning the version constraints in the same manner as loadSingleModule.	5 years ago
Alisdair McDiarmid	b699391d04	json-output: Add change reasons to explain deletes ... The extra feedback information for why resource instance deletion is planned is now included in the streaming JSON UI output. We also add an explicit case for no-op actions to switch statements in this package to ensure exhaustiveness, for future linting.	5 years ago
James Bardin	9847eaa9cf	remove usage of MinItems/MaxItems ... MinItems and MaxItems are not used on nested types in the protocol, so remove their usage in Terraform to prevent future confusion.	5 years ago
Martin Atkins	04f9e7148c	command/format: Include deletion reasons in plan report ... The core runtime is now able to specify a reason for some situations when Terraform plans to delete a resource instance. This commit makes that information visible in the human-oriented UI. A previous commit already made the underlying data informing these new hints visible as part of the machine-oriented (JSON) plan output. This also removes the bold formatting from the existing "has moved to" hints, because subjectively it seemed like the result was emphasizing too many parts of the output and thus somewhat defeating the benefit of the emphasis in trying to create additional visual hierarchy for sighted users running Terraform in a terminal. Now only the first line containing the main action statement will be in bold, and all of the parenthesized follow-up notes will be unformatted.	5 years ago
Martin Atkins	a1a713cf28	core: Report ActionReasons when we plan to delete "orphans" ... There are a few different reasons why a resource instance tracked in the prior state might be considered an "orphan", but previously we reported them all identically in the planned changes. In order to help users understand the reason for a surprising planned delete, we'll now try to specify an additional reason for the planned deletion, covering all of the main reasons why that could happen. This commit only introduces the new detail to the plans.Changes result, though it also incidentally exposes it as part of the JSON plan result in order to keep that working without returning errors in these new cases. We'll expose this information in the human-oriented UI output in a subsequent commit.	5 years ago
Chris Arcand	171cdbbf93	command: Clean up testInputResponseMap before failing on unused answers ... If you don't, the unused answers will persist in the package-level var and bleed in to other tests.	5 years ago
Chris Arcand	60bc7aa05d	command: Auto-select single workspace if necessary ... When initializing a backend, if the currently selected workspace does not exist, the user is prompted to select from the list of workspaces the backend provides. Instead, we should automatically select the only workspace available _if_ that's all that's there. Although with being a nice bit of polish, this enables future improvments with Terraform Cloud in potentially removing the implicit depenency on always using the 'default' workspace when the current configuration is mapped to a single TFC workspace.	5 years ago
Chris Arcand	08a86b6c13	Merge pull request #29621 from hashicorp/error-on-unused-test-answers ... command: Ensure all answers were used in command.testInputResponseMap	5 years ago
Chris Arcand	8684a85e26	command: Ensure all answers were used in command.testInputResponseMap ... Remove answers from testInputResponse as they are given, and raise an error during cleanup if any answers remain unused. This enables tests to ensure that the expected mock answers are actually used in a test; previously, an entire branch of code including an input sequence could be omitted and the test(s) would not fail. The only test that had unused answers in this map is one leftover from legacy state migrations, a prompt that was removed in 7c93b2e5e6	5 years ago
Alisdair McDiarmid	b59b057591	json-output: Config-driven move support in JSON UI ... Add previous address information to the `planned_change` and `resource_drift` messages for the streaming JSON UI output of plan and apply operations. Here we also add a "move" action value to the `change` object of these messages, to represent a move-only operation. As part of this work we also simplify this code to use the plan's DriftedResources values instead of recomputing the drift from state.	5 years ago
Alisdair McDiarmid	78c4a8c461	json-output: Previous address for resource changes ... Configuration-driven moves are represented in the plan file by setting the resource's `PrevRunAddr` to a different value than its `Addr`. For JSON plan output, we here add a new field to resource changes, `previous_address`, which is present and non-empty only if the resource is planned to be moved. Like the CLI UI, refresh-only plans will include move-only changes in the resource drift JSON output. In normal plan mode, these are elided to avoid redundancy with planned changes.	5 years ago
Martin Atkins	f0034beb33	core: refactoring.ImpliedMoveStatements replaces NodeCountBoundary ... Going back a long time we've had a special magic behavior which tries to recognize a situation where a module author either added or removed the "count" argument from a resource that already has instances, and to silently rename the zeroth or no-key instance so that we don't plan to destroy and recreate the associated object. Now we have a more general idea of "move statements", and specifically the idea of "implied" move statements which replicates the same heuristic we used to use for this behavior, we can treat this magic renaming rule as just another "move statement", special only in that Terraform generates it automatically rather than it being written out explicitly in the configuration. In return for wiring that in, we can now remove altogether the NodeCountBoundary graph node type and its associated graph transformer, CountBoundaryTransformer. We handle moves as a preprocessing step before building the plan graph, so we no longer need to include any special nodes in the graph to deal with that situation. The test updates here are mainly for the graph builders themselves, to acknowledge that indeed we're no longer inserting the NodeCountBoundary vertices. The vertices that NodeCountBoundary previously depended on now become dependencies of the special "root" vertex, although in many cases here we don't see that explicitly because of the transitive reduction algorithm, which notices when there's already an equivalent indirect dependency chain and removes the redundant edge. We already have plenty of test coverage for these "count boundary" cases in the context tests whose names start with TestContext2Plan_count and TestContext2Apply_resourceCount, all of which continued to pass here without any modification and so are not visible in the diff. The test functions particularly relevant to this situation are: - TestContext2Plan_countIncreaseFromNotSet - TestContext2Plan_countDecreaseToOne - TestContext2Plan_countOneIndex - TestContext2Apply_countDecreaseToOneCorrupted The last of those in particular deals with the situation where we have both a no-key instance _and_ a zero-key instance in the prior state, which is interesting here because to exercises an intentional interaction between refactoring.ImpliedMoveStatements and refactoring.ApplyMoves, where we intentionally generate an implied move statement that produces a collision and then expect ApplyMoves to deal with it in the same way as it would deal with all other collisions, and thus ensure we handle both the explicit and implied collisions in the same way. This does affect some UI-level tests, because a nice side-effect of this new treatment of this old feature is that we can now report explicitly in the UI that we're assigning new addresses to these objects, whereas before we just said nothing and hoped the user would just guess what had happened and why they therefore weren't seeing a diff. The backend/local plan tests actually had a pre-existing bug where they were using a state with a different instance key than the config called for but getting away with it because we'd previously silently fix it up. That's still fixed up, but now done with an explicit mention in the UI and so I made the state consistent with the configuration here so that the tests would be able to recognize _real_ differences where present, as opposed to the errant difference caused by that inconsistency.	5 years ago
Alisdair McDiarmid	638784b195	cli: Omit move-only drift, except for refresh-only ... The set of drifted resources now includes move-only changes, where the object value is identical but a move has been executed. In normal operation, we previousl displayed these moves twice: once as part of drift output, and once as part of planned changes. As of this commit we omit move-only changes from drift display, except for refresh-only plans. This fixes the redundant output.	5 years ago
Alisdair McDiarmid	9a7bbdab6f	Fix terraform add test failure due to bad merge	5 years ago
Alisdair McDiarmid	d425c26d77	Merge pull request #29589 from hashicorp/alisdair/planfile-drifted-resources ... core: Compute resource drift during plan phase, store in plan file	5 years ago
Alisdair McDiarmid	1f57b7a8bd	Merge pull request #29235 from magodo/terraform_add_output_append ... `terraform add`: `-out` option append to existing config & optionally check resource existance	5 years ago
Alisdair McDiarmid	d5b5407ccc	format: Fix incorrect nesting of Color/Sprintf ... Colorizing the result of an interpolated string can result in incorrect output, if the values used to generate the string happen to include color codes such as `[red]` or `[bold]`. Instead we should always colorize the format string before calling functions like `Sprintf`. This commit fixes all instances in this file.	5 years ago
Alisdair McDiarmid	f0cf4235f9	cli: Refactor resource drift rendering	5 years ago
Martin Atkins	718fa3895f	backend: Remove Operation.Parallelism field ... The presence of this field was confusing because in practice the local backend doesn't use it for anything and the remote backend was using it only to return an error if it's set to anything other than the default, under the assumption that it would always match ContextOpts.Parallelism. The "command" package is the one actually responsible for handling this option, and it does so by placing it into the partial ContextOpts which it passes into the backend when preparing for a local operation. To make that clearer, here we remove Operation.Parallelism and change the few uses of it to refer to ContextOpts.Parallelism instead, so that everyone is reading and writing this value from the same place.	5 years ago
James Bardin	d0993b0e80	fix temp directory handling in some tests ... Cleanup some more test fixtures to use t.TempDir Use EvalSymlinks with temp dir paths to help with MacOS errors from various terraform components.	5 years ago
Martin Atkins	65e0c448a0	workdir: Start of a new package for working directory state management ... Thus far our various interactions with the bits of state we keep associated with a working directory have all been implemented directly inside the "command" package -- often in the huge command.Meta type -- and not managed collectively via a single component. There's too many little codepaths reading and writing from the working directory and data directory to refactor it all in one step, but this is an attempt at a first step towards a future where everything that reads and writes from the current working directory would do so via an object that encapsulates the implementation details and offers a high-level API to read and write all of these session-persistent settings. The design here continues our gradual path towards using a dependency injection style where "package main" is solely responsible for directly interacting with the OS command line, the OS environment, the OS working directory, the stdio streams, and the CLI configuration, and then communicating the resulting information to the rest of Terraform by wiring together objects. It seems likely that eventually we'll have enough wiring code in package main to justify a more explicit organization of that code, but for this commit the new "workdir.Dir" object is just wired directly in place of its predecessors, without any significant change of code organization at that top layer. This first commit focuses on the main files and directories we use to find provider plugins, because a subsequent commit will lightly reorganize the separation of concerns for plugin launching with a similar goal of collecting all of the relevant logic together into one spot.	5 years ago
Martin Atkins	343279110a	core: Graph walk loads plugin schemas opportunistically ... Previously our graph walker expected to recieve a data structure containing schemas for all of the provider and provisioner plugins used in the configuration and state. That made sense back when terraform.NewContext was responsible for loading all of the schemas before taking any other action, but it no longer has that responsiblity. Instead, we'll now make sure that the "contextPlugins" object reaches all of the locations where we need schema -- many of which already had access to that object anyway -- and then load the needed schemas just in time. The contextPlugins object memoizes schema lookups, so we can safely call it many times with the same provider address or provisioner type name and know that it'll still only load each distinct plugin once per Context object. As of this commit, the Context.Schemas method is now a public interface only and not used by logic in the "terraform" package at all. However, that does leave us in a rather tenuous situation of relying on the fact that all practical users of terraform.Context end up calling "Schemas" at some point in order to verify that we have all of the expected versions of plugins. That's a non-obvious implicit dependency, and so in subsequent commits we'll gradually move all responsibility for verifying plugin versions into the caller of terraform.NewContext, which'll heal a long-standing architectural wart whereby the caller is responsible for installing and locating the plugin executables but not for verifying that what's installed is conforming to the current configuration and dependency lock file.	5 years ago
Martin Atkins	80b3fcf93e	core: Replace contextComponentFactory with contextPlugins ... In the v0.12 timeframe we made contextComponentFactory an interface with the expectation that we'd write mocks of it for tests, but in practice we ended up just always using the same "basicComponentFactory" implementation throughout. In the interests of simplification then, here we replace that interface and its sole implementation with a new concrete struct type contextPlugins. Along with the general benefit that this removes an unneeded indirection, this also means that we can add additional methods to the struct type without the usual restriction that interface types prefer to be small. In particular, in a future commit I'm planning to add methods for loading provider and provisioner schemas, working with the currently-unused new fields this commit has included in contextPlugins, as compared to its predecessor basicComponentFactory.	5 years ago
Alisdair McDiarmid	cd29c3e5fd	Revert "json-output: Release format version 1.0"	5 years ago
Laura Pacilio	a819d7db3a	Merge pull request #29502 from hashicorp/alisdair/json-format-version ... json-output: Release format version 1.0	5 years ago
Alisdair McDiarmid	7f3a29b46e	Merge pull request #29523 from hashicorp/alisdair/moved-ui ... command: Render "moved" annotations in plan UI	5 years ago
James Bardin	1ad0421e15	Merge pull request #29522 from hashicorp/jbardin/json-blocktoattr ... allow json output to marshal ConfigModeAttr blocks	5 years ago
Alisdair McDiarmid	29999c1d6f	command: Render "moved" annotations in plan UI ... For resources which are planned to move, render the previous run address as additional information in the plan UI. For the case of a move-only resource (which otherwise is unchanged), we also render that as a planned change, but without any corresponding action symbol. If all changes in the plan are moves without changes, the plan is no longer considered "empty". In this case, we skip rendering the action symbols in the UI.	5 years ago
James Bardin	ac2a870ea0	allow json output to marshal ConfigModeAttr blocks ... In order to marshal config blocks using ConfigModeAttr, we need to insert the fixup body to map hcl blocks to the attribute in the schema.	5 years ago
Alisdair McDiarmid	b60f201eed	json-output: Release format version 1.0	5 years ago
James Bardin	863963e7a6	de-linting	5 years ago
Martin Atkins	89b05050ec	core: Functional-style API for terraform.Context ... Previously terraform.Context was built in an unfortunate way where all of the data was provided up front in terraform.NewContext and then mutated directly by subsequent operations. That made the data flow hard to follow, commonly leading to bugs, and also meant that we were forced to take various actions too early in terraform.NewContext, rather than waiting until a more appropriate time during an operation. This (enormous) commit changes terraform.Context so that its fields are broadly just unchanging data about the execution context (current workspace name, available plugins, etc) whereas the main data Terraform works with arrives via individual method arguments and is returned in return values. Specifically, this means that terraform.Context no longer "has-a" config, state, and "planned changes", instead holding on to those only temporarily during an operation. The caller is responsible for propagating the outcome of one step into the next step so that the data flow between operations is actually visible. However, since that's a change to the main entry points in the "terraform" package, this commit also touches every file in the codebase which interacted with those APIs. Most of the noise here is in updating tests to take the same actions using the new API style, but this also affects the main-code callers in the backends and in the command package. My goal here was to refactor without changing observable behavior, but in practice there are a couple externally-visible behavior variations here that seemed okay in service of the broader goal: - The "terraform graph" command is no longer hooked directly into the core graph builders, because that's no longer part of the public API. However, I did include a couple new Context functions whose contract is to produce a UI-oriented graph, and _for now_ those continue to return the physical graph we use for those operations. There's no exported API for generating the "validate" and "eval" graphs, because neither is particularly interesting in its own right, and so "terraform graph" no longer supports those graph types. - terraform.NewContext no longer has the responsibility for collecting all of the provider schemas up front. Instead, we wait until we need them. However, that means that some of our error messages now have a slightly different shape due to unwinding through a differently-shaped call stack. As of this commit we also end up reloading the schemas multiple times in some cases, which is functionally acceptable but likely represents a performance regression. I intend to rework this to use caching, but I'm saving that for a later commit because this one is big enough already. The proximal reason for this change is to resolve the chicken/egg problem whereby there was previously no single point where we could apply "moved" statements to the previous run state before creating a plan. With this change in place, we can now do that as part of Context.Plan, prior to forking the input state into the three separate state artifacts we use during planning. However, this is at least the third project in a row where the previous API design led to piling more functionality into terraform.NewContext and then working around the incorrect order of operations that produces, so I intend that by paying the cost/risk of this large diff now we can in turn reduce the cost/risk of future projects that relate to our main workflow actions.	5 years ago
Alisdair McDiarmid	40dd4292b8	Merge pull request #29438 from hashicorp/alisdair/init-force-copy-multiple-workspaces ... command: Suppress prompt for init -force-copy	5 years ago
Alisdair McDiarmid	2762a940c0	command: Suppress prompt for init -force-copy ... The -force-copy flag to init should automatically migrate state. Previously this was not applied to one case: when migrating from a backend with multiple workspaces to another backend supporting multiple workspaces. I believe this was an oversight so this commit fixes that.	5 years ago
Alisdair McDiarmid	70a4f7a6b6	command: Fix stale lock after running add	5 years ago
Martin Atkins	43d753e727	command: "terraform add" is experimental ... We're aware of several quirks of this command's current design, which result from some existing architectural limitations that we can't address immediately. However, we do still want to make this command available in its current capacity as an incremental improvement, so as a compromise we'll document it as experimental. Our intent here is to exclude it from the Terraform 1.0 Compatibility Promises so that we can have the space to continue to improve the design as other parts of the overall Terraform system gain new capabilities. We don't currently have any concrete plan for this command to be stabilized and subject to compatibility promises. That decision will follow from ongoing discussions with other teams whose systems may need to change in order to support the final design of "terraform add".	5 years ago
James Bardin	68ed50616e	handle null and unknown values in attr diffs ... The code adopted from block diffs was not set to handle null and unknown values, as those are not allowed for blocks. We also revert the change to formatting nested object types as single attributes, because the attribute formatter cannot handle sensitive values from the schema. This presents some awkward syntax for diffs for now, but should suffice until the entire formatter can be refactored to better handle these new nested types.	5 years ago
Martin Atkins	383bbdeebc	Upgrade to Go 1.17 ... This includes the addition of the new "//go:build" comment form in addition to the legacy "// +build" notation, as produced by gofmt to ensure consistent behavior between Go versions. The new directives are all equivalent to what was present before, so there's no change in behavior. Go 1.17 continues to use the Unicode 13 tables as in Go 1.16, so this upgrade does not require also upgrading our Unicode-related dependencies. This upgrade includes the following breaking changes which will also appear as breaking changes for Terraform users, but that are consistent with the Terraform v1.0 compatibility promises. - On MacOS, Terraform now requires macOS 10.13 High Sierra or later. This upgrade also includes the following breaking changes which will appear as breaking changes for Terraform users that are inconsistent with our compatibility promises, but have justified exceptions as follows: - cidrsubnet, cidrhost, and cidrnetmask will now reject IPv4 CIDR addresses whose decimal components have leading zeros, where previously they would just silently ignore those leading zeros. This is a security-motivated exception to our compatibility promises, because some external systems interpret zero-prefixed octets as octal numbers rather than decimal, and thus the previous lenient parsing could lead to a different interpretation of the address between systems, and thus potentially allow bypassing policy when configuring firewall rules etc. This upgrade also includes the following breaking changes which could _potentially_ appear as breaking changes for Terraform users, but that do not in practice for the reasons given: - The Go net/url package no longer allows query strings with pairs separated by semicolons instead of ampersands. This primarily affects HTTP servers written in Go, and Terraform includes a special temporary HTTP server as part of its implementation of OAuth for "terraform login", but that server only needs to accept URLs created by Terraform itself and Terraform does not generate any URLs that would be rejected.	5 years ago
James Bardin	a94155d0ca	Merge pull request #29397 from hashicorp/jbardin/format-id-name-marks ... unmark object ID or Name for formatting	5 years ago
James Bardin	a48b024c0a	unmark object ID or Name for formatting	5 years ago
James Bardin	296a757ab4	check for null sets in diff rendering	5 years ago
James Bardin	fbfb14142e	render empty nested containers as attributes ... Don't try to break down containers that are empty to render the diff, so we can avoid having to check for empty vs null in all cases.	5 years ago
Vojtěch Srdečný	35115e9685	Fix CLI help text for output	5 years ago
Alisdair McDiarmid	3b33dc1105	json-output: Add output changes to plan logs ... Extend the outputs JSON log message to support an `action` field (and make the `type` and `value` fields optional). This allows us to emit a useful output change summary as part of the plan, bringing the JSON log output into parity with the text output. While we do have access to the before/after values in the output changes, attempting to wedge those into a structured log message is not appropriate. That level of detail can be extracted from the JSON plan output from `terraform show -json`.	5 years ago
magodo	90e6a3dffb	modify test case	5 years ago