76 Commits (4ec80fa18d1a4e6df2fce9f708b8fed6653e1d0b)

Author	SHA1	Message	Date
Nick Fagerlund	ed7f97c34f	Update language docs URLs in code and tests	5 years ago
Kristin Laemmert	8bab3dd374	command/state list: list resources in nested and expanded modules (#27268) ... * command/state list: list resources in nested and expaneded modules A few distinct bugs fixed in here: There was a bug in the logic checking if a given module was the child of the targetAddr, now fixed. That resolved the basic issue where resources in nested submodules were not listed. The logic around allowMissing needed some tweaking to allow for empty modules, as long as those modules had submodules with resources. state list is the only command using allowMissing with false so this felt safe to do. Finally I extended the logic so list would included expanded modules, which is to say giving module.foo would result in resources from module.foo[1], module.foo[0], etc. * update state list docs to show that module filtering includes any nested modules	5 years ago
Alisdair McDiarmid	f1b95788b9	command: Add tests for terraform validate -json ... Also uncomment and fix some tests which had been skipped for a couple of years. Those validate cases work now! Note that these test cases and the JSON output are not especially minimized, making them snapshot/golden tests. The output looks correct at time of writing, and we don't expect to change validate significantly any time soon, but if we do there will be some churn here.	5 years ago
Alisdair McDiarmid	21d80a26ea	command: Fix fmt to preserve blank block labels	5 years ago
Pam Selle	66091ae36c	Unmark values before showing in JSON ... This prevents "sensitive" values from unintentionally showing as nil when running terraform show -json	5 years ago
Martin Atkins	e70ab09bf1	command: new cache directory .terraform/providers for providers ... Terraform v0.10 introduced .terraform/plugins as a cache directory for automatically-installed plugins, Terraform v0.13 later reorganized the directory structure inside but retained its purpose as a cache. The local cache used to also serve as a record of specifically which packages were selected in a particular working directory, with the intent that a second run of "terraform init" would always select the same packages again. That meant that in some sense it behaved a bit like a local filesystem mirror directory, even though that wasn't its intended purpose. Due to some unfortunate miscommunications, somewhere a long the line we published some documentation that _recommended_ using the cache directory as if it were a filesystem mirror directory when working with Terraform Cloud. That was really only working as an accident of implementation details, and Terraform v0.14 is now going to break that because the source of record for the currently-selected provider versions is now the public-facing dependency lock file rather than the contents of an existing local cache directory on disk. After some consideration of how to move forward here, this commit implements a compromise that tries to avoid silently doing anything surprising while still giving useful guidance to folks who were previously using the unsupported strategy. Specifically: - The local cache directory will now be .terraform/providers rather than .terraform/plugins, because .terraform/plugins is effectively "poisoned" by the incorrect usage that we can't reliably distinguish from prior version correct usage. - The .terraform/plugins directory is now the "legacy cache directory". It is intentionally _not_ now a filesystem mirror directory, because that would risk incorrectly interpreting providers automatically installed by Terraform v0.13 as if they were a local mirror, and thus upgrades and checksum fetches from the origin registry would be blocked. - Because of the previous two points, someone who _was_ trying to use the legacy cache directory as a filesystem mirror would see installation fail for any providers they manually added to the legacy directory. To avoid leaving that user stumped as to what went wrong, there's a heuristic for the case where a non-official provider fails installation and yet we can see it in the legacy cache directory. If that heuristic matches then we'll produce a warning message hinting to move the provider under the terraform.d/plugins directory, which is a _correct_ location for "bundled" provider plugins that belong only to a single configuration (as opposed to being installed globally on a system). This does unfortunately mean that anyone who was following the incorrectly-documented pattern will now encounter an error (and the aforementioned warning hint) after upgrading to Terraform v0.14. This seems like the safest compromise because Terraform can't automatically infer the intent of files it finds in .terraform/plugins in order to decide automatically how best to handle them. The internals of the .terraform directory are always considered implementation detail for a particular Terraform version and so switching to a new directory for the _actual_ cache directory fits within our usual set of guarantees, though it's definitely non-ideal in isolation but okay when taken in the broader context of this problem, where the alternative would be silent misbehavior when upgrading.	5 years ago
James Bardin	d2514a9abd	update new outputs plan json	5 years ago
Pam Selle	bc57c20d10	Remove sensitive_variables experiment ... Ahead of the beta, remove the sensitive_variable experiment and update tests accordingly	5 years ago
Pam Selle	c57ca152e6	Obfuscate sensitive vals in console ... Updates terraform console to show "(sensitive)" when a value is marked as sensitive.	5 years ago
Kristin Laemmert	479655ad47	refactor tests to use modern states.State in favor of terraform.State where possible	5 years ago
Martin Atkins	ff0dbd6215	command/fmt: Restore some opinionated behaviors ... In Terraform 0.11 and earlier, the "terraform fmt" command was very opinionated in the interests of consistency. While that remains its goal, for pragmatic reasons Terraform 0.12 significantly reduced the number of formatting behaviors in the fmt command. We've held off on introducing 0.12-and-later-flavored cleanups out of concern it would make it harder to maintain modules that are cross-compatible with both Terraform 0.11 and 0.12, but with this aimed to land in 0.14 -- two major releases later -- our new goal is to help those who find older Terraform language examples learn about the more modern idiom. More rules may follow later, now that the implementation is set up to allow modifications to tokens as well as modifications to whitespace, but for this initial pass the command will now apply the following formatting conventions: - 0.11-style quoted variable type constraints will be replaced with their 0.12 syntax equivalents. For example, "string" becomes just string. (This change quiets a deprecation warning.) - Collection type constraints that don't specify an element type will be rewritten to specify the "any" element type explicitly, so list becomes list(any). - Arguments whose expressions consist of a quoted string template with only a single interpolation sequence inside will be "unwrapped" to be the naked expression instead, which is functionally equivalent. (This change quiets a deprecation warning.) - Block labels are given in quotes. Two of the rules above are coming from a secondary motivation of continuing down the deprecation path for two existing warnings, so authors can have two active deprecation warnings quieted automatically by "terraform fmt", without the need to run any third-party tools. All of these rules match with current documented idiom as shown in the Terraform documentation, so anyone who follows the documented style should see no changes as a result of this. Those who have adopted other local style will see their configuration files rewritten to the standard Terraform style, but it should not make any changes that affect the functionality of the configuration. There are some further similar rewriting rules that could be added in future, such as removing 0.11-style quotes around various keyword or static reference arguments, but this initial pass focused only on some rules that have been proven out in the third-party tool terraform-clean-syntax, from which much of this commit is a direct port. For now this doesn't attempt to re-introduce any rules about vertical whitespace, even though the 0.11 "terraform fmt" would previously apply such changes. We'll be more cautious about those because the results of those rules in Terraform 0.11 were often sub-optimal and so we'd prefer to re-introduce those with some care to the implications for those who may be using vertical formatting differences for some semantic purpose, like grouping together related arguments.	5 years ago
Alisdair McDiarmid	14a233b019	command: Taint should respect required_version ... Despite not requiring the configuration for any other reason, the taint subcommand should not execute if the required_version constraints cannot be met. Doing so can result in an undesirable state file upgrade.	5 years ago
Alisdair McDiarmid	18f9ea53b9	command: Providers schema shows required_providers ... The providers schema command is using the Config.ProviderTypes method, which had not been kept up to date with the changes to provider requirements detection made in Config.ProviderRequirements. This resulted in any currently-unused providers being omitted from the output. This commit changes the ProviderTypes method to use the same underlying logic as ProviderRequirements, which ensures that `required_providers` blocks are taken into account. Includes an integration test case to verify that this fixes the provider schemas command bug.	5 years ago
James Bardin	3b3ff98356	Revert "fix show -json tests" ... This reverts commit e54949f2e1. Changes incorrectly applied to the planned state tests	6 years ago
James Bardin	e54949f2e1	fix show -json tests ... The prior state recorded in the plans did not match the actual prior state. Make the plans and state match depending on whether there was existing state or not.	6 years ago
Alisdair McDiarmid	9f824c53a5	command: Better in-house provider install errors ... When init attempts to install a legacy provider required by state and fails, but another provider with the same type is successfully installed, this almost definitely means that the user is migrating an in-house provider. The solution here is to use the `terraform state replace-provider` subcommand. This commit makes that next step clearer, by detecting this specific case, and displaying a list of commands to fix the existing state provider references.	6 years ago
Alisdair McDiarmid	fc7e467d19	command: Add redirect support to 0.13upgrade ... If a provider changes namespace in the registry, we can detect this when running the 0.13upgrade command. As long as there is a version matching the user's constraints, we now use the provider's new source address. Otherwise, warn the user that the provider has moved and a version upgrade is necessary to move to it.	6 years ago
Alisdair McDiarmid	677aabc767	command: Fix backend config override validation ... When loading a backend config override file, init was doing two things wrong: - First, if the file failed to parse, we accidentally didn't return, which caused a panic due to the parsed body being nil; - Secondly, we were overzealous with the validation of the file, allowing only attributes. While most backend configs are attributes only, the enhanced remote backend body also contains a `workspaces` block, which we need to support here. This commit fixes the first bug with an early return and adds test cases for missing file and intentionally-blank filename (to clear the config). We also add a schema validation for the backend block, based on the backend schema itself. This requires constructing an HCL body schema so that we can call `Content` and check for diagnostic errors. The result is more useful errors when an invalid backend config override file is used, while also supporting the enhanced remote backend config fully. Does not include tests specific to the remote backend, because the mocking involved to allow the backend to fully initialize is too involved to be worth it.	6 years ago
Cameron Stitt	54e32652f7	Ensure depends_on is in module calls for config	6 years ago
Alisdair McDiarmid	30c7dfca62	Merge pull request #25898 from hashicorp/alisdair/fix-required-version-diags ... terraform: Fix required version constraint diags	6 years ago
Alisdair McDiarmid	c98f352dc8	terraform: Fix required version constraint diags ... If a module has multiple terraform.required_version constraints, any failures would point at the last constraint in the error diagnostics. If an earlier constraint was the actual problem, this leads to confusing errors like this: Error: Unsupported Terraform Core version on main.tf line 6, in terraform: 6: required_version = ">= 0.13.0" This configuration does not support Terraform version 0.13.0. The error was due to storing the declaration range of the constraint as a pointer to the contents of a loop variable, which was later overwritten in later iterations of the loop. Instead we now use HCL's handy Ptr() method to create a direct pointer to the range struct.	6 years ago
Alisdair McDiarmid	d8e9964363	terraform: Eval module call arguments for import ... Include the import walk in the list of operations for which we create an EvalModuleCallArgument node. This causes module call arguments to be evaluated even if the module variables have defaults, ensuring that invalid default values (such as the common "{}" for variables thought of as maps) do not cause failures specific to import. This fixes a bug where a child module evaluates an input variable in its locals block, assuming that it is a nested object structure. The bug report includes a default value of "{}", which is overridden by a root variable value. Without the eval node added in this commit, the default value is used and the local evaluation errors.	6 years ago
James Bardin	1c09df1a66	Merge pull request #25779 from hashicorp/jbardin/remove-state-attrs ... Remove resource state attributes that are no longer in the schema	6 years ago
Kristin Laemmert	6621501ae3	state: remove deprecated state package (#25490) ... Most of the state package has been deprecated by the states package. This PR replaces all the references to the old state package that can be done simply - the low-hanging fruit. * states: move state.Locker to statemgr The state.Locker interface was a wrapper around a statemgr.Full, so moving this was relatively straightforward. * command: remove unnecessary use of state package for writing local terraform state files * move state.LocalState into terraform package state.LocalState is responsible for managing terraform.States, so it made sense (to me) to move it into the terraform package. * slight change of heart: move state.LocalState into clistate instead of terraform	6 years ago
James Bardin	99cd3ab223	fix command tests ... A number of tests had invalid configs or providers, but were never properly validated	6 years ago
Kristin Laemmert	f8e3456867	command/show: fix bug displaying provider config in json output of tf plan (#25577) ... A lingering FIXME caused missing configuration from provider config blocks in the json output of terraform plan. This fixes the regression and adds a test. For the sake of testing, I added an optional attribute to the show test provider, which resulted in the providers schema test getting an update - not a bad addition, but we can always add a test-specific provider schema as needed.	6 years ago
Alisdair McDiarmid	53e587e1a6	Merge pull request #25504 from hashicorp/alisdair/post-install-cache-validate ... Add post-install provider cache validation and error reporting	6 years ago
Martin Atkins	80ab867e57	command/init: Remove special 0.12upgrade heuristic ... For Terraform v0.12 we introduced a special loading mode where we would use the 0.11-syntax-compatible "earlyconfig" package as a heuristic to identify situations where it was likely that the user was trying to use 0.11-only syntax that the upgrade tool might help with. However, as the language has moved on that is no longer a suitable heuristic in Terraform 0.13 and later: other new additions to the language can cause the main loader to disagree with earlyconfig, which would lead us to give poor advice about how to respond. Instead, we'll now return the same generic "there are errors" message in all syntax error cases. We have an extra message for errors in this case (as compared to other commands) because "terraform init" is usually the first command a new user interacts with and so this message gives some extra explanation about what "terraform init" will do with the configuration once it's valid. This also includes a reset control character in the output of the message as part of our ongoing mission to stop Terraform printing out whole paragraphs of colored text, which can often be hard to read for various reasons.	6 years ago
Alisdair McDiarmid	87d1fb4006	command/init: Display provider validation errors ... After installing providers, we validate the presence of an executable file, and generate a selected versions lockfile. If this process fails, notify the user. One possible cause for this is an invalid provider package with a missing or misnamed executable file.	6 years ago
Kristin Laemmert	df244b87c2	command/init: return an error with invalid -backend-config files (#25411) ... * command/init: return an error with invalid -backend-config files The -backend-config flag expects a set of key-value pairs or a file containing key-value pairs. If the file instead contains a full backend configuration block, it was silently ignored. This commit adds a check for blocks in the file and returns an error if they are encountered. Fixes #24845 * emphasize backend configuration file in docs	6 years ago
Alisdair McDiarmid	b6739829e7	command: Fix 0.13upgrade to preserve more comments ... Previously, any comments inside the required provider configuration for a given provider would be wiped out upon rerunning the 0.13upgrade command. This commit attempts to preserve those comments if the existing entry is semantically equivalent to the entry we are about to write.	6 years ago
Kristin Laemmert	b611bd7209	reproduction test	6 years ago
Alisdair McDiarmid	08b735984a	Merge pull request #25191 from hashicorp/alisdair/better-provider-upgrade-hints-on-init ... command/init: Improve diags for legacy providers	6 years ago
Martin Atkins	17feb2abfc	vendor: go get github.com/apparentlymart/go-versions@v1.0.0 ... This new version permits omitting the space between the operator and the boundary in a ruby-style version constraint, like ">1.0.0" instead of "> 1.0.0".	6 years ago
Alisdair McDiarmid	9263b28e99	command/init: Improve diags for legacy providers ... When initializing a configuration which refers to re-namespaced legacy providers, we attempt to detect this and display a diagnostic message. Previously this message would direct the user to run the 0.13upgrade command, but without specifying in which directories. This commit detects which modules are using the providers in question, and for local modules displays a list of upgrade commands which specify the source directories of these modules. For remote modules, we display a separate list noting that they need to be upgraded elsewhere, providing both the local module call name and the module source address.	6 years ago
Kristin Laemmert	5450e8515d	command/013upgrade: detect builtin terraform provider (#25215) ... * command/013upgrade: detect builtin terraform provider	6 years ago
Alisdair McDiarmid	1c1e4a4de0	command/providers: Show provider requirements tree ... Providers can be required from multiple sources. The previous implementation of the providers sub-command displayed only a flat list of provider requirements, which made it difficult to see which modules required each provider. This commit reintroduces the tree display of provider requirements, and adds a separate output block for providers required by existing state.	6 years ago
Alisdair McDiarmid	ca40107066	command/init: Better diagnostics for provider 404s ... Fetching a default namespace provider from the public registry can result in 404 Not Found error. This might be caused by a previously- default provider moving to a new namespace, which means that the configuration needs to be upgraded to use an explicit provider source. This commit adds a more detailed diagnostic for this situation, suggesting that the intended provider might be in a new namespace. The recommended course of action is to run the 0.13upgrade command to generate the correct required_providers configuration.	6 years ago
Alisdair McDiarmid	62d826e066	command/init: Use full config for provider reqs ... Relying on the early config for provider requirements was necessary in Terraform 0.12, to allow the 0.12upgrade command to run after init installs providers. However in 0.13, the same restrictions do not apply, and the detection of provider requirements has changed. As a result, the early config loader gives incorrect provider requirements in some circumstances, such as those in the new test in this commit. Therefore we are changing the init command to use the requirements found by the full configuration loader. This also means that we can remove the internal initwd CheckCoreVersionRequirements function.	6 years ago
Alisdair McDiarmid	1fdcbc4825	command: Fix 0.13upgrade bug with multiple blocks ... If a configuration had multiple blocks in the versions.tf file, it would be added to the `rewritePaths` list multiple times. We would then remove it from this slice, but only once, and so the output file would later be rewritten to remove the required providers block. This commit uses a set instead of a list to prevent this case, and adds a regression test.	6 years ago
Alisdair McDiarmid	a740b739e0	command: Change 0.13upgrade default to versions.tf ... Instead of using providers.tf as the default output file for the upgrader, we now default to versions.tf. This means that if the configuration has no `required_providers` blocks at all, or has multiple, the provider version requirements will be stored in the versions.tf file. We now also update the versions.tf file to set a `required_version` attribute in the first `terraform` block, with value ">= 0.13". This is similar to the behaviour of the 0.12upgrade command, and signals that the configuration should not be used with older versions of Terraform.	6 years ago
Alisdair McDiarmid	01a3376ead	command: Check required_version before upgrading ... If a configuration has a version constraint which prevents use with Terraform 0.13, the upgrade command should exit before making any changes.	6 years ago
Alisdair McDiarmid	ae98bd12a7	command: Rework 0.13upgrade sub-command ... This commit implements most of the intended functionality of the upgrade command for rewriting configurations. For a given module, it makes a list of all providers in use. Then it attempts to detect the source address for providers without an explicit source. Once this step is complete, the tool rewrites the relevant configuration files. This results in a single "required_providers" block for the module, with a source for each provider. Any providers for which the source cannot be detected (for example, unofficial providers) will need a source to be defined by the user. The tool writes an explanatory comment to the configuration to help with this.	6 years ago
Mike Morris	9568de6b90	command: Fix OAuth2 PKCE arguments ... Providers like Okta and AWS Cognito expect that the PKCE challenge uses base64 URL Encoding without any padding (base64.RawURLEncoding) Additionally, Okta strictly adheres to section 4.2 of RFC 7636 and requires that the unencoded key for the PKCE data is at least 43 characters in length.	6 years ago
Pam Selle	87bce5f9dd	Support reading module outputs in terraform console (#24808) ... * Include eval in output walk This allows outputs to be evaluated in the evalwalk, impacting terraform console. Outputs are still not evaluated for terraform console in the root module, so this has no impact on writing to state (as child module outputs are not written to state). Also adds test coverage to the console command, including for evaluating locals (another use of the evalwalk)	6 years ago
Kristin Laemmert	f09ae6f862	provider source tests: added test suite to exercise hyphenated providers (#24685)	6 years ago
Kristin Laemmert	269d511481	command/providers: refactor with new provider types and functions ... The providers command has been refactored to use the modern provider types and ProviderRequirements() functions. This resulted in a breaking change to the output: it no longer outputs the providers by module and no longer prints `(inherited)` or `(from state)` to show why a provider is included. We decided that at this time it was best to stick with the existing functions and make this change, but if we get feedback from the community we will revisit. Additional tests to exercise providers in modules and providers from state have been included.	6 years ago
Kristin Laemmert	0a5fb40fdf	configs: include provider configs in ProviderRequirements() ... This PR adds iteration through any provider configuration blocks in the config in addProviderRequirements(). A stale comment (of mine!) would leave one expecting the module.ProviderRequirements to include any requirements from provider configs. The comment was inaccurate and has been updated.	6 years ago
Kristin Laemmert	5852a5c33d	Mildwonkey/last tests (#24553) ... * show text fixture update * temporarily disable providers tests	6 years ago
Martin Atkins	958ea4f7d1	internal/providercache: Handle built-in providers ... Built-in providers are special providers that are distributed as part of Terraform CLI itself, rather than being installed separately. They always live in the terraform.io/builtin/... namespace so it's easier to see that they are special, and currently there is only one built-in provider named "terraform". Previous commits established the addressing scheme for built-in providers. This commit makes the installer aware of them to the extent that it knows not to try to install them the usual way and it's able to report an error if the user requests a built-in provider that doesn't exist or tries to impose a particular version constraint for a built-in provider. For the moment the tests for this are the ones in the "command" package because that's where the existing testing infrastructure for this functionality lives. A later commit should add some more focused unit tests here in the internal/providercache package, too.	6 years ago