79 Commits (c945ef129a40366013d7c34f04c8b2fddb71db26)

Author	SHA1	Message	Date
Martin Atkins	958ea4f7d1	internal/providercache: Handle built-in providers ... Built-in providers are special providers that are distributed as part of Terraform CLI itself, rather than being installed separately. They always live in the terraform.io/builtin/... namespace so it's easier to see that they are special, and currently there is only one built-in provider named "terraform". Previous commits established the addressing scheme for built-in providers. This commit makes the installer aware of them to the extent that it knows not to try to install them the usual way and it's able to report an error if the user requests a built-in provider that doesn't exist or tries to impose a particular version constraint for a built-in provider. For the moment the tests for this are the ones in the "command" package because that's where the existing testing infrastructure for this functionality lives. A later commit should add some more focused unit tests here in the internal/providercache package, too.	6 years ago
Martin Atkins	de6c9ccec1	command/init: Move "vendored provider" test to e2etests ... In the new design the ProviderSource is decided by package main, not by the "command" package, and so making sure the vendor directory is included is the responsibility of that package instead. Therefore we can no longer test this at the "command" package level, but we'll retain a test for it in e2etests to record that it isn't currently working, so that we have a prompt to fix it before releasing.	6 years ago
Martin Atkins	fe9105abfe	command: Fix TestInit_getProviderMissing ... This one just needs a tweak for the new wording of the error message.	6 years ago
Martin Atkins	add7006de6	command: Fix TestInit_pluginDirProviders and _pluginDirProvidersDoesNotGet ... Both of these are attempting to test -plugin-dir, which means we need some additional help to populate some suitable directories for -plugin-dir to refer to. The new installFakeProviderPackagesElsewhere helper generalizes the earlier installFakeProviderPackages to allow installing fake provider packages to an arbitrary other directory.	6 years ago
Martin Atkins	c81eebe0ac	command/init: Fix TestInit_providerSource ... This test is focused on making sure that the required_providers syntax is working, so the rewritten version does not include any special handling of pre-installed packages or "vendored" packages. Pre-installed plugins are tested in other tests such as TestInit_getUpgradePlugins.	6 years ago
Martin Atkins	14701b8300	command/init: Fix TestInit_getUpgradePlugins ... This test now requires a bit of a different approach because it was previously directly constructing a cache directory but we now use a different directory layout. Rather than manually constructing the new heirarchical directory layout (which would've required a lot more inline code), this introduces a helper function installFakeProviderPackages that installs a fake provider package directly into the local cache directory associated with a Meta object, with the correct directory layout.	6 years ago
Martin Atkins	d40085f374	command: Make the tests compile again ... They still aren't passing, but this is just enough updating to make the test program compile successfully after the refactoring related to provider installation. They are now using the mock provider source offered by the getproviders package, which is similar but not totally identical to the idea of mocking the entire installer as these tests used to do, and so many of them need further adjustment to still be testing what they intended to test under this new architecture. Subsequent commits will gradually repair the failing tests.	6 years ago
Kristin Laemmert	4f141ae365	rename provider-specific functions (#24417) ... missingPlugins was hard-coded to work only with provider plugins, so I renamed it to clarify the usage. Also renamed a test provider from greater_than to greater-than as the underscore is an invalid provider name character and this will become a hard error in the near future.	6 years ago
Kristin Laemmert	5f313a65ad	command: remove 0.12upgrade (#24403) ... * command: remove 0.12upgrade and related `configupgrade` library * leave deprecation warning for 0.12upgrade to point users to v0.12	6 years ago
Kristin Laemmert	5901952882	command: tests should not leave dirs behind thank you (#24340)	6 years ago
Kristin Laemmert	92f427779d	command/show: fix issue with show and aliased provider (#23848) ... The formatter in `command/format/state.go`, when formatting a resource with an aliased provider, was looking for a schema with the alias (ie, test.foo), but the schemas are only listed by provider type (test). Update the state formatter to lookup schemas by provider type only. Some of the show tests (and a couple others) were not properly cleaning up the created tmpdirs, so I fixed those. Also, the show tests are using a statefile named `state.tfstate`, but were not passing that path to the show command, so we were getting some false positives (a `show` command that returns `no state` exits 0). Fixes #21462	6 years ago
Kristin Laemmert	18dd1bb4d6	Mildwonkey/tfconfig upgrade (#23670) ... * deps: bump terraform-config-inspect library * configs: parse `version` in new required_providers block With the latest version of `terraform-config-inspect`, the required_providers attribute can now be a string or an object with attributes "source" and "version". This change allows parsing the version constraint from the new object while ignoring any given source attribute.	6 years ago
Kristin Laemmert	a16e1fc0a1	command/init: omit a warning if -backend-config is used with no backend (#22164) ... * command/init: omit a warning if -backend-config is used with no backend block Terraform would silently accept - and swallow - `-backend-config` on the CLI when there was no `backend` block. Since it is mostly expected to override existing backend configuration, terraform should omit a warning if there is no backend configuration to override. If the user intended to override the default (local) backend configuration, they can first add a `backend` block to the `terraform` block to silence the warning (or just ignore it): ```hcl terraform { backend "local" {} } ```	7 years ago
James Bardin	06dfc4abd8	allow setting -backend-config='' to unset override ... There is currently no way to unset -backend-config during init, since not setting that option assumes the user will use the saved config. Allow setting `-backend-config=""` to specify no overrides.	7 years ago
James Bardin	c017149b31	don't store prepared backend config ... The backend gets to "prepare" the configuration before Configure is called, in order to validate the values and insert defaults. We don't want to store this value in the "config state", because it will often not match the raw config after it is prepared, forcing unecessary backend migrations during init. Since PrepareConfig is always called before Configure, we can store the config value directly, and assume that it will be prepared in the same manner each time.	7 years ago
James Bardin	ee9a618369	don't migrate backend during init without override ... If the backend config hashes match during init, and there are no new backend override options, then we assume the existing config is OK. Since init should be idempotent, we should be able to run init with no options or config changes, and not effect the backends at all.	7 years ago
James Bardin	cefc927e48	failing test for backend re-init	7 years ago
Martin Atkins	63fa1ac418	vendor: go get github.com/hashicorp/hcl2@master ... This includes a small fix to ensure the parser doesn't produce an invalid body for block parsing syntax errors, and instead produces an incomplete result that calling applications like Terraform can still analyze. The problem here was affecting our version-constraint-sniffing code, which intentionally tried to find a core version constraint even if there's a syntax error so that it can report that a new version of Terraform is a likely cause of the syntax error. It was working in most cases, unless it was the "terraform" block itself that contained the error, because then we'd try to analyze a broken hcl.Block with a nil body. This includes a new test for "terraform init" that exercises this recovery codepath.	7 years ago
Justin Campbell	24e13d8ec1	plugin/discovery: Return tfdiags from Get ... Allows us to surface warnings to the user using the tfdiags interfaces.	7 years ago
Radek Simko	37f397dded	command/test: Clarify failure (add full path)	7 years ago
Martin Atkins	86c02d5c35	command: "terraform init" can partially initialize for 0.12upgrade ... There are a few constructs from 0.11 and prior that cause 0.12 parsing to fail altogether, which previously created a chicken/egg problem because we need to install the providers in order to run "terraform 0.12upgrade" and thus fix the problem. This changes "terraform init" to use the new "early configuration" loader for module and provider installation. This is built on the more permissive parser in the terraform-config-inspect package, and so it allows us to read out the top-level blocks from the configuration while accepting legacy HCL syntax. In the long run this will let us do version compatibility detection before attempting a "real" config load, giving us better error messages for any future syntax additions, but in the short term the key thing is that it allows us to install the dependencies even if the configuration isn't fully valid. Because backend init still requires full configuration, this introduces a new mode of terraform init where it detects heuristically if it seems like we need to do a configuration upgrade and does a partial init if so, before finally directing the user to run "terraform 0.12upgrade" before running any other commands. The heuristic here is based on two assumptions: - If the "early" loader finds no errors but the normal loader does, the configuration is likely to be valid for Terraform 0.11 but not 0.12. - If there's already a version constraint in the configuration that excludes Terraform versions prior to v0.12 then the configuration is probably _already_ upgraded and so it's just a normal syntax error, even if the early loader didn't detect it. Once the upgrade process is removed in 0.13.0 (users will be required to go stepwise 0.11 -> 0.12 -> 0.13 to upgrade after that), some of this can be simplified to remove that special mode, but the idea of doing the dependency version checks against the liberal parser will remain valuable to increase our chances of reporting version-based incompatibilities rather than syntax errors as we add new features in future.	7 years ago
James Bardin	149ccd929e	missing commits from 19688	7 years ago
Martin Atkins	8c54da0ad2	command: TestInit_fromModule_explicitDest guard against other tests ... Some other test is leaving behind a terraform.tfstate after it concludes, which can cause this test to fail in a strange way due to picking up extra provider requirements from that state. This check doesn't fix that problem, but it at least makes the test fail in a more helpful way to avoid time wasted trying to debug this test when it's some other test that actually has the bug.	7 years ago
Martin Atkins	544c2932ce	command: Fix TestInit_checkRequiredVersion ... In prior refactoring we lost the required core version check from "terraform init", which we restore here. Additionally, this test used to have an incorrect name that suggested it was testing something in the "getProvider" codepath, but version checking happens regardless of what other options are selected.	7 years ago
Martin Atkins	9ba399bca8	command: Fix TestInit_getProvider ... After all of the refactoring we were no longer checking the Terraform version field in a state file, causing this test to fail. This restores that check, though with a slightly different error message.	7 years ago
Martin Atkins	f6d468ffd5	command: Fix TestInit_inputFalse ... This test was using old-style state files as its input, differing only by lineage. Since lineages are now managed within the state manager itself, the test can't use that to distinguish the two files and so we put a different output in each one instead. This also introduces some TRACE logging to the migration codepaths. There's some hard-to-follow control flow here and so this extra logging helps to understand the reason for a particular outcome, and since this codepath is visited only in "terraform init" anyway it doesn't hurt to be a bit more verbose here.	7 years ago
Martin Atkins	297b3b8830	command: Fix TestInit_backendReinitConfigToExtra ... This test was re-using the same InitCommand value to run multiple times, which is not realistic. Since we now cache configuration source code inside command.Meta on load, it's important that we use a fresh InitCommand instance here so it'll see the modified configuration file we've left on disk.	7 years ago
Martin Atkins	7b77e20bdc	command: Fix several "terraform init" tests	7 years ago
Martin Atkins	a3403f2766	terraform: Ugly huge change to weave in new State and Plan types ... Due to how often the state and plan types are referenced throughout Terraform, there isn't a great way to switch them out gradually. As a consequence, this huge commit gets us from the old world to a _compilable_ new world, but still has a large number of known test failures due to key functionality being stubbed out. The stubs here are for anything that interacts with providers, since we now need to do the follow-up work to similarly replace the old terraform.ResourceProvider interface with its replacement in the new "providers" package. That work, along with work to fix the remaining failing tests, will follow in subsequent commits. The aim here was to replace all references to terraform.State and its downstream types with states.State, terraform.Plan with plans.Plan, state.State with statemgr.State, and switch to the new implementations of the state and plan file formats. However, due to the number of times those types are used, this also ended up affecting numerous other parts of core such as terraform.Hook, the backend.Backend interface, and most of the CLI commands. Just as with 5861dbf3fc before, I apologize in advance to the person who inevitably just found this huge commit while spelunking through the commit history.	7 years ago
Martin Atkins	ebafa51723	command: Various updates for the new backend package API ... This is a rather-messy, complex change to get the "command" package building again against the new backend API that was updated for the new configuration loader. A lot of this is mechanical rewriting to the new API, but meta_config.go and meta_backend.go in particular saw some major changes to interface with the new loader APIs and to deal with the change in order of steps in the backend API.	7 years ago
James Bardin	e980156451	cleanup temp files from command tests ... Rather than try to modify all the hundreds of calls to the temp helper functions, and cleanup the temp files at every call site, have all tests work within a single temp directory that is removed at the end of TestMain.	8 years ago
James Bardin	61c2be3e95	Merge pull request #16969 from hashicorp/jbardin/reset-plugin-dir ... allow init to reset -plugin-dir	8 years ago
James Bardin	504ea578ee	Merge pull request #16920 from hashicorp/jbardin/init-future-state ... check state version during init	8 years ago
James Bardin	5a975d9997	add test for internal plugins with -plugin-dir ... -plugin-dir was short-circuiting the discovery for internal plugins	8 years ago
James Bardin	ba84faf4e1	remove the plugin path only for an empty string ... To avoid breaking automation where plugin-path was assumed to be set permanently, only remove the plugin-path record if it was explicitly set to and empty string.	8 years ago
James Bardin	79e985366f	allow init to reset -plugin-dir ... Remove the recorded -plugin-dir during init if the flag is not provided.	8 years ago
James Bardin	7d2da9865e	inputFalse test should attempt migration and check error ... Make sure the init inputFalse test actually errors from missing input, since skipping input will still fail later during provider initialization. We need to make sure there are two different states that aren't a noop for migration, and reset the command struct for each run. Also verify that we don't go into an infinite loop if there is no input.	8 years ago
James Bardin	ea4cb6a20e	check state version during init ... The init command needs to parse the state to resolve providers, but changes to the state format can cause that to fail with difficult to understand errors. Check the terraform version during init and provide the same error that would be returned by plan or apply.	8 years ago
James Bardin	1bf64ec788	add "Updating" output and fix output tests	8 years ago
Sunny	2d849f8650	command/init: check required_version ... Previously we were checking required_version only during "real" operations, and not during initialization. Catching it during init is better because that's the first command users run on a new working directory.	9 years ago
James Bardin	fa20d43d80	test loading of Meta.PluginOverrides ... These are currently being skipped in discovery	9 years ago
Martin Atkins	8a7a0a7459	command: terraform init -from-module=... ... This restores the earlier behavior of the first positional argument to terraform init in 0.9, but as a command line option. The positional argument was removed to improve consistency with other commands that take a working directory as their first positional argument. It was originally intended that this functionality would return in a later release along with some other general improvements to Terraform's module handling, but we're introducing here an interim solution that uses the existing module source concept, to allow for easier porting of workflows that previously depended on the automatic copy behavior. In a future release this feature may change again as the module improvements design firms up, but we expect it to be broadly compatible with this temporary state.	9 years ago
Martin Atkins	40e2fbb8e9	command: init to allow plugin init without backend init ... Previously init would crash if given these options: -backend=false -get-plugins=true This is because the state is used as a source of provider dependency information, and we need to instantiate the backend to get the state. To avoid the crash, we now use the following adjusted behavior: - if -backend=true, we behave as before - if -backend=false, we instead try to instantiate the backend the same way any other command would, without modifying its configuration - if we're able to instantiate the backend, we use it to fetch state for dependency resolution purposes - if the backend is not instantiable then we assume it's not yet configured and proceed with a nil state, which may cause us to see an incomplete picture of the dependencies but still allows the install to succeed. Subsequently running "terraform plan" will not work until the backend is (re-)initialized, so the incomplete picture of required plugins is safe.	9 years ago
Martin Atkins	9c2fe3456b	command: purge unused plugins as a side effect of plugin installation ... Previously we only did this when _upgrading_, but that's unnecessarily specific and confusing since e.g. plugins can get upgraded implicitly by constraint changes, which would not then trigger the purge process. Instead, we'll assume that the user is able to easily re-download plugins that were purged here, or if they need more specific guarantees they will manage manually a plugin directory and disable the auto-install behavior using `-plugin-dir`.	9 years ago
James Bardin	270eedd4b8	always pass in the full plugin path to dicovery ... Discovery no longer tries to walk into OS_ARCH dirs, so always pass in the full search path.	9 years ago
James Bardin	be2069ac81	add -plugin-dir option ... The -plugin-dir option lets the user specify custom search paths for plugins. This overrides all other plugin search paths, and prevents the auto-installation of plugins. We also make sure that the availability of plugins is always checked during init, even if -get-plugins=false or -plugin-dir is set.	9 years ago
James Bardin	f723270e3e	search the vendor directory for plugins ... The default location for users to manually add plugins will be ./terraform.d/plugins/	9 years ago
James Bardin	4f5e92e4c0	reverse init test to check for dataDir in PWD ... init should always write intternal data to the current directory, even when a path is provided. The inherited behavior no longer applies to the new use of init.	9 years ago
Martin Atkins	6979a07754	command: init -upgrade for provider plugins ... Now when -upgrade is provided to "terraform init" (and plugin installation isn't disabled) it will: - ignore the contents of the auto-install plugin directory when deciding what is "available", thus causing anything there to be reinstalled, possibly at a newer version. - if installation completes successfully, purge from the auto-install plugin directory any plugin-looking files that aren't in the set of chosen plugins. As before, plugins outside of the auto-install directory are able to take precedence over the auto-install ones, and these will never be upgraded nor purged. The thinking here is that the auto-install directory is an implementation detail directly managed by Terraform, and so it's Terraform's responsibility to automatically keep it clean as plugins are upgraded. We don't yet have the -plugin-dir option implemented, but once it is it should circumvent all of this behavior and just expect providers to be already available in the given directory, meaning that nothing will be auto-installed, -upgraded or -purged.	9 years ago
Martin Atkins	f753974bb3	plugin/discovery: Installer interface, and provider implementation ... Previously we had a "getProvider" function type used to implement plugin fetching. Here we replace that with an interface type, initially with just a "Get" function. For now this just simplifies the interface by allowing the target directory and protocol version to be members of the struct rather than passed as arguments. A later change will extend this interface to also include a method to purge unused plugins, so that upgrading frequently doesn't leave behind a trail of unused executable files.	9 years ago