68 Commits (paddy_module_attribution_tmp)

Author	SHA1	Message	Date
Kristin Laemmert	18dd1bb4d6	Mildwonkey/tfconfig upgrade (#23670) ... * deps: bump terraform-config-inspect library * configs: parse `version` in new required_providers block With the latest version of `terraform-config-inspect`, the required_providers attribute can now be a string or an object with attributes "source" and "version". This change allows parsing the version constraint from the new object while ignoring any given source attribute.	6 years ago
Kristin Laemmert	a16e1fc0a1	command/init: omit a warning if -backend-config is used with no backend (#22164) ... * command/init: omit a warning if -backend-config is used with no backend block Terraform would silently accept - and swallow - `-backend-config` on the CLI when there was no `backend` block. Since it is mostly expected to override existing backend configuration, terraform should omit a warning if there is no backend configuration to override. If the user intended to override the default (local) backend configuration, they can first add a `backend` block to the `terraform` block to silence the warning (or just ignore it): ```hcl terraform { backend "local" {} } ```	7 years ago
James Bardin	06dfc4abd8	allow setting -backend-config='' to unset override ... There is currently no way to unset -backend-config during init, since not setting that option assumes the user will use the saved config. Allow setting `-backend-config=""` to specify no overrides.	7 years ago
James Bardin	c017149b31	don't store prepared backend config ... The backend gets to "prepare" the configuration before Configure is called, in order to validate the values and insert defaults. We don't want to store this value in the "config state", because it will often not match the raw config after it is prepared, forcing unecessary backend migrations during init. Since PrepareConfig is always called before Configure, we can store the config value directly, and assume that it will be prepared in the same manner each time.	7 years ago
James Bardin	ee9a618369	don't migrate backend during init without override ... If the backend config hashes match during init, and there are no new backend override options, then we assume the existing config is OK. Since init should be idempotent, we should be able to run init with no options or config changes, and not effect the backends at all.	7 years ago
James Bardin	cefc927e48	failing test for backend re-init	7 years ago
Martin Atkins	63fa1ac418	vendor: go get github.com/hashicorp/hcl2@master ... This includes a small fix to ensure the parser doesn't produce an invalid body for block parsing syntax errors, and instead produces an incomplete result that calling applications like Terraform can still analyze. The problem here was affecting our version-constraint-sniffing code, which intentionally tried to find a core version constraint even if there's a syntax error so that it can report that a new version of Terraform is a likely cause of the syntax error. It was working in most cases, unless it was the "terraform" block itself that contained the error, because then we'd try to analyze a broken hcl.Block with a nil body. This includes a new test for "terraform init" that exercises this recovery codepath.	7 years ago
Justin Campbell	24e13d8ec1	plugin/discovery: Return tfdiags from Get ... Allows us to surface warnings to the user using the tfdiags interfaces.	7 years ago
Radek Simko	37f397dded	command/test: Clarify failure (add full path)	7 years ago
Martin Atkins	86c02d5c35	command: "terraform init" can partially initialize for 0.12upgrade ... There are a few constructs from 0.11 and prior that cause 0.12 parsing to fail altogether, which previously created a chicken/egg problem because we need to install the providers in order to run "terraform 0.12upgrade" and thus fix the problem. This changes "terraform init" to use the new "early configuration" loader for module and provider installation. This is built on the more permissive parser in the terraform-config-inspect package, and so it allows us to read out the top-level blocks from the configuration while accepting legacy HCL syntax. In the long run this will let us do version compatibility detection before attempting a "real" config load, giving us better error messages for any future syntax additions, but in the short term the key thing is that it allows us to install the dependencies even if the configuration isn't fully valid. Because backend init still requires full configuration, this introduces a new mode of terraform init where it detects heuristically if it seems like we need to do a configuration upgrade and does a partial init if so, before finally directing the user to run "terraform 0.12upgrade" before running any other commands. The heuristic here is based on two assumptions: - If the "early" loader finds no errors but the normal loader does, the configuration is likely to be valid for Terraform 0.11 but not 0.12. - If there's already a version constraint in the configuration that excludes Terraform versions prior to v0.12 then the configuration is probably _already_ upgraded and so it's just a normal syntax error, even if the early loader didn't detect it. Once the upgrade process is removed in 0.13.0 (users will be required to go stepwise 0.11 -> 0.12 -> 0.13 to upgrade after that), some of this can be simplified to remove that special mode, but the idea of doing the dependency version checks against the liberal parser will remain valuable to increase our chances of reporting version-based incompatibilities rather than syntax errors as we add new features in future.	7 years ago
James Bardin	149ccd929e	missing commits from 19688	7 years ago
Martin Atkins	8c54da0ad2	command: TestInit_fromModule_explicitDest guard against other tests ... Some other test is leaving behind a terraform.tfstate after it concludes, which can cause this test to fail in a strange way due to picking up extra provider requirements from that state. This check doesn't fix that problem, but it at least makes the test fail in a more helpful way to avoid time wasted trying to debug this test when it's some other test that actually has the bug.	7 years ago
Martin Atkins	544c2932ce	command: Fix TestInit_checkRequiredVersion ... In prior refactoring we lost the required core version check from "terraform init", which we restore here. Additionally, this test used to have an incorrect name that suggested it was testing something in the "getProvider" codepath, but version checking happens regardless of what other options are selected.	7 years ago
Martin Atkins	9ba399bca8	command: Fix TestInit_getProvider ... After all of the refactoring we were no longer checking the Terraform version field in a state file, causing this test to fail. This restores that check, though with a slightly different error message.	7 years ago
Martin Atkins	f6d468ffd5	command: Fix TestInit_inputFalse ... This test was using old-style state files as its input, differing only by lineage. Since lineages are now managed within the state manager itself, the test can't use that to distinguish the two files and so we put a different output in each one instead. This also introduces some TRACE logging to the migration codepaths. There's some hard-to-follow control flow here and so this extra logging helps to understand the reason for a particular outcome, and since this codepath is visited only in "terraform init" anyway it doesn't hurt to be a bit more verbose here.	7 years ago
Martin Atkins	297b3b8830	command: Fix TestInit_backendReinitConfigToExtra ... This test was re-using the same InitCommand value to run multiple times, which is not realistic. Since we now cache configuration source code inside command.Meta on load, it's important that we use a fresh InitCommand instance here so it'll see the modified configuration file we've left on disk.	7 years ago
Martin Atkins	7b77e20bdc	command: Fix several "terraform init" tests	7 years ago
Martin Atkins	a3403f2766	terraform: Ugly huge change to weave in new State and Plan types ... Due to how often the state and plan types are referenced throughout Terraform, there isn't a great way to switch them out gradually. As a consequence, this huge commit gets us from the old world to a _compilable_ new world, but still has a large number of known test failures due to key functionality being stubbed out. The stubs here are for anything that interacts with providers, since we now need to do the follow-up work to similarly replace the old terraform.ResourceProvider interface with its replacement in the new "providers" package. That work, along with work to fix the remaining failing tests, will follow in subsequent commits. The aim here was to replace all references to terraform.State and its downstream types with states.State, terraform.Plan with plans.Plan, state.State with statemgr.State, and switch to the new implementations of the state and plan file formats. However, due to the number of times those types are used, this also ended up affecting numerous other parts of core such as terraform.Hook, the backend.Backend interface, and most of the CLI commands. Just as with 5861dbf3fc before, I apologize in advance to the person who inevitably just found this huge commit while spelunking through the commit history.	7 years ago
Martin Atkins	ebafa51723	command: Various updates for the new backend package API ... This is a rather-messy, complex change to get the "command" package building again against the new backend API that was updated for the new configuration loader. A lot of this is mechanical rewriting to the new API, but meta_config.go and meta_backend.go in particular saw some major changes to interface with the new loader APIs and to deal with the change in order of steps in the backend API.	7 years ago
James Bardin	e980156451	cleanup temp files from command tests ... Rather than try to modify all the hundreds of calls to the temp helper functions, and cleanup the temp files at every call site, have all tests work within a single temp directory that is removed at the end of TestMain.	8 years ago
James Bardin	61c2be3e95	Merge pull request #16969 from hashicorp/jbardin/reset-plugin-dir ... allow init to reset -plugin-dir	8 years ago
James Bardin	504ea578ee	Merge pull request #16920 from hashicorp/jbardin/init-future-state ... check state version during init	8 years ago
James Bardin	5a975d9997	add test for internal plugins with -plugin-dir ... -plugin-dir was short-circuiting the discovery for internal plugins	8 years ago
James Bardin	ba84faf4e1	remove the plugin path only for an empty string ... To avoid breaking automation where plugin-path was assumed to be set permanently, only remove the plugin-path record if it was explicitly set to and empty string.	8 years ago
James Bardin	79e985366f	allow init to reset -plugin-dir ... Remove the recorded -plugin-dir during init if the flag is not provided.	8 years ago
James Bardin	7d2da9865e	inputFalse test should attempt migration and check error ... Make sure the init inputFalse test actually errors from missing input, since skipping input will still fail later during provider initialization. We need to make sure there are two different states that aren't a noop for migration, and reset the command struct for each run. Also verify that we don't go into an infinite loop if there is no input.	8 years ago
James Bardin	ea4cb6a20e	check state version during init ... The init command needs to parse the state to resolve providers, but changes to the state format can cause that to fail with difficult to understand errors. Check the terraform version during init and provide the same error that would be returned by plan or apply.	8 years ago
James Bardin	1bf64ec788	add "Updating" output and fix output tests	8 years ago
Sunny	2d849f8650	command/init: check required_version ... Previously we were checking required_version only during "real" operations, and not during initialization. Catching it during init is better because that's the first command users run on a new working directory.	9 years ago
James Bardin	fa20d43d80	test loading of Meta.PluginOverrides ... These are currently being skipped in discovery	9 years ago
Martin Atkins	8a7a0a7459	command: terraform init -from-module=... ... This restores the earlier behavior of the first positional argument to terraform init in 0.9, but as a command line option. The positional argument was removed to improve consistency with other commands that take a working directory as their first positional argument. It was originally intended that this functionality would return in a later release along with some other general improvements to Terraform's module handling, but we're introducing here an interim solution that uses the existing module source concept, to allow for easier porting of workflows that previously depended on the automatic copy behavior. In a future release this feature may change again as the module improvements design firms up, but we expect it to be broadly compatible with this temporary state.	9 years ago
Martin Atkins	40e2fbb8e9	command: init to allow plugin init without backend init ... Previously init would crash if given these options: -backend=false -get-plugins=true This is because the state is used as a source of provider dependency information, and we need to instantiate the backend to get the state. To avoid the crash, we now use the following adjusted behavior: - if -backend=true, we behave as before - if -backend=false, we instead try to instantiate the backend the same way any other command would, without modifying its configuration - if we're able to instantiate the backend, we use it to fetch state for dependency resolution purposes - if the backend is not instantiable then we assume it's not yet configured and proceed with a nil state, which may cause us to see an incomplete picture of the dependencies but still allows the install to succeed. Subsequently running "terraform plan" will not work until the backend is (re-)initialized, so the incomplete picture of required plugins is safe.	9 years ago
Martin Atkins	9c2fe3456b	command: purge unused plugins as a side effect of plugin installation ... Previously we only did this when _upgrading_, but that's unnecessarily specific and confusing since e.g. plugins can get upgraded implicitly by constraint changes, which would not then trigger the purge process. Instead, we'll assume that the user is able to easily re-download plugins that were purged here, or if they need more specific guarantees they will manage manually a plugin directory and disable the auto-install behavior using `-plugin-dir`.	9 years ago
James Bardin	270eedd4b8	always pass in the full plugin path to dicovery ... Discovery no longer tries to walk into OS_ARCH dirs, so always pass in the full search path.	9 years ago
James Bardin	be2069ac81	add -plugin-dir option ... The -plugin-dir option lets the user specify custom search paths for plugins. This overrides all other plugin search paths, and prevents the auto-installation of plugins. We also make sure that the availability of plugins is always checked during init, even if -get-plugins=false or -plugin-dir is set.	9 years ago
James Bardin	f723270e3e	search the vendor directory for plugins ... The default location for users to manually add plugins will be ./terraform.d/plugins/	9 years ago
James Bardin	4f5e92e4c0	reverse init test to check for dataDir in PWD ... init should always write intternal data to the current directory, even when a path is provided. The inherited behavior no longer applies to the new use of init.	9 years ago
Martin Atkins	6979a07754	command: init -upgrade for provider plugins ... Now when -upgrade is provided to "terraform init" (and plugin installation isn't disabled) it will: - ignore the contents of the auto-install plugin directory when deciding what is "available", thus causing anything there to be reinstalled, possibly at a newer version. - if installation completes successfully, purge from the auto-install plugin directory any plugin-looking files that aren't in the set of chosen plugins. As before, plugins outside of the auto-install directory are able to take precedence over the auto-install ones, and these will never be upgraded nor purged. The thinking here is that the auto-install directory is an implementation detail directly managed by Terraform, and so it's Terraform's responsibility to automatically keep it clean as plugins are upgraded. We don't yet have the -plugin-dir option implemented, but once it is it should circumvent all of this behavior and just expect providers to be already available in the given directory, meaning that nothing will be auto-installed, -upgraded or -purged.	9 years ago
Martin Atkins	f753974bb3	plugin/discovery: Installer interface, and provider implementation ... Previously we had a "getProvider" function type used to implement plugin fetching. Here we replace that with an interface type, initially with just a "Get" function. For now this just simplifies the interface by allowing the target directory and protocol version to be members of the struct rather than passed as arguments. A later change will extend this interface to also include a method to purge unused plugins, so that upgrading frequently doesn't leave behind a trail of unused executable files.	9 years ago
Gavin Williams	5834333ea3	command: terraform get -upgrade ... As of this commit this just upgrades modules, but this option will also later upgrade plugins and indeed anything else that's being downloaded and installed as part of the init.	9 years ago
James Bardin	2bfcdbbd08	change init args to remove source copy ... When init was modified in 0.9 to initialize a terraform working directory, the legacy behavior was kept to copy or fetch module sources. This left the init command without the ability that the plan and apply commands have to target a specific directory for the operation. This commit removes the legacy behavior altogether, and allows init to target a directory for initialization, bringing it into parity with plan and apply. If one want to copy a module to the target or current directory, that will have to be done manually before calling init. We can later reintroduce fetching modules with init without breaking this new behavior, by adding the source as an optional second argument. The unit tests testing the copying of sources with init have been removed, as well as some out of date (and commented out) init tests regarding remote states.	9 years ago
James Bardin	d1c50efb01	change providers.json to lock.json ... It might not just be for providers, and it's in the plugins dir, so lock.json seems descriptive enough.	9 years ago
James Bardin	fdbfc17fae	missing constraints passed erroniously ... ConstrainVersions was documented as returning nil, but it was instead returning an empty set. Use the Count() method to check for nil or empty. Add test to verify failed constraints will show up as missing.	9 years ago
Martin Atkins	032f71f1ff	command: produce provider lock file during "terraform init" ... Once we've installed the necessary plugins, we'll do one more walk of the available plugins and record the SHA256 hashes of all of the plugins we select in the provider lock file. The file we write here gets read when we're building ContextOpts to initialize the main terraform context, so any command that works with the context will then fail if any of the provider binaries change.	9 years ago
James Bardin	2994c6ac5d	add init getPlugin test ... add a mock plugin getter, and test that we can fetch requested version of the plugins.	9 years ago
Martin Atkins	8364383c35	Push plugin discovery down into command package ... Previously we did plugin discovery in the main package, but as we move towards versioned plugins we need more information available in order to resolve plugins, so we move this responsibility into the command package itself. For the moment this is just preserving the existing behavior as long as there are only internal and unversioned plugins present. This is the final state for provisioners in 0.10, since we don't want to support versioned provisioners yet. For providers this is just a checkpoint along the way, since further work is required to apply version constraints from configuration and support additional plugin search directories. The automatic plugin discovery behavior is not desirable for tests because we want to mock the plugins there, so we add a new backdoor for the tests to use to skip the plugin discovery and just provide their own mock implementations. Most of this diff is thus noisy rework of the tests to use this new mechanism.	9 years ago
James Bardin	50023e9a60	honor `input=false` in state migration ... return an error when confirming a copy if -input=false	9 years ago
James Bardin	c891ab50b7	detect when backend.Hash needs update ... It's possible to not change the backend config, but require updating the stored backend state by moving init options from the config file to the `-backend-config` flag. If the config is the same, but the hash doesn't match, update the stored state.	9 years ago
James Bardin	ff2d753062	add Rehash to terraform.BackendState ... This method mirrors that of config.Backend, so we can compare the configration of a backend read from a config vs that of a backend read from a state. This will prevent init from reinitializing when using `-backend-config` options that match the existing state.	9 years ago
James Bardin	54e536cfe0	add `-force-copy` option to init command ... The `-force-copy` option will suppress confirmation for copying state data. Modify some tests to use the option, making sure to leave coverage of the Input code path.	9 years ago