82 Commits (642dadcd9681ee894f8777033c2dfd9216255e25)

Author	SHA1	Message	Date
Alisdair McDiarmid	0ed04d05aa	command: Fix command fixture modify-in-place bugs ... Some of the tests in the command package were running directly on the fixture directories, and modifying or locking files within them. This could cause state to leak between tests. This commit cleans up all such cases that I could find.	6 years ago
Alisdair McDiarmid	ca40107066	command/init: Better diagnostics for provider 404s ... Fetching a default namespace provider from the public registry can result in 404 Not Found error. This might be caused by a previously- default provider moving to a new namespace, which means that the configuration needs to be upgraded to use an explicit provider source. This commit adds a more detailed diagnostic for this situation, suggesting that the intended provider might be in a new namespace. The recommended course of action is to run the 0.13upgrade command to generate the correct required_providers configuration.	6 years ago
James Bardin	756e889697	remove unused DependsOn usage in state	6 years ago
Kristin Laemmert	27a794062e	Mildwonkey/command tests (#24535) ... * command: refactor testBackendState to write states.State testBackendState was using the older terraform.State format, which is no longer sufficient for most tests since the state upgrader does not encode provider FQNs automatically. Users will run `terraform 0.13upgrade` to update their state to include provider FQNs in resources, but tests need to use the modern state format instead of relying on the automatic upgrade. * plan tests passing * graph tests passing * json packages test update * command test updates * update show test fixtures * state show tests passing	6 years ago
Kristin Laemmert	0af09b23ca	command: apply and most of import tests passing	6 years ago
Martin Atkins	549aede792	Remove terraform.ResourceProvider, use providercache.Installer instead ... Back when we first introduced provider versioning in Terraform 0.10, we did the provider version resolution in terraform.NewContext because we weren't sure yet how exactly our versioning model was going to play out (whether different versions could be selected per provider configuration, for example) and because we were building around the limitations of our existing filesystem-based plugin discovery model. However, the new installer codepath is new able to do all of the selections up front during installation, so we don't need such a heavy inversion of control abstraction to get this done: the command package can select the exact provider versions and pass their factories directly to terraform.NewContext as a simple static map. The result of this commit is that CLI commands other than "init" are now able to consume the local cache directory and selections produced by the installation process in "terraform init", passing all of the selected providers down to the terraform.NewContext function for use in implementing the main operations. This commit is just enough to get the providers passing into the terraform.Context. There's still plenty more to do here, including to repair all of the tests this change has additionally broken.	6 years ago
James Bardin	b3fc0dab94	use addrs.ConfigResource for dependency tracking ... We can't get module instances during transformation, so we need to reduce the Dependencies to using `addrs.ConfigResource` for now.	6 years ago
James Bardin	e13eecbc5b	finish provider ModuleInstance replacement	6 years ago
Kristin Laemmert	47a16b0937	addrs: embed Provider in AbsProviderConfig instead of Type ... a large refactor to addrs.AbsProviderConfig, embedding the addrs.Provider instead of a Type string. I've added and updated tests, added some Legacy functions to support older state formats and shims, and added a normalization step when reading v4 (current) state files (not the added tests under states/statefile/roundtrip which work with both current and legacy-style AbsProviderConfig strings). The remaining 'fixme' and 'todo' comments are mostly going to be addressed in a subsequent PR and involve looking up a given local provider config's FQN. This is fine for now as we are only working with default assumption.	6 years ago
Martin Atkins	8b511524d6	Initial steps towards AbsProviderConfig/LocalProviderConfig separation (#23978) ... * Introduce "Local" terminology for non-absolute provider config addresses In a future change AbsProviderConfig and LocalProviderConfig are going to become two entirely distinct types, rather than Abs embedding Local as written here. This naming change is in preparation for that subsequent work, which will also include introducing a new "ProviderConfig" type that is an interface that AbsProviderConfig and LocalProviderConfig both implement. This is intended to be largely just a naming change to get started, so we can deal with all of the messy renaming. However, this did also require a slight change in modeling where the Resource.DefaultProviderConfig method has become Resource.DefaultProvider returning a Provider address directly, because this method doesn't have enough information to construct a true and accurate LocalProviderConfig -- it would need to refer to the configuration to know what this module is calling the provider it has selected. In order to leave a trail to follow for subsequent work, all of the changes here are intended to ensure that remaining work will become obvious via compile-time errors when all of the following changes happen: - The concept of "legacy" provider addresses is removed from the addrs package, including removing addrs.NewLegacyProvider and addrs.Provider.LegacyString. - addrs.AbsProviderConfig stops having addrs.LocalProviderConfig embedded in it and has an addrs.Provider and a string alias directly instead. - The provider-schema-handling parts of Terraform core are updated to work with addrs.Provider to identify providers, rather than legacy strings. In particular, there are still several codepaths here making legacy provider address assumptions (in order to limit the scope of this change) but I've made sure each one is doing something that relies on at least one of the above changes not having been made yet. * addrs: ProviderConfig interface In a (very) few special situations in the main "terraform" package we need to make runtime decisions about whether a provider config is absolute or local. We currently do that by exploiting the fact that AbsProviderConfig has LocalProviderConfig nested inside of it and so in the local case we can just ignore the wrapping AbsProviderConfig and use the embedded value. In a future change we'll be moving away from that embedding and making these two types distinct in order to represent that mapping between them requires consulting a lookup table in the configuration, and so here we introduce a new interface type ProviderConfig that can represent either AbsProviderConfig or LocalProviderConfig decided dynamically at runtime. This also includes the Config.ResolveAbsProviderAddr method that will eventually be responsible for that local-to-absolute translation, so that callers with access to the configuration can normalize to an addrs.AbsProviderConfig given a non-nil addrs.ProviderConfig. That's currently unused because existing callers are still relying on the simplistic structural transform, but we'll switch them over in a later commit. * rename LocalType to LocalName Co-authored-by: Kristin Laemmert <mildwonkey@users.noreply.github.com>	6 years ago
Kristin Laemmert	6541775ce4	addrs: roll back change to Type field in ProviderConfig (#23937)	6 years ago
James Bardin	98c02ac114	remove stale dependencies on `state mv` ... Clear any Dependencies if there is an entry matching a `state mv` from address. While stale dependencies won't directly effect any current operations, clearing the list will allow them to be recreated in their entirety during refresh. This will help future releases that may rely solely on the pre-calculated dependencies for destruction ordering.	6 years ago
James Bardin	42a2bb4da3	expand test tempdir symlinks to fix tests on macos	6 years ago
Kristin Laemmert	e3416124cc	addrs: replace "Type string" with "Type Provider" in ProviderConfig ... * huge change to weave new addrs.Provider into addrs.ProviderConfig * terraform: do not include an empty string in the returned Providers / Provisioners - Fixed a minor bug where results included an extra empty string	6 years ago
Kristin Laemmert	9891d0354a	providers: use addrs.Provider as map keys for provider.Factory (#23548) ... * terraform/context: use new addrs.Provider as map key in provider factories * added NewLegacyProviderType and LegacyString funcs to make it explicit that these are temporary placeholders This PR introduces a new concept, provider fully-qualified name (FQN), encapsulated by the `addrs.Provider` struct.	6 years ago
James Bardin	46dbb3dde5	use Dependencies to connect creator and destroyer ... The DestroyEdgeTransformer cannot determine ordering from the graph when the destroyers are from orphaned resources, because there are no references to resolve. The new stored Dependencies provides what we need to connect the instances in this case. We also add the StateDependencies method directly in the GraphNodeResourceInstance interface, since all instances already implement this, and we don't need another optional interface to check. The old code in DestroyEdgeTransformer may no longer be needed in the long run, but that can be determined separately, since too many of the tests start with an incomplete state and rely on the Dependencies being determined from the configuration alone.	6 years ago
Radek Simko	5b9f2fafc8	Standardise directory name for test data	7 years ago
James Bardin	ac2219ba6e	don't lose Private state data during copy ... Fix the scope of the private data copy in DeepCopy. Make sure Dependencies matches nil vs empty so that Equal compares correctly between copied states	7 years ago
Martin Atkins	86c02d5c35	command: "terraform init" can partially initialize for 0.12upgrade ... There are a few constructs from 0.11 and prior that cause 0.12 parsing to fail altogether, which previously created a chicken/egg problem because we need to install the providers in order to run "terraform 0.12upgrade" and thus fix the problem. This changes "terraform init" to use the new "early configuration" loader for module and provider installation. This is built on the more permissive parser in the terraform-config-inspect package, and so it allows us to read out the top-level blocks from the configuration while accepting legacy HCL syntax. In the long run this will let us do version compatibility detection before attempting a "real" config load, giving us better error messages for any future syntax additions, but in the short term the key thing is that it allows us to install the dependencies even if the configuration isn't fully valid. Because backend init still requires full configuration, this introduces a new mode of terraform init where it detects heuristically if it seems like we need to do a configuration upgrade and does a partial init if so, before finally directing the user to run "terraform 0.12upgrade" before running any other commands. The heuristic here is based on two assumptions: - If the "early" loader finds no errors but the normal loader does, the configuration is likely to be valid for Terraform 0.11 but not 0.12. - If there's already a version constraint in the configuration that excludes Terraform versions prior to v0.12 then the configuration is probably _already_ upgraded and so it's just a normal syntax error, even if the early loader didn't detect it. Once the upgrade process is removed in 0.13.0 (users will be required to go stepwise 0.11 -> 0.12 -> 0.13 to upgrade after that), some of this can be simplified to remove that special mode, but the idea of doing the dependency version checks against the liberal parser will remain valuable to increase our chances of reporting version-based incompatibilities rather than syntax errors as we add new features in future.	7 years ago
Martin Atkins	0c0a437bcb	Move module install functionality over to internal/initwd	7 years ago
James Bardin	149ccd929e	missing commits from 19688	7 years ago
Sander van Harmelen	79a9a15879	command/state: lock when pushing state ... Next to adding the locking for the `state push` command, this commit also fixes a small bug where the lock would not be propertly released when running the `state show` command. And finally it renames some variables in the `[un]taint` code in order to try to standardize the var names of a few frequently used variables (e.g. statemgr.Full, states.State, states.SyncState).	7 years ago
Martin Atkins	884aa387b8	command: Use vendoring when building helper programs in tests ... In a couple places in tests we execute a child "go build" to make a helper program. Now that we're running in module mode, "go build" will normally default to downloading and caching dependencies, which we don't want because we're still using vendoring for the moment. Therefore we need to instruct these child builds to use vendoring too, avoiding the need to download all of the dependencies and ensuring that we'll be building with the same dependencies that we'd use for a normal build.	7 years ago
Martin Atkins	ad276142b4	command: Fix the command package test build ... Some merging conflict shenanigans here led to this usage not lining up with the imported symbol name, meaning that the tests couldn't compile any more.	7 years ago
Martin Atkins	c3d11b762b	command: Fix testBackendState ... The hashing function for cached backend configuration is different now, so our hard-coded hash of the HTTP backend address wasn't working anymore. Here we update the hash so that tests using this test backend will work again. Rather than leaving it hard-coded, we'll instead compute it the same way as "terraform init" would. In practice only one test is actually using this function right now, so we also update the test fixture for that test (TestPlan_outBackend) to match the new expectations, though as of this commit it's still failing with an unrelated error.	7 years ago
Sander van Harmelen	52a1b22f7a	Implement the remote enhanced backend ... This is a refactored version of the `remote` backend that was initially added to Terraform v0.11.8 which should now be compatible with v0.12.0.	7 years ago
Martin Atkins	541952bb8f	Revert some work that happened since v0.12-dev branched ... This work was done against APIs that were already changed in the branch before work began, and so it doesn't apply to the v0.12 development work. To allow v0.12 to merge down to master, we'll revert this work out for now and then re-introduce equivalent functionality in later commits that works against the new APIs.	7 years ago
Martin Atkins	7b77e20bdc	command: Fix several "terraform init" tests	7 years ago
Martin Atkins	89d0944e5b	command: Fix even more of the "terraform refresh" tests	7 years ago
Martin Atkins	98bbd560b5	command: Fix most (but not all) "terraform plan" tests	7 years ago
Martin Atkins	741d334ee4	command: Even more fixes for "apply" command tests	7 years ago
Martin Atkins	fc2614c939	command: Fix some tests for the "apply" command	7 years ago
Martin Atkins	d9051be66d	command: Use new provider and provisioner types in test helpers ... This moves our build errors from the test helpers out into the tests themselves. We'll update the tests in subsequent commits.	7 years ago
Martin Atkins	a3403f2766	terraform: Ugly huge change to weave in new State and Plan types ... Due to how often the state and plan types are referenced throughout Terraform, there isn't a great way to switch them out gradually. As a consequence, this huge commit gets us from the old world to a _compilable_ new world, but still has a large number of known test failures due to key functionality being stubbed out. The stubs here are for anything that interacts with providers, since we now need to do the follow-up work to similarly replace the old terraform.ResourceProvider interface with its replacement in the new "providers" package. That work, along with work to fix the remaining failing tests, will follow in subsequent commits. The aim here was to replace all references to terraform.State and its downstream types with states.State, terraform.Plan with plans.Plan, state.State with statemgr.State, and switch to the new implementations of the state and plan file formats. However, due to the number of times those types are used, this also ended up affecting numerous other parts of core such as terraform.Hook, the backend.Backend interface, and most of the CLI commands. Just as with 5861dbf3fc before, I apologize in advance to the person who inevitably just found this huge commit while spelunking through the commit history.	7 years ago
Martin Atkins	a270a18a4d	command: Update tests for changes in "terraform" package ... This is not exhaustive, but it gets the tests in this package compiling again and gets most of them working.	7 years ago
Martin Atkins	ebafa51723	command: Various updates for the new backend package API ... This is a rather-messy, complex change to get the "command" package building again against the new backend API that was updated for the new configuration loader. A lot of this is mechanical rewriting to the new API, but meta_config.go and meta_backend.go in particular saw some major changes to interface with the new loader APIs and to deal with the change in order of steps in the backend API.	7 years ago
Sander van Harmelen	7fb2d1b8de	Implement the Enterprise enhanced remote backend	8 years ago
Sander van Harmelen	179b32d426	Add a `CredentialsForHost` method to disco.Disco ... By adding this method you now only have to pass a `*disco.Disco` object around in order to do discovery and use any configured credentials for the discovered hosts. Of course you can also still pass around both a `*disco.Disco` and a `auth.CredentialsSource` object if there is a need or a reason for that!	8 years ago
James Bardin	e980156451	cleanup temp files from command tests ... Rather than try to modify all the hundreds of calls to the temp helper functions, and cleanup the temp files at every call site, have all tests work within a single temp directory that is removed at the end of TestMain.	8 years ago
James Bardin	5203c66116	pass command credentials into module.Storage	8 years ago
James Bardin	3a495ffe56	rename ModuleStorage to Storage ... get rid of stutter and use module.Storage	8 years ago
James Bardin	f2a7b94692	use the new ModuleStorage in the command package ... Update the command package to use the new module storage. Move the old command output strings into the module storage itself. This could be moved back later either by using ui callbacks, or designing a module storage interface once we know what the final requirements will look like.	8 years ago
Martin Atkins	c12d64f340	Use t.Helper() in our test helpers ... Go 1.9 adds this new function which, when called, marks the caller as being a "helper function". Helper function stack frames are then skipped when trying to find a line of test code to blame for a test failure, so that the code in the main test function appears in the test failure output rather than a line within the helper function itself. This covers many -- but probaly not all -- of our test helpers across various packages.	9 years ago
James Bardin	054716c397	use testStateRead helper in apply tests ... We have a helper function that we hardly ever use. TODO: convert the rest of the manual ReadState calls eventually.	9 years ago
James Bardin	501cbeaffe	testState shouldn't rely on mods from WriteState ... The state returned from the testState helper shouldn't rely on any mutations caused by WriteState. The Init function (which is analogous to NewState) shoudl set any required fields.	9 years ago
Martin Atkins	c835ef8ff3	Update tests for the new ProviderResolver interface ... Rather than providing an already-resolved map of plugins to core, we now provide a "provider resolver" which knows how to resolve a set of provider dependencies, to be determined later, and produce that map. This requires the context to be instantiated in a different way, so this very noisy diff is a mostly-mechanical update of all of the existing places where contexts get created for testing, using some adapted versions of the pre-existing utilities for passing in mock providers.	9 years ago
Martin Atkins	8364383c35	Push plugin discovery down into command package ... Previously we did plugin discovery in the main package, but as we move towards versioned plugins we need more information available in order to resolve plugins, so we move this responsibility into the command package itself. For the moment this is just preserving the existing behavior as long as there are only internal and unversioned plugins present. This is the final state for provisioners in 0.10, since we don't want to support versioned provisioners yet. For providers this is just a checkpoint along the way, since further work is required to apply version constraints from configuration and support additional plugin search directories. The automatic plugin discovery behavior is not desirable for tests because we want to mock the plugins there, so we add a new backdoor for the tests to use to skip the plugin discovery and just provide their own mock implementations. Most of this diff is thus noisy rework of the tests to use this new mechanism.	9 years ago
Mitchell Hashimoto	1d8b76c89d	command: initial work on migrating envs, basic cases first	9 years ago
James Bardin	f2e496a14c	Have backend operations properly unlock state ... Make sure unlock is called with the correct LockID during operations	9 years ago
James Bardin	b80ae5e13e	Add source path argument to testLockState ... The new test pattern is to chdir into a temp location for the test, but the prevents us from locating the testdata directory in the source. Add a source path to testLockState so we can find the statelocker.go source.	9 years ago