74 Commits (449ea72e87da7e697ef7b68283a852cc9cc09045)

Author	SHA1	Message	Date
Kristin Laemmert	47a16b0937	addrs: embed Provider in AbsProviderConfig instead of Type ... a large refactor to addrs.AbsProviderConfig, embedding the addrs.Provider instead of a Type string. I've added and updated tests, added some Legacy functions to support older state formats and shims, and added a normalization step when reading v4 (current) state files (not the added tests under states/statefile/roundtrip which work with both current and legacy-style AbsProviderConfig strings). The remaining 'fixme' and 'todo' comments are mostly going to be addressed in a subsequent PR and involve looking up a given local provider config's FQN. This is fine for now as we are only working with default assumption.	6 years ago
Martin Atkins	8b511524d6	Initial steps towards AbsProviderConfig/LocalProviderConfig separation (#23978) ... * Introduce "Local" terminology for non-absolute provider config addresses In a future change AbsProviderConfig and LocalProviderConfig are going to become two entirely distinct types, rather than Abs embedding Local as written here. This naming change is in preparation for that subsequent work, which will also include introducing a new "ProviderConfig" type that is an interface that AbsProviderConfig and LocalProviderConfig both implement. This is intended to be largely just a naming change to get started, so we can deal with all of the messy renaming. However, this did also require a slight change in modeling where the Resource.DefaultProviderConfig method has become Resource.DefaultProvider returning a Provider address directly, because this method doesn't have enough information to construct a true and accurate LocalProviderConfig -- it would need to refer to the configuration to know what this module is calling the provider it has selected. In order to leave a trail to follow for subsequent work, all of the changes here are intended to ensure that remaining work will become obvious via compile-time errors when all of the following changes happen: - The concept of "legacy" provider addresses is removed from the addrs package, including removing addrs.NewLegacyProvider and addrs.Provider.LegacyString. - addrs.AbsProviderConfig stops having addrs.LocalProviderConfig embedded in it and has an addrs.Provider and a string alias directly instead. - The provider-schema-handling parts of Terraform core are updated to work with addrs.Provider to identify providers, rather than legacy strings. In particular, there are still several codepaths here making legacy provider address assumptions (in order to limit the scope of this change) but I've made sure each one is doing something that relies on at least one of the above changes not having been made yet. * addrs: ProviderConfig interface In a (very) few special situations in the main "terraform" package we need to make runtime decisions about whether a provider config is absolute or local. We currently do that by exploiting the fact that AbsProviderConfig has LocalProviderConfig nested inside of it and so in the local case we can just ignore the wrapping AbsProviderConfig and use the embedded value. In a future change we'll be moving away from that embedding and making these two types distinct in order to represent that mapping between them requires consulting a lookup table in the configuration, and so here we introduce a new interface type ProviderConfig that can represent either AbsProviderConfig or LocalProviderConfig decided dynamically at runtime. This also includes the Config.ResolveAbsProviderAddr method that will eventually be responsible for that local-to-absolute translation, so that callers with access to the configuration can normalize to an addrs.AbsProviderConfig given a non-nil addrs.ProviderConfig. That's currently unused because existing callers are still relying on the simplistic structural transform, but we'll switch them over in a later commit. * rename LocalType to LocalName Co-authored-by: Kristin Laemmert <mildwonkey@users.noreply.github.com>	6 years ago
Kristin Laemmert	6541775ce4	addrs: roll back change to Type field in ProviderConfig (#23937)	6 years ago
James Bardin	98c02ac114	remove stale dependencies on `state mv` ... Clear any Dependencies if there is an entry matching a `state mv` from address. While stale dependencies won't directly effect any current operations, clearing the list will allow them to be recreated in their entirety during refresh. This will help future releases that may rely solely on the pre-calculated dependencies for destruction ordering.	6 years ago
James Bardin	42a2bb4da3	expand test tempdir symlinks to fix tests on macos	6 years ago
Kristin Laemmert	e3416124cc	addrs: replace "Type string" with "Type Provider" in ProviderConfig ... * huge change to weave new addrs.Provider into addrs.ProviderConfig * terraform: do not include an empty string in the returned Providers / Provisioners - Fixed a minor bug where results included an extra empty string	6 years ago
Kristin Laemmert	9891d0354a	providers: use addrs.Provider as map keys for provider.Factory (#23548) ... * terraform/context: use new addrs.Provider as map key in provider factories * added NewLegacyProviderType and LegacyString funcs to make it explicit that these are temporary placeholders This PR introduces a new concept, provider fully-qualified name (FQN), encapsulated by the `addrs.Provider` struct.	6 years ago
James Bardin	46dbb3dde5	use Dependencies to connect creator and destroyer ... The DestroyEdgeTransformer cannot determine ordering from the graph when the destroyers are from orphaned resources, because there are no references to resolve. The new stored Dependencies provides what we need to connect the instances in this case. We also add the StateDependencies method directly in the GraphNodeResourceInstance interface, since all instances already implement this, and we don't need another optional interface to check. The old code in DestroyEdgeTransformer may no longer be needed in the long run, but that can be determined separately, since too many of the tests start with an incomplete state and rely on the Dependencies being determined from the configuration alone.	6 years ago
Radek Simko	5b9f2fafc8	Standardise directory name for test data	7 years ago
James Bardin	ac2219ba6e	don't lose Private state data during copy ... Fix the scope of the private data copy in DeepCopy. Make sure Dependencies matches nil vs empty so that Equal compares correctly between copied states	7 years ago
Martin Atkins	86c02d5c35	command: "terraform init" can partially initialize for 0.12upgrade ... There are a few constructs from 0.11 and prior that cause 0.12 parsing to fail altogether, which previously created a chicken/egg problem because we need to install the providers in order to run "terraform 0.12upgrade" and thus fix the problem. This changes "terraform init" to use the new "early configuration" loader for module and provider installation. This is built on the more permissive parser in the terraform-config-inspect package, and so it allows us to read out the top-level blocks from the configuration while accepting legacy HCL syntax. In the long run this will let us do version compatibility detection before attempting a "real" config load, giving us better error messages for any future syntax additions, but in the short term the key thing is that it allows us to install the dependencies even if the configuration isn't fully valid. Because backend init still requires full configuration, this introduces a new mode of terraform init where it detects heuristically if it seems like we need to do a configuration upgrade and does a partial init if so, before finally directing the user to run "terraform 0.12upgrade" before running any other commands. The heuristic here is based on two assumptions: - If the "early" loader finds no errors but the normal loader does, the configuration is likely to be valid for Terraform 0.11 but not 0.12. - If there's already a version constraint in the configuration that excludes Terraform versions prior to v0.12 then the configuration is probably _already_ upgraded and so it's just a normal syntax error, even if the early loader didn't detect it. Once the upgrade process is removed in 0.13.0 (users will be required to go stepwise 0.11 -> 0.12 -> 0.13 to upgrade after that), some of this can be simplified to remove that special mode, but the idea of doing the dependency version checks against the liberal parser will remain valuable to increase our chances of reporting version-based incompatibilities rather than syntax errors as we add new features in future.	7 years ago
Martin Atkins	0c0a437bcb	Move module install functionality over to internal/initwd	7 years ago
James Bardin	149ccd929e	missing commits from 19688	7 years ago
Sander van Harmelen	79a9a15879	command/state: lock when pushing state ... Next to adding the locking for the `state push` command, this commit also fixes a small bug where the lock would not be propertly released when running the `state show` command. And finally it renames some variables in the `[un]taint` code in order to try to standardize the var names of a few frequently used variables (e.g. statemgr.Full, states.State, states.SyncState).	7 years ago
Martin Atkins	884aa387b8	command: Use vendoring when building helper programs in tests ... In a couple places in tests we execute a child "go build" to make a helper program. Now that we're running in module mode, "go build" will normally default to downloading and caching dependencies, which we don't want because we're still using vendoring for the moment. Therefore we need to instruct these child builds to use vendoring too, avoiding the need to download all of the dependencies and ensuring that we'll be building with the same dependencies that we'd use for a normal build.	7 years ago
Martin Atkins	ad276142b4	command: Fix the command package test build ... Some merging conflict shenanigans here led to this usage not lining up with the imported symbol name, meaning that the tests couldn't compile any more.	7 years ago
Martin Atkins	c3d11b762b	command: Fix testBackendState ... The hashing function for cached backend configuration is different now, so our hard-coded hash of the HTTP backend address wasn't working anymore. Here we update the hash so that tests using this test backend will work again. Rather than leaving it hard-coded, we'll instead compute it the same way as "terraform init" would. In practice only one test is actually using this function right now, so we also update the test fixture for that test (TestPlan_outBackend) to match the new expectations, though as of this commit it's still failing with an unrelated error.	7 years ago
Sander van Harmelen	52a1b22f7a	Implement the remote enhanced backend ... This is a refactored version of the `remote` backend that was initially added to Terraform v0.11.8 which should now be compatible with v0.12.0.	7 years ago
Martin Atkins	541952bb8f	Revert some work that happened since v0.12-dev branched ... This work was done against APIs that were already changed in the branch before work began, and so it doesn't apply to the v0.12 development work. To allow v0.12 to merge down to master, we'll revert this work out for now and then re-introduce equivalent functionality in later commits that works against the new APIs.	7 years ago
Martin Atkins	7b77e20bdc	command: Fix several "terraform init" tests	7 years ago
Martin Atkins	89d0944e5b	command: Fix even more of the "terraform refresh" tests	7 years ago
Martin Atkins	98bbd560b5	command: Fix most (but not all) "terraform plan" tests	7 years ago
Martin Atkins	741d334ee4	command: Even more fixes for "apply" command tests	7 years ago
Martin Atkins	fc2614c939	command: Fix some tests for the "apply" command	7 years ago
Martin Atkins	d9051be66d	command: Use new provider and provisioner types in test helpers ... This moves our build errors from the test helpers out into the tests themselves. We'll update the tests in subsequent commits.	7 years ago
Martin Atkins	a3403f2766	terraform: Ugly huge change to weave in new State and Plan types ... Due to how often the state and plan types are referenced throughout Terraform, there isn't a great way to switch them out gradually. As a consequence, this huge commit gets us from the old world to a _compilable_ new world, but still has a large number of known test failures due to key functionality being stubbed out. The stubs here are for anything that interacts with providers, since we now need to do the follow-up work to similarly replace the old terraform.ResourceProvider interface with its replacement in the new "providers" package. That work, along with work to fix the remaining failing tests, will follow in subsequent commits. The aim here was to replace all references to terraform.State and its downstream types with states.State, terraform.Plan with plans.Plan, state.State with statemgr.State, and switch to the new implementations of the state and plan file formats. However, due to the number of times those types are used, this also ended up affecting numerous other parts of core such as terraform.Hook, the backend.Backend interface, and most of the CLI commands. Just as with 5861dbf3fc before, I apologize in advance to the person who inevitably just found this huge commit while spelunking through the commit history.	7 years ago
Martin Atkins	a270a18a4d	command: Update tests for changes in "terraform" package ... This is not exhaustive, but it gets the tests in this package compiling again and gets most of them working.	7 years ago
Martin Atkins	ebafa51723	command: Various updates for the new backend package API ... This is a rather-messy, complex change to get the "command" package building again against the new backend API that was updated for the new configuration loader. A lot of this is mechanical rewriting to the new API, but meta_config.go and meta_backend.go in particular saw some major changes to interface with the new loader APIs and to deal with the change in order of steps in the backend API.	7 years ago
Sander van Harmelen	7fb2d1b8de	Implement the Enterprise enhanced remote backend	8 years ago
Sander van Harmelen	179b32d426	Add a `CredentialsForHost` method to disco.Disco ... By adding this method you now only have to pass a `*disco.Disco` object around in order to do discovery and use any configured credentials for the discovered hosts. Of course you can also still pass around both a `*disco.Disco` and a `auth.CredentialsSource` object if there is a need or a reason for that!	8 years ago
James Bardin	e980156451	cleanup temp files from command tests ... Rather than try to modify all the hundreds of calls to the temp helper functions, and cleanup the temp files at every call site, have all tests work within a single temp directory that is removed at the end of TestMain.	8 years ago
James Bardin	5203c66116	pass command credentials into module.Storage	8 years ago
James Bardin	3a495ffe56	rename ModuleStorage to Storage ... get rid of stutter and use module.Storage	8 years ago
James Bardin	f2a7b94692	use the new ModuleStorage in the command package ... Update the command package to use the new module storage. Move the old command output strings into the module storage itself. This could be moved back later either by using ui callbacks, or designing a module storage interface once we know what the final requirements will look like.	8 years ago
Martin Atkins	c12d64f340	Use t.Helper() in our test helpers ... Go 1.9 adds this new function which, when called, marks the caller as being a "helper function". Helper function stack frames are then skipped when trying to find a line of test code to blame for a test failure, so that the code in the main test function appears in the test failure output rather than a line within the helper function itself. This covers many -- but probaly not all -- of our test helpers across various packages.	9 years ago
James Bardin	054716c397	use testStateRead helper in apply tests ... We have a helper function that we hardly ever use. TODO: convert the rest of the manual ReadState calls eventually.	9 years ago
James Bardin	501cbeaffe	testState shouldn't rely on mods from WriteState ... The state returned from the testState helper shouldn't rely on any mutations caused by WriteState. The Init function (which is analogous to NewState) shoudl set any required fields.	9 years ago
Martin Atkins	c835ef8ff3	Update tests for the new ProviderResolver interface ... Rather than providing an already-resolved map of plugins to core, we now provide a "provider resolver" which knows how to resolve a set of provider dependencies, to be determined later, and produce that map. This requires the context to be instantiated in a different way, so this very noisy diff is a mostly-mechanical update of all of the existing places where contexts get created for testing, using some adapted versions of the pre-existing utilities for passing in mock providers.	9 years ago
Martin Atkins	8364383c35	Push plugin discovery down into command package ... Previously we did plugin discovery in the main package, but as we move towards versioned plugins we need more information available in order to resolve plugins, so we move this responsibility into the command package itself. For the moment this is just preserving the existing behavior as long as there are only internal and unversioned plugins present. This is the final state for provisioners in 0.10, since we don't want to support versioned provisioners yet. For providers this is just a checkpoint along the way, since further work is required to apply version constraints from configuration and support additional plugin search directories. The automatic plugin discovery behavior is not desirable for tests because we want to mock the plugins there, so we add a new backdoor for the tests to use to skip the plugin discovery and just provide their own mock implementations. Most of this diff is thus noisy rework of the tests to use this new mechanism.	9 years ago
Mitchell Hashimoto	1d8b76c89d	command: initial work on migrating envs, basic cases first	9 years ago
James Bardin	f2e496a14c	Have backend operations properly unlock state ... Make sure unlock is called with the correct LockID during operations	9 years ago
James Bardin	b80ae5e13e	Add source path argument to testLockState ... The new test pattern is to chdir into a temp location for the test, but the prevents us from locating the testdata directory in the source. Add a source path to testLockState so we can find the statelocker.go source.	9 years ago
James Bardin	f3e4c05250	build the statelocker binary before running ... this way we can signal it directly to amke sure it exits cleanly.	9 years ago
James Bardin	bd65ddbcaa	Add test for apply/refresh on locked state files ... Verify that these operations fail when a state file is locked.	9 years ago
Mitchell Hashimoto	ad7b063262	command: convert to use backends	9 years ago
Mitchell Hashimoto	2f8bf5b7ec	terraform: add variables to Interpolator value ... Fixes #10412 The context wasn't properly adding variable values to the Interpolator instance which made it so that the `console` command couldn't access variables set via tfvars and the CLI. This also adds better test coverage in command itself for this.	9 years ago
Mitchell Hashimoto	2019a44f04	command/apply: apply from plan respects -backup and -state-out ... Fixes #5409 I didn't expect this to be such a rabbit hole! Based on git history, it appears that for "historical reasons"(tm), setting up the various `state.State` structures for a plan were _completely different logic_ than a normal `terraform apply`. This meant that it was skipping things like disabling backups with `-backup="-"`. This PR unifies loading from a plan to the normal state setup mechanism. A few tests that were failing prior to this PR were added, no existing tests were changed.	9 years ago
Mitchell Hashimoto	e8267f4907	command/push: failing test for pushing with no modules	10 years ago
James Bardin	cdb80f68a8	Ensure better state normalization ... Fix checksum issue with remote state If we read a state file with "null" objects in a module and they become initialized to an empty map the state file may be written out with empty objects rather than "null", changing the checksum. If we can detect this, increment the serial number to prevent a conflict in atlas. Our fakeAtlas test server now needs to decode the state directly rather than using the ReadState function, so as to be able to read the state unaltered. The terraform.State data structures have initialization spread out throughout the package. More thoroughly initialize State during ReadState, and add a call to init() during WriteState as another normalization safeguard. Expose State.init through an exported Init() method, so that a new State can be completely realized outside of the terraform package. Additionally, the internal init now completely walks all internal state structures ensuring that all maps and slices are initialized. While it was mentioned before that the `init()` methods are problematic with too many call sites, expanding this out better exposes the entry points that will need to be refactored later for improved concurrency handling. The State structures had a mix of `omitempty` fields. Remove omitempty for all maps and slices as part of this normalization process. Make Lineage mandatory, which is now explicitly set in some tests.	10 years ago
James Bardin	1af7ee87a2	Silence log output when not verbose ... Set the default log package output to iotuil.Discard during tests if the `-v` flag isn't set. If we are verbose, then apply the filter according to the TF_LOG env variable.	10 years ago