115 Commits (6440c839474f052d8997bb5cec19c2743e5108e6)

Author	SHA1	Message	Date
Kristin Laemmert	c4873778c8	Mildwonkey/tests (#24548) ... * helper/resource: remove provider resolver test * repl tests passing * helper/resource: add some extra shimming to ShimLegacyState Some of the tests in helper/resource have to shim between legacy and modern states. Terraform expects modern states to have the Provider set for each resource (and not be a legacy provider). This PR adds a wrapper around terraform.ShimLegacyState which iterates through the resources in the state (after the shim) and adds the provider FQN.	6 years ago
Martin Atkins	549aede792	Remove terraform.ResourceProvider, use providercache.Installer instead ... Back when we first introduced provider versioning in Terraform 0.10, we did the provider version resolution in terraform.NewContext because we weren't sure yet how exactly our versioning model was going to play out (whether different versions could be selected per provider configuration, for example) and because we were building around the limitations of our existing filesystem-based plugin discovery model. However, the new installer codepath is new able to do all of the selections up front during installation, so we don't need such a heavy inversion of control abstraction to get this done: the command package can select the exact provider versions and pass their factories directly to terraform.NewContext as a simple static map. The result of this commit is that CLI commands other than "init" are now able to consume the local cache directory and selections produced by the installation process in "terraform init", passing all of the selected providers down to the terraform.NewContext function for use in implementing the main operations. This commit is just enough to get the providers passing into the terraform.Context. There's still plenty more to do here, including to repair all of the tests this change has additionally broken.	6 years ago
James Bardin	e13eecbc5b	finish provider ModuleInstance replacement	6 years ago
Kristin Laemmert	47a16b0937	addrs: embed Provider in AbsProviderConfig instead of Type ... a large refactor to addrs.AbsProviderConfig, embedding the addrs.Provider instead of a Type string. I've added and updated tests, added some Legacy functions to support older state formats and shims, and added a normalization step when reading v4 (current) state files (not the added tests under states/statefile/roundtrip which work with both current and legacy-style AbsProviderConfig strings). The remaining 'fixme' and 'todo' comments are mostly going to be addressed in a subsequent PR and involve looking up a given local provider config's FQN. This is fine for now as we are only working with default assumption.	6 years ago
Martin Atkins	8b511524d6	Initial steps towards AbsProviderConfig/LocalProviderConfig separation (#23978) ... * Introduce "Local" terminology for non-absolute provider config addresses In a future change AbsProviderConfig and LocalProviderConfig are going to become two entirely distinct types, rather than Abs embedding Local as written here. This naming change is in preparation for that subsequent work, which will also include introducing a new "ProviderConfig" type that is an interface that AbsProviderConfig and LocalProviderConfig both implement. This is intended to be largely just a naming change to get started, so we can deal with all of the messy renaming. However, this did also require a slight change in modeling where the Resource.DefaultProviderConfig method has become Resource.DefaultProvider returning a Provider address directly, because this method doesn't have enough information to construct a true and accurate LocalProviderConfig -- it would need to refer to the configuration to know what this module is calling the provider it has selected. In order to leave a trail to follow for subsequent work, all of the changes here are intended to ensure that remaining work will become obvious via compile-time errors when all of the following changes happen: - The concept of "legacy" provider addresses is removed from the addrs package, including removing addrs.NewLegacyProvider and addrs.Provider.LegacyString. - addrs.AbsProviderConfig stops having addrs.LocalProviderConfig embedded in it and has an addrs.Provider and a string alias directly instead. - The provider-schema-handling parts of Terraform core are updated to work with addrs.Provider to identify providers, rather than legacy strings. In particular, there are still several codepaths here making legacy provider address assumptions (in order to limit the scope of this change) but I've made sure each one is doing something that relies on at least one of the above changes not having been made yet. * addrs: ProviderConfig interface In a (very) few special situations in the main "terraform" package we need to make runtime decisions about whether a provider config is absolute or local. We currently do that by exploiting the fact that AbsProviderConfig has LocalProviderConfig nested inside of it and so in the local case we can just ignore the wrapping AbsProviderConfig and use the embedded value. In a future change we'll be moving away from that embedding and making these two types distinct in order to represent that mapping between them requires consulting a lookup table in the configuration, and so here we introduce a new interface type ProviderConfig that can represent either AbsProviderConfig or LocalProviderConfig decided dynamically at runtime. This also includes the Config.ResolveAbsProviderAddr method that will eventually be responsible for that local-to-absolute translation, so that callers with access to the configuration can normalize to an addrs.AbsProviderConfig given a non-nil addrs.ProviderConfig. That's currently unused because existing callers are still relying on the simplistic structural transform, but we'll switch them over in a later commit. * rename LocalType to LocalName Co-authored-by: Kristin Laemmert <mildwonkey@users.noreply.github.com>	6 years ago
Kristin Laemmert	6541775ce4	addrs: roll back change to Type field in ProviderConfig (#23937)	6 years ago
Kristin Laemmert	e3416124cc	addrs: replace "Type string" with "Type Provider" in ProviderConfig ... * huge change to weave new addrs.Provider into addrs.ProviderConfig * terraform: do not include an empty string in the returned Providers / Provisioners - Fixed a minor bug where results included an extra empty string	6 years ago
Martin Atkins	30bf83cdeb	helper/logging: Bring the LevelFilter into our own codebase ... In order to make this work reasonably we can't avoid using some funny heuristics, which are somewhat reasonable to apply within the context of Terraform itself but would not be good to add to the general "logutils". Specifically, this is adding the additional heuristic that lines starting with spaces are continuation lines and so should inherit the log level of the most recent non-continuation line.	6 years ago
Kristin Laemmert	9891d0354a	providers: use addrs.Provider as map keys for provider.Factory (#23548) ... * terraform/context: use new addrs.Provider as map key in provider factories * added NewLegacyProviderType and LegacyString funcs to make it explicit that these are temporary placeholders This PR introduces a new concept, provider fully-qualified name (FQN), encapsulated by the `addrs.Provider` struct.	6 years ago
Radek Simko	a08ad08169	Merge pull request #18277 from ndmckinley/patch-1 ... helper/resource: Minor docs tweak on ImportStateVerify	7 years ago
James Bardin	c02f1d7256	allow 0 and unset to be equal in count tests ... This was changed in the single attribute test cases, but the AttrPair test is used a lot for data source. As far as tests are concerned, 0 and unset should be treated equally for flatmapped collections.	7 years ago
Martin Atkins	4c99864dad	helper/resource: TestCheckResourceAttrPair allow nonexist ... This checking helper is frequently used in provider tests for data sources, as a shorthand to verify that an attribute of the data source matches with the corresponding attribute on a managed resource. Since we now leave empty collections null in more cases, this function is sometimes effectively asked to verify that a given attribute is _unset_ in both the data source and the resource, so here we slightly adjust the definition of the check to consider two nulls to be equal to one another, which at this layer manifests as the keys not being present in the state attributes map at all. This check function didn't previously have tests, so this commit also adds a basic suite of tests, including coverage for the new behavior.	7 years ago
Martin Atkins	86c02d5c35	command: "terraform init" can partially initialize for 0.12upgrade ... There are a few constructs from 0.11 and prior that cause 0.12 parsing to fail altogether, which previously created a chicken/egg problem because we need to install the providers in order to run "terraform 0.12upgrade" and thus fix the problem. This changes "terraform init" to use the new "early configuration" loader for module and provider installation. This is built on the more permissive parser in the terraform-config-inspect package, and so it allows us to read out the top-level blocks from the configuration while accepting legacy HCL syntax. In the long run this will let us do version compatibility detection before attempting a "real" config load, giving us better error messages for any future syntax additions, but in the short term the key thing is that it allows us to install the dependencies even if the configuration isn't fully valid. Because backend init still requires full configuration, this introduces a new mode of terraform init where it detects heuristically if it seems like we need to do a configuration upgrade and does a partial init if so, before finally directing the user to run "terraform 0.12upgrade" before running any other commands. The heuristic here is based on two assumptions: - If the "early" loader finds no errors but the normal loader does, the configuration is likely to be valid for Terraform 0.11 but not 0.12. - If there's already a version constraint in the configuration that excludes Terraform versions prior to v0.12 then the configuration is probably _already_ upgraded and so it's just a normal syntax error, even if the early loader didn't detect it. Once the upgrade process is removed in 0.13.0 (users will be required to go stepwise 0.11 -> 0.12 -> 0.13 to upgrade after that), some of this can be simplified to remove that special mode, but the idea of doing the dependency version checks against the liberal parser will remain valuable to increase our chances of reporting version-based incompatibilities rather than syntax errors as we add new features in future.	7 years ago
Martin Atkins	0c0a437bcb	Move module install functionality over to internal/initwd	7 years ago
James Bardin	a7b399cb4c	use actual schema.Resources for state shims ... Provider tests often rely on checking values contained within sets, by directly accessing their flatmapped representation. In order to provider the test harness with the expected set hashes, the sets must be generated by the schema.Resource itself. During the test we now build a fixed map of the providers, which should only contain schema.Provider instances, and pass them into each TestStep. The individual schema.Resource instances can then be pulled from the providers, and used to recreate the state from the cty.Value returned by the core operations.	7 years ago
James Bardin	7973872524	allow TestCheckNoResourceAttr for empty containers ... Stricter type handling in the new shims may add empty containers into the state where they were previously elided. Since the detection of missing and empty containers in the legacy state was never reliable, allow TestCheckNoResourceAttr to succeed if the key is a container count index, and the value is "0"	7 years ago
James Bardin	c63040c737	have TestCheckResourceAttr accept missing counts ... Missing containers were often erroneously kept in the state, but since the addition of the new provider shims, they can often be correctly eliminated. There are however many tests that check for a "0" count in the flatmap state when there shouldn't be a key at all. This addition looks for a container count key and "0" pair, and allows for the key to be missing. There may be some tests negatively effected by this which were legitimately checking for empty containers, but those were also not reliably detected, and there should be much fewer tests involved.	7 years ago
Martin Atkins	cdad78d69b	helper/resource: Allow multiple providers in a single TestCase ... Due to incorrect use of a loop iterator variable inside a closure, all of the given providers were ending up with the same factory function. Now we copy the factory function to a local within the loop first so that each iteration has its own variable. This is the second round of similar bugs in this function, so we'll also add a test case for it to reduce the risk of future regressions given that most real callers don't exercise this with multiple providers in practice.	7 years ago
Martin Atkins	f9fef56167	helper/resource: print full diagnostics for operation errors in tests ... This causes the output to include additional helpful context such as the values of variables referenced in the config, etc. The output is in the same format as normal Terraform CLI error output, though we don't retain a source code cache in this codepath so it will not include a source code snippet.	7 years ago
Martin Atkins	55469cd416	helper/resource: Get schemas from Terraform context ... Previously the test harness was preloading schemas from the providers before running any test steps. Since terraform.NewContext already deals with loading provider schemas, we can instead just use the schemas it loaded for our shimming needs, avoiding the need to reimplement the schema lookup behavior and thus the need to create a throwaway provider instance with which to do it.	7 years ago
Martin Atkins	a4991c5780	helper/resource: Create a separate provider instance each call ... Previously we were running the factory function only once when constructing the provider resolver, which means that all contexts created from that resolver share the same provider instance. Instead now we will call the given factory function once for each instantiation, ensuring that each caller ends up with a separate object as would be the case in real-world use.	7 years ago
James Bardin	0d4d572c39	start work on helper/resource test fixtures ... The helper resource tests won't pass for now, as they use a terraform.MockProvider which can't be used in the schema.Provider shims.	8 years ago
Martin Atkins	44bc7519a6	terraform: More wiring in of new provider types ... This doesn't actually work yet, but it builds and then panics in a pretty satisfying way.	8 years ago
Martin Atkins	a3403f2766	terraform: Ugly huge change to weave in new State and Plan types ... Due to how often the state and plan types are referenced throughout Terraform, there isn't a great way to switch them out gradually. As a consequence, this huge commit gets us from the old world to a _compilable_ new world, but still has a large number of known test failures due to key functionality being stubbed out. The stubs here are for anything that interacts with providers, since we now need to do the follow-up work to similarly replace the old terraform.ResourceProvider interface with its replacement in the new "providers" package. That work, along with work to fix the remaining failing tests, will follow in subsequent commits. The aim here was to replace all references to terraform.State and its downstream types with states.State, terraform.Plan with plans.Plan, state.State with statemgr.State, and switch to the new implementations of the state and plan file formats. However, due to the number of times those types are used, this also ended up affecting numerous other parts of core such as terraform.Hook, the backend.Backend interface, and most of the CLI commands. Just as with 5861dbf3fc before, I apologize in advance to the person who inevitably just found this huge commit while spelunking through the commit history.	8 years ago
Martin Atkins	c937c06a03	terraform: ugly huge change to weave in new HCL2-oriented types ... Due to how deeply the configuration types go into Terraform Core, there isn't a great way to switch out to HCL2 gradually. As a consequence, this huge commit gets us from the old state to a _compilable_ new state, but does not yet attempt to fix any tests and has a number of known missing parts and bugs. We will continue to iterate on this in forthcoming commits, heading back towards passing tests and making Terraform fully-functional again. The three main goals here are: - Use the configuration models from the "configs" package instead of the older models in the "config" package, which is now deprecated and preserved only to help us write our migration tool. - Do expression inspection and evaluation using the functionality of the new "lang" package, instead of the Interpolator type and related functionality in the main "terraform" package. - Represent addresses of various objects using types in the addrs package, rather than hand-constructed strings. This is not critical to support the above, but was a big help during the implementation of these other points since it made it much more explicit what kind of address is expected in each context. Since our new packages are built to accommodate some future planned features that are not yet implemented (e.g. the "for_each" argument on resources, "count"/"for_each" on modules), and since there's still a fair amount of functionality still using old-style APIs, there is a moderate amount of shimming here to connect new assumptions with old, hopefully in a way that makes it easier to find and eliminate these shims later. I apologize in advance to the person who inevitably just found this huge commit while spelunking through the commit history.	8 years ago
Brian Flad	798162125b	helper/resource: Add ParallelTest() function to allow opt-in acceptance testing concurrency with t.Parallel() ... While this initial implementation is a very simple wrapper function, implementing this in the helper/resource package provides some downstream benefits: * Provides a standard interface for plugin developers to enable parallel acceptance testing * Existing plugins can simply convert resource.Test to resource.ParallelTest references (as appropriate) to enable the functionality, rather than worrying about additional line(s) to each acceptance test function or TestCase * Potential enhancements to ParallelTest (e.g. adding an environment variable to skip enabling the behavior) are consistently propagated	8 years ago
Nathan McKinley	6c40b9520e	Minor docs tweak on ImportStateVerify ... ImportStateVerify does not respect DiffSuppressFunc or CustomizeDiff, which is worth documenting (and maybe, possibly, worth changing?). ImportStateVerifyIgnore is a list of prefixes, rather than a list of fields.	8 years ago
Chris Marchesi	3505769600	helper/resource: Add ability to pre-taint resources ... This adds the Taint field to the acceptance testing framework, allowing the ability to pre-taint resources at the beginning of a particular TestStep. This can be useful for when an explicit ForceNew is required for a specific resource for troubleshooting things like diff mismatches, etc. The field accepts resource addresses as a list of strings. To keep things simple for the time being, only addresses in the root module are accepted. If we ever want to expand this past that, I'd be almost inclined to add some facilities to the core terraform package to help translate actual module resource addresses (ie: module.foo.module.bar.some_resource.baz) into the correct state, versus the current convention in some acceptance testing facilities that take the module address as a list of strings (ie: []string{"root", "foo", "bar"}).	8 years ago
Stuart Auld	7d1f594f54	helper/resource: don't panic if a requested module does not exist	8 years ago
Stuart Auld	df9446f490	helper/resource: allow tests to opt out of temporary file cleanup ... This is rarely needed, but sometimes tests need to create temporary files as part of their operation. This should be used sparingly, since it prevents the pro-active cleanup of the temporary working directory.	8 years ago
Stuart Auld	1d1984771b	helper/resource: test check helpers for resources in non-root modules	8 years ago
Chris Marchesi	3f8dad30c9	helper/resource: Fail tests with no error that have ExpectError set ... Looks like while we were checking errors correctly when ExpectError was set, we weren't checking for the *absence* of an error, which is should be checked as well (no error is still not the error we are looking for). Added a few more tests for ExpectError as well.	8 years ago
Martin Atkins	ba0514106a	return tfdiags.Diagnostics from validation methods ... Validation is the best time to return detailed diagnostics to the user since we're much more likely to have source location information, etc than we are in later operations. This change doesn't actually add any detail to the messages yet, but it changes the interface so that we can gradually introduce more detailed diagnostics over time. While here there are some minor adjustments to some of the messages to improve their consistency with terminology we use elsewhere.	9 years ago
Radek Simko	d39025b9d8	helper: Escape test name in TF_LOG_PATH_MASK	9 years ago
Radek Simko	48fbd60e33	helper: Fall back to TF_LOG_PATH instead of os.Stderr	9 years ago
James Bardin	3a495ffe56	rename ModuleStorage to Storage ... get rid of stutter and use module.Storage	9 years ago
James Bardin	f2a7b94692	use the new ModuleStorage in the command package ... Update the command package to use the new module storage. Move the old command output strings into the module storage itself. This could be moved back later either by using ui callbacks, or designing a module storage interface once we know what the final requirements will look like.	9 years ago
Radek Simko	4052a8f956	helper: Allow logs isolation per acceptance test	9 years ago
Chris Marchesi	7bba1d0c95	helper/resource: Add ImportStateIdFunc ... Add an ImportStateIdFunc field to the ImportState testing functionality. This will allow for more powerful generation of complex import state IDs that can't be accomplished by ImportStateId or ImportStateIdPrefix themselves.	9 years ago
Radek Simko	db97555e3e	helper/resource: Add TestStep.SkipFunc	9 years ago
James Bardin	60ea42cd1c	Add testProviderConfig to import tests ... Provider import tests previously didn't have to supply a config, but terraform now requires the provider to be declared for discovery. testProviderConfig returns a stub config with provider blocks based on the TestCase Providers. This allows basic import tests in providers to remain unchanged.	9 years ago
Martin Atkins	c835ef8ff3	Update tests for the new ProviderResolver interface ... Rather than providing an already-resolved map of plugins to core, we now provide a "provider resolver" which knows how to resolve a set of provider dependencies, to be determined later, and produce that map. This requires the context to be instantiated in a different way, so this very noisy diff is a mostly-mechanical update of all of the existing places where contexts get created for testing, using some adapted versions of the pre-existing utilities for passing in mock providers.	9 years ago
Martin Atkins	0573ff6793	helper/resource: pass config when testing import ... Previously having a config was mutually exclusive with running an import, but we need to provide a config so that the provider is declared, or else we can't actually complete the import in the future world where providers are installed dynamically based on their declarations.	9 years ago
Clint	372a80bc42	provider/aws: Add Sweeper setup, Sweepers for DB Option Group, Key Pair (#14773) ... * provider/aws: Add Sweeper setup, Sweepers for DB Option Group, Key Pair * provider/google: Add sweeper for any leaked databases * more recursion and added LC sweeper, to test out the Dependency path * implement a dependency example * implement sweep-run flag to filter runs * stub a test for TestMain * test for multiple -sweep-run list	9 years ago
=	1608f5544f	Added error check	9 years ago
=	6bc3610c09	removing extraneous code	9 years ago
Jake Champlin	f24087ee54	core: Add ImportStateIdPrefix field for testing ... Adds the `ImportStateIdPrefix` field for import acceptance tests. There are (albeit fairly rare) import cases where a resource needs to be imported with a combination of the resource's ID and a known string prefix. This allows the developer to specify the known prefix, and omit the `ImportStateId` field. ``` $ make test TEST=./helper/resource TESTARGS="-run=TestTest_importStateIdPrefix" ==> Checking that code complies with gofmt requirements... ==> Checking AWS provider for unchecked errors... ==> NOTE: at this time we only look for uncheck errors in the AWS package go generate $(go list ./... | grep -v /terraform/vendor/) 2017/03/30 18:08:36 Generated command/internal_plugin_list.go go test -i ./helper/resource || exit 1 echo ./helper/resource | \ xargs -t -n4 go test -run=TestTest_importStateIdPrefix -timeout=60s -parallel=4 go test -run=TestTest_importStateIdPrefix -timeout=60s -parallel=4 ./helper/resource ok github.com/hashicorp/terraform/helper/resource 0.025s ```	9 years ago
clint shryock	61355c33c5	testing: Add option to run only a plan on a TestStep configuration	9 years ago
James Bardin	6fcb55d09e	reword TestProvider doc	9 years ago
James Bardin	1eb9a2d083	Reset ResourceProviders ... Call all ResourceProviderFactories and reset the providers before tests. Store the provider and/or the error in a fixed factory function to be returned again later.	9 years ago