10 Commits (ccf19fbf5e2b61ba62efd1ac47a3a3fd3917d494)

Author	SHA1	Message	Date
James Bardin	2c4c027a97	Add ModuleOutputs method to states ... In order to efficiently build the module objects for evaluation, we need to collect the outputs from a set of module instances. The ModuleOutputs method will return a copy of the state outputs, while not requiring the unnecessary copying of each entire module.	6 years ago
Martin Atkins	4061cbed38	internal/getproviders: A new shared model for provider requirements ... We've been using the models from the "moduledeps" package to represent our provider dependencies everywhere since the idea of provider dependencies was introduced in Terraform 0.10, but that model is not convenient to use for any use-case other than the "terraform providers" command that needs individual-module-level detail. To make things easier for new codepaths working with the new-style provider installer, here we introduce a new model type getproviders.Requirements which is based on the type the new installer was already taking as its input. We have new methods in the states, configs, and earlyconfig packages to produce values of this type, and a helper to merge Requirements together so we can combine config-derived and state-derived requirements together during installation. The advantage of this new model over the moduledeps one is that all of recursive module walking is done up front and we produce a simple, flat structure that is more convenient for the main use-cases of selecting providers for installation and then finding providers in the local cache to use them for other operations. This new model is _not_ suitable for implementing "terraform providers" because it does not retain module-specific requirement details. Therefore we will likely keep using moduledeps for "terraform providers" for now, and then possibly at a later time consider specializing the moduledeps logic for only what "terraform providers" needs, because it seems to be the only use-case that needs to retain that level of detail.	6 years ago
James Bardin	3729e6a705	update MaybeFixUpResourceInstanceAddressForCount ... We need to fixup count by config path, not by absolute path, so update MaybeFixUpResourceInstanceAddressForCount to work with addrs.ConfigResource.	6 years ago
Martin Atkins	b0016e9cf6	command: Allow tests to run to completion without panics or hangs ... There are still 160 test failures as of this commit, but at least the test program can run to completion and list out all the failures.	8 years ago
Martin Atkins	ec2e6cb06f	terraform: Prune resource husks at the end of "terraform destroy" ... When we're being asked to destroy everything, we ideally want to end up with a totally empty state. Normally we will conservatively keep around the "husks" of resources (what's left after all of the instances have been destroyed) unless they are configured without count or for_each, but in this special case we'll prune those out. The implication of this is that in "weird" expression contexts that happen before the next "terraform plan", such as evaluation in "terraform console" or expressions in data resources and provider blocks that get evaluated during the refresh walk, we will see these results as unknown rather than as empty lists of objects. We accept that weirdness for now because in a future release we are likely to remove "refresh" as a separate walk anyway, doing all of that work during the plan walk where we can ensure that these values are properly re-populated before trying to use them.	8 years ago
Martin Atkins	334c6f1c2c	core: Be more explicit in how we handle create_before_destroy ... Previously our handling of create_before_destroy -- and of deposed objects in particular -- was rather "implicit" and spread over various different subsystems. We'd quietly just destroy every deposed object during a destroy operation, without any user-visible plan to do so. Here we make things more explicit by tracking each deposed object individually by its pseudorandomly-allocated key. There are two different mechanisms at play here, building on the same concepts: - During a replace operation with create_before_destroy, we *pre-allocate* a DeposedKey to use for the prior object in the "apply" node and then pass that exact id to the destroy node, ensuring that we only destroy the single object we planned to destroy. In the happy path here the user never actually sees the allocated deposed key because we use it and then immediately destroy it within the same operation. However, that destroy may fail, which brings us to the second mechanism: - If any deposed objects are already present in state during _plan_, we insert a destroy change for them into the plan so that it's explicit to the user that we are going to destroy these additional objects, and then create an individual graph node for each one in DiffTransformer. The main motivation here is to be more careful in how we handle these destroys so that from a user's standpoint we never destroy something without the user knowing about it ahead of time. However, this new organization also hopefully makes the code itself a little easier to follow because the connection between the create and destroy steps of a Replace is reprseented in a single place (in DiffTransformer) and deposed instances each have their own explicit graph node rather than being secretly handled as part of the main instance-level graph node.	8 years ago
Martin Atkins	a3403f2766	terraform: Ugly huge change to weave in new State and Plan types ... Due to how often the state and plan types are referenced throughout Terraform, there isn't a great way to switch them out gradually. As a consequence, this huge commit gets us from the old world to a _compilable_ new world, but still has a large number of known test failures due to key functionality being stubbed out. The stubs here are for anything that interacts with providers, since we now need to do the follow-up work to similarly replace the old terraform.ResourceProvider interface with its replacement in the new "providers" package. That work, along with work to fix the remaining failing tests, will follow in subsequent commits. The aim here was to replace all references to terraform.State and its downstream types with states.State, terraform.Plan with plans.Plan, state.State with statemgr.State, and switch to the new implementations of the state and plan file formats. However, due to the number of times those types are used, this also ended up affecting numerous other parts of core such as terraform.Hook, the backend.Backend interface, and most of the CLI commands. Just as with 5861dbf3fc before, I apologize in advance to the person who inevitably just found this huge commit while spelunking through the commit history.	8 years ago
Martin Atkins	f77e7a61b0	various: helpers for collecting necessary provider types ... Since schemas are required to interpret provider, resource, and provisioner attributes in configs, states, and plans, these helpers intend to make it easier to gather up the the necessary provider types in order to preload all of the needed schemas before beginning further processing. Config.ProviderTypes returns directly the list of provider types, since at this level further detail is not useful: we've not yet run the provider allocation algorithm, and so the only thing we can reliably extract here is provider types themselves. State.ProviderAddrs and Plan.ProviderAddrs each return a list of absolute provider addresses, which can then be turned into a list of provider types using the new helper providers.AddressedTypesAbs. Since we're already using configs.Config throughout core, this also updates the terraform.LoadSchemas helper to use Config.ProviderTypes to find the necessary providers, rather than implementing its own discovery logic. states.State is not yet plumbed in, so we cannot yet use State.ProviderAddrs to deal with the state but there's a TODO comment to remind us to update that in a later commit when we swap out terraform.State for states.State. A later commit will probably refactor this further so that we can easily obtain schema for the providers needed to interpret a plan too, but that is deferred here because further work is required to make core work with the new plan types first. At that point, terraform.LoadSchemas may become providers.LoadSchemas with a different interface that just accepts lists of provider and provisioner names that have been gathered by the caller using these new helpers.	8 years ago
Martin Atkins	a33f941778	states: New SyncState type ... This is a wrapper around State that is able to perform higher-level manipulations (at the granularity of the entire state) in a concurrency-safe manner, using the lower-level APIs exposed by State and all of the types it contains. The granularity of a SyncState operation roughly matches the granularity off a state-related EvalNode in the "terraform" package, performing a sequence of more primitive operations while guaranteeing atomicity of the entire change. As a compromise for convenience of usage, it's still possible to access the individual state data objects via this API, but they are always copied before returning to ensure that two distinct callers cannot have data races. Callers should access the most granular object possible for their operation.	8 years ago
Martin Atkins	b975ada8db	states: New package with modern models for Terraform state ... Our previous state models in the "terraform" package had a few limitations that are addressed here: - Instance attributes were stored as map[string]string with dot-separated keys representing traversals through a data structure. Now that we have a full type system, it's preferable to store it as a real data structure. - The existing state structures skipped over the "resource" concept and went straight to resource instance, requiring heuristics to decide whether a particular resource should appear as a single object or as a list of objects when used in configuration expressions. - Related to the previous point, the state models also used incorrect terminology where "ResourceState" was really a resource instance state and "InstanceState" was really the state of a particular remote object associated with an instance. These new models use the correct names for each of these, introducing the idea of a "ResourceInstanceObject" as the local record of a remote object associated with an instance. This is a first pass at fleshing out a new model for state. Undoubtedly there will be further iterations of this as we work on integrating these new models into the "terraform" package. These new model types no longer serve double-duty as a description of the JSON state file format, since they are for in-memory use only. A subsequent commit will introduce a separate package that deals with persisting state to files and reloading those files later.	8 years ago