46 Commits (cef46ef953face71f52f4e900a0ea2132ebfae88)

Author	SHA1	Message	Date
James Bardin	f3fe6184a0	test for destroy plan round trip	7 years ago
James Bardin	1b8617cef0	don't attempt to decode empty changes values ... An empty DynamicValue can't be decoded but indicates no state, so just return a NullVal.	7 years ago
James Bardin	a915f3f13e	don't convert empty DynamicValue to nil	7 years ago
James Bardin	78256ae225	return early when comparing Null values	8 years ago
Martin Atkins	300eceeb25	plans/planfile: fix TestRoundtrip ... This was broken by an earlier change to verify the Terraform version number when reading a state file. To fix it, we'll use our current version in our constructed file which should then match when it's read back in.	8 years ago
Martin Atkins	ab62b330c1	core: Allow planned output changes to be updated during apply ... If plan and apply are both run against the same context then we still have the planned output values in memory while we're doing the apply walk, so we need to make sure to update them along with the state as we learn the final known values of each output. There were actually two different bugs here: - We weren't removing any existing planned change for an output when setting a new one. In retrospect a map would've been a better data structure for the output changes, rather than a slice to mimic what we do for resource instance objects, but for now we'll leave the structures alone and clean up as needed. (The set of outputs should be small for any reasonable configuration, so the main impact of this is some ugly code in RemoveOutputChange.) - RemoveOutputChange itself had a bug where it was iterating over the resource changes rather than the output changes. This didn't matter before because we weren't actually using that function, but now we are. This fix is confirmed by restoring various existing context apply tests back to passing again.	8 years ago
Martin Atkins	bbf8dacac8	plans: OutputChange.Encode must preserve Addr field	8 years ago
James Bardin	e93d69f18b	more nil/known checks before val.LengthInt	8 years ago
James Bardin	e08a388d3c	check IsKnown on values that may panic	8 years ago
Martin Atkins	ec57927ea3	build: Take protoc out of the "go generate" path ... Since protoc is not go-gettable, and most development tasks in Terraform won't involve recompiling protoc files anyway, we'll use a separate mechanism for these. This way "go generate" only depends on things we can "go get" in the "make tools" target. In a later commit we should also in some way specify a particular version of protoc to use so that we don't get "flapping" regenerations as developers work with different versions, but the priority here is just to make "make generate" minimally usable again to restore the dev workflow documented in the README. This also includes some updates that resulted from running "make generate" and "make protobuf" after those Makefile changes were in place.	8 years ago
Martin Atkins	9b4b43c077	plans/objchange: Don't panic when a prior value with a set is null ... ProposedNewObject intentionally replaces a null prior with an unknown prior in order to easily fill in unknown values where they "show through" under values not set explicitly in config, but it was failing to handle that situation when dealing with nested blocks that are backed by sets.	8 years ago
Martin Atkins	2b80df0163	backend/local: Require caller to set PlanOutBackend with PlanOutPath ... We can't generate a valid plan file without a backend configuration to write into it, but it's the responsibility of the caller (the command package) to manage the backend configuration mechanism, so we require it to tell us what to write here. This feels a little strange because the backend in principle knows its own config, but in practice the backend only knows the _processed_ version of the config, not the raw configuration value that was used to configure it.	8 years ago
Kristin Laemmert	2d3cb87789	backend/local tests tests tests ... converted the existing testPlanState() from terraform.State to states.State to fix various plan tests. reverted the "bandaid" in plans/planfile/tfplan.go - at this moment the backend tests do not include backend configuration, and so the planfile package can write the plan file but not read it back in. That will be revisted in a separate track of work.	8 years ago
Kristin Laemmert	6a37ee9277	backend/local: more tests passing ... I have no confidence in the change to plans/planfile/tfplan.go. The tests were passing an empty backend config, which planfile was able to write to a file but not read from the same file. This change let me move past that and it did not break any tests in the planfile package, but I am concerned that it introduces undesired behavior.	8 years ago
Martin Atkins	b0016e9cf6	command: Allow tests to run to completion without panics or hangs ... There are still 160 test failures as of this commit, but at least the test program can run to completion and list out all the failures.	8 years ago
Kristin Laemmert	c661157999	plans/objchange: further harden ProposedNewObject against ~weird~ ... incoming values Addresses an odd state where the priorV of an object to be changed is known but null. While this situation should not happen, it seemed prudent to ensure that core is resilient to providers sending incorrect values (which might also occur with manually edited state).	8 years ago
Kristin Laemmert	2a8aa6a139	plans/objchange: if priorV is unknown, fall through to the recursive call to `ProposedNewObject`	8 years ago
Martin Atkins	32974549cd	plans/objchange: Fix handling of unknown in AssertValueCompatible ... We need to check for the known-ness of the prior value before we check for the null-ness of actual, because it's valid for an unknown value to become a null.	8 years ago
Martin Atkins	896b6bc897	core: Fix test build for ./plans/planfile ... This was missed in the splitting of "Replace" into "DeleteThenCreate" and "CreateThenDelete".	8 years ago
Martin Atkins	9179cdcbc6	plans/planfile: allow resource instances with no instance key ... This happens for resources that don't have either "count" or "for_each" set.	8 years ago
Martin Atkins	3e9d23b120	plans: Regenerate Action.String for new action values ... This was missed on the initial update of these because at the time the package wasn't generate-able due to other problems.	8 years ago
Martin Atkins	a43b7df282	core: Handle forced-create_before_destroy during the plan walk ... Previously we used a single plan action "Replace" to represent both the destroy-before-create and the create-before-destroy variants of replacing. However, this forces the apply graph builder to jump through a lot of hoops to figure out which nodes need it forced on and rebuild parts of the graph to represent that. If we instead decide between these two cases at plan time, the actual determination of it is more straightforward because each resource is represented by only one node in the plan graph, and then we can ensure we put the right nodes in the graph during DiffTransformer and thus avoid the logic for dealing with deposed instances being spread across various different transformers and node types. As a nice side-effect, this also allows us to show the difference between destroy-then-create and create-then-destroy in the rendered diff in the CLI, although this change doesn't fully implement that yet.	8 years ago
James Bardin	b738cdb19f	make Changes.Empty() not count NoOps ... This make the plan `Empty` concept more closely match the legacy diff behavior. Remove old unknown field from plan_test	8 years ago
Martin Atkins	d53c3d5c1b	plans: Retain output value changes for all outputs in memory ... During the plan operation we need to retain _somewhere_ the planned changes for all outputs so we can refer to them during expression evaluation. For consistency with how we handle resource instance changes, we'll keep them in the plan so we can properly retain unknown values, which cannot be written to state. As with output values in the state, only root output plans are retained in a round-trip through the on-disk plan file format, but that's okay because we can trivially re-calculate all of these during apply. We include the _root_ outputs in the plan file only because they are externally-visible side effects that ought to be included in any rendering of the plan made from the plan file for user inspection.	8 years ago
Martin Atkins	8048e9a585	plans/objchange: Don't panic if old or new values are null	8 years ago
Martin Atkins	6fd82ef97e	core: Split Replace changes into separate Delete/Create changes ... Since we do our deletes using a separate graph node from all of the other actions, and a "Replace" change implies both a delete _and_ a create, we need to pretend at apply time that a single replace change was actually two separate changes. This will also early-exit eval if a destroy node finds a non-Delete change or if an apply node finds a Delete change. These should not happen in practice because we leave these nodes out of the graph when they are not needed for the given action, but we do this here for robustness so as not to have an invisible dependency between the graph builder and the eval phase.	8 years ago
Martin Atkins	1aa9ac14cc	plans/objchange: LongestCommonSubsequence ... This algorithm is the usual first step when generating diffs. This package is a bit of a strange home for it, but since it works with changes to cty.Value this feels more natural than any other place it could be.	8 years ago
Martin Atkins	1ced176fc6	plans: Track RequiredReplace as a cty.PathSet ... We were previously tracking this as a []cty.Path, but having it turned into a pathset on creation makes downstream use of it more convenient and ensures that it'll obey expected invariants like not containing the same path twice.	8 years ago
Martin Atkins	3f8a973846	plans/objchange: when prior is null, computed attributes are unknown	8 years ago
Martin Atkins	1d672623eb	core: Remove changes from the plan after they are applied	8 years ago
Martin Atkins	2f0e5d93c8	plans: ChangesSync.GetResourceInstanceChange must copy the change ... This is promised in its doc comment, but wasn't actually done in practice.	8 years ago
Martin Atkins	3d86dc51e0	core: Re-implement EvalReadDiff for the new plan types ... This also includes passing in the provider schema to a few more EvalNodes that were expecting it but not getting it, in order to be able to successfully test the implementation of EvalReadDiff here.	8 years ago
Martin Atkins	bd299b9a22	core: Re-implement EvalWriteDiff to work with new plan types	8 years ago
Martin Atkins	44bc7519a6	terraform: More wiring in of new provider types ... This doesn't actually work yet, but it builds and then panics in a pretty satisfying way.	8 years ago
Martin Atkins	549544f201	configschema: Handle nested blocks containing dynamic-typed attributes ... We need to make the collection itself be a tuple or object rather than list or map in this case, since otherwise all of the elements of the collection are constrained to be of the same type and that isn't the intent of a provider indicating that it accepts any type.	8 years ago
Martin Atkins	4c78539c2b	plans/objchange: AssertObjectSuperset function ... This function's goal is to ensure that the "final" plan value produced by a provider during the apply step is always consistent with the known parts of the planned value produced during the plan step. Any error produced here indicates a bug in the provider.	8 years ago
Martin Atkins	0317da9911	plans/objchange: logic for merging prior state with config ... This produces a "proposed new state", which already has prior computed values propagated into it (since that behavior is standard for all resource types) but could be customized further by the provider to make the "_planned_ new state". In the process of implementing this it became clear that our configschema DecoderSpec behavior is incorrect, since it's producing list values for NestingList and map values for NestingMap. While that seems like it should be right, we should actually be using tuple and object types respectively to allow each block to have a different runtime type in situations where an attribute is given the type cty.DynamicPseudoType. That's not fixed here, and so without a further fix list and map blocks will panic here. The DecoderSpec implementation will be fixed in a subsequent commit.	8 years ago
Martin Atkins	dfe0988f10	plans: Record private data bytes as part of resource change ... This allows a provider to retain arbitrary extra data in the plan and make use of it during apply. The contents are not used by Terraform and never shown to the user.	8 years ago
Martin Atkins	a3403f2766	terraform: Ugly huge change to weave in new State and Plan types ... Due to how often the state and plan types are referenced throughout Terraform, there isn't a great way to switch them out gradually. As a consequence, this huge commit gets us from the old world to a _compilable_ new world, but still has a large number of known test failures due to key functionality being stubbed out. The stubs here are for anything that interacts with providers, since we now need to do the follow-up work to similarly replace the old terraform.ResourceProvider interface with its replacement in the new "providers" package. That work, along with work to fix the remaining failing tests, will follow in subsequent commits. The aim here was to replace all references to terraform.State and its downstream types with states.State, terraform.Plan with plans.Plan, state.State with statemgr.State, and switch to the new implementations of the state and plan file formats. However, due to the number of times those types are used, this also ended up affecting numerous other parts of core such as terraform.Hook, the backend.Backend interface, and most of the CLI commands. Just as with 5861dbf3fc before, I apologize in advance to the person who inevitably just found this huge commit while spelunking through the commit history.	8 years ago
Martin Atkins	b7db32b819	plans: separate types for encoded and decoded changes ... The types here were originally written to allow us to defer decoding of object values until schemas are available, but it turns out that this was forcing us to defer decoding longer than necessary and potentially decode the same value multiple times. To avoid this, we create pairs of types to represent the encoded and decoded versions and methods for moving between them. These types are identical to one another apart from how the dynamic values are represented.	8 years ago
Martin Atkins	074db88636	plans: Include target addresses in the plan ... We shouldn't really need these because the plan is already filtered to include diffs only for targeted resources, but we currently rely on this to filter out non-resource items from the diff, and so we'll retain it for now to avoid reworking how the apply-time graph builder works.	8 years ago
Martin Atkins	d9dfd135c6	plans: Include backend settings in plan and plan files ... On the first pass here we erroneously assumed that this was redundant with the backend settings embedded in the configuration itself. In practice, users can override backend configuration when running "terraform init" and so we need to record the _effective_ backend configuration. Along with this, we also return the selected workspace name at the time the plan was created so we'll later be able to produce a specialized error for the situation of having the wrong workspace selected. This isn't strictly required because we'll also check the lineage of the state, but the error message that would result from that failure would be relatively opaque and thus less helpful to the user.	8 years ago
Martin Atkins	f77e7a61b0	various: helpers for collecting necessary provider types ... Since schemas are required to interpret provider, resource, and provisioner attributes in configs, states, and plans, these helpers intend to make it easier to gather up the the necessary provider types in order to preload all of the needed schemas before beginning further processing. Config.ProviderTypes returns directly the list of provider types, since at this level further detail is not useful: we've not yet run the provider allocation algorithm, and so the only thing we can reliably extract here is provider types themselves. State.ProviderAddrs and Plan.ProviderAddrs each return a list of absolute provider addresses, which can then be turned into a list of provider types using the new helper providers.AddressedTypesAbs. Since we're already using configs.Config throughout core, this also updates the terraform.LoadSchemas helper to use Config.ProviderTypes to find the necessary providers, rather than implementing its own discovery logic. states.State is not yet plumbed in, so we cannot yet use State.ProviderAddrs to deal with the state but there's a TODO comment to remind us to update that in a later commit when we swap out terraform.State for states.State. A later commit will probably refactor this further so that we can easily obtain schema for the providers needed to interpret a plan too, but that is deferred here because further work is required to make core work with the new plan types first. At that point, terraform.LoadSchemas may become providers.LoadSchemas with a different interface that just accepts lists of provider and provisioner names that have been gathered by the caller using these new helpers.	8 years ago
Martin Atkins	51aeba2cf2	plans: Record provider config address in each resource change ... This allows us to record the resource-to-provider associations we made during the plan phase and ensure that we use the same config during apply. This isn't technically necessary, since we can in principle just repeat the same matching algorithm against the config, but that algorithm is relatively complicated and so if we execute it just once during plan we remove the risk of bugs causing different decisions to be made during the apply phase. This also includes updates to the plan file format to include the provider addresses. Since we've not actually shipped any version of Terraform using our protobuf schema yet, we renumber here all of the fields after the new one to keep them incrementing consecutively. This example should not be followed after we have a released version of Terraform using this, as a courtesy to anyone trying to parse these files (even though we're not yet guaranteeing compatibility between releases).	8 years ago
Martin Atkins	a2eb462f5d	plans/planfile: Reading and writing the new plan format ... The new format is radically different in than the old in physical structure, but still has the same logical parts: the plan itself, a snapshot of the input configuration, and a snapshot of the state as it existed when the plan was created. Rather than creating plan-specific serializations of state and config, the new format instead leans on the existing file formats implemented elsewhere, wrapping the result up in a zip archive with some internal file naming conventions. The plan portion of the file is serialized with protobuf, consistent with our general strategy of replacing all use of encoding/gob with protobuf moving forward.	8 years ago
Martin Atkins	7357e7f734	plans: New package for in-memory plan models ... The types in this package are intended to replace plan- and diff-related types from the "terraform" package, although those older types must remain for now so that they can be used to implement shims for older codepaths. type "Changes" is approximately equivalent to terraform.Diff, but renamed since it now describes whole objects before and after rather than an attribute-level diff as before. The term "diff" is now reserved for the visual rendition of the changes we'll display to the user, although rendering of this new Changes model is not yet implemented.	8 years ago