18 Commits (6440c839474f052d8997bb5cec19c2743e5108e6)

Author	SHA1	Message	Date
Pam Selle	3e7be13dff	Update ordering for marking/unmarking and asserting plan valid ... Update when we unmark objects so we can assert the plan is valid, and process UnknownAsNull on the unmarked value	5 years ago
Pam Selle	0a02e7040f	Store sensitive attribute paths in state (#26338) ... * Add creation test and simplify in-place test * Add deletion test * Start adding marking from state Start storing paths that should be marked when pulled out of state. Implements deep copy for attr paths. This commit also includes some comment noise from investigations, and fixing the diff test * Fix apply stripping marks * Expand diff tests * Basic apply test * Update comments on equality checks to clarify current understanding * Add JSON serialization for sensitive paths We need to serialize a slice of cty.Path values to be used to re-mark the sensitive values of a resource instance when loading the state file. Paths consist of a list of steps, each of which may be either getting an attribute value by name, or indexing into a collection by string or number. To serialize these without building a complex parser for a compact string form, we render a nested array of small objects, like so: [ [ { type: "get_attr", value: "foo" }, { type: "index", value: { "type": "number", "value": 2 } } ] ] The above example is equivalent to a path `foo[2]`. * Format diffs with map types Comparisons need unmarked values to operate on, so create unmarked values for those operations. Additionally, change diff to cover map types * Remove debugging printing * Fix bug with marking non-sensitive values When pulling a sensitive value from state, we were previously using those marks to remark the planned new value, but that new value might *not* be sensitive, so let's not do that * Fix apply test Apply was not passing the second state through to the third pass at apply * Consistency in checking for length of paths vs inspecting into value * In apply, don't mark with before paths * AttrPaths test coverage for DeepCopy * Revert format changes Reverts format changes in format/diff for this branch so those changes can be discussed on a separate PR * Refactor name of AttrPaths to AttrSensitivePaths * Rename AttributePaths/attributePaths for naming consistency Co-authored-by: Alisdair McDiarmid <alisdair@users.noreply.github.com>	6 years ago
Pam Selle	843ed8911b	Don't save PathValueMarks on instance_object	6 years ago
Pam Selle	bc55b6a28b	Use UnmarkDeepWithPaths and MarkWithPaths ... Updates existing code to use the new Value methods for unmarking/marking and removes panics/workarounds in cty marshall methods	6 years ago
James Bardin	9fe87fe520	remove unused depends_on field and add test ... The depends_on fields was deprecated during 0.12, and is no longer used. Add a roundtrip test for the create_before_destroy status.	6 years ago
James Bardin	23f5fc3e5a	thread create_before_destroy to the state file	6 years ago
James Bardin	b3fc0dab94	use addrs.ConfigResource for dependency tracking ... We can't get module instances during transformation, so we need to reduce the Dependencies to using `addrs.ConfigResource` for now.	6 years ago
James Bardin	dd8ab5812e	add CreateBeforeDestroy to state ... Added to src and object, but still need serialization and tests.	6 years ago
James Bardin	2c3c011f20	change state dependencies to AbsResource addrs ... We need to be able to reference all possible dependencies for ordering when the configuration is no longer present, which means that absolute addresses must be used. Since this is only to recreate the proper ordering for instance destruction, only resources addresses need to be listed rather than individual instance addresses.	6 years ago
Radek Simko	7860f55e4f	Version tools per Go convention under tools.go	6 years ago
Paul Tyng	ab2d09b0cb	godoc typo	7 years ago
Martin Atkins	9eb32c4536	core: Reinstaint instance tainting, but without mutating objects ... Our previous mechanism for dealing with tainting relied on directly mutating the InstanceState object to mark it as such. In our new state models we consider the instance objects to be immutable by convention, and so we frequently copy them. As a result, the taint flagging was no longer making it all the way through the apply evaluation process. Here we now implement tainting as a separate step in the evaluation process, creating a copy of the object with a tainted status if there were any errors during creation. This introduces a new behavior where any provider-level errors during creation will also cause an instance to be marked as tainted if any object is returned at all. Create-time errors _normally_ result in no object at all, but the provider might return an object if the failure occurred at a subsequent step of a multi-step creation process and so left behind a remote object that needs to be cleaned up on a future run.	7 years ago
Martin Atkins	9af67806fc	core: Prune placeholder objects from state after refresh ... Prior to our refactoring here, we were relying on a lucky coincidence for correct behavior of the plan walk following a refresh in the same run: - The refresh phase created placeholder objects in the state to represent any resource instance pending creation, to allow the interpolator to read attributes from them when evaluating "provider" and "data" blocks. In effect, the refresh walk is creating a partial plan that only covers creation actions, but was immediately discarding the actual diff entries and storing only the planned new state. - It happened that objects pending creation showed up in state with an empty ID value, since that only gets assigned by the provider during apply. - The Refresh function concluded by calling terraform.State.Prune, which deletes from the state any objects that have an empty ID value, which therefore prevented these temporary objects from surviving into the plan phase. After refactoring, we no longer have this special ID field on instance object state, and we instead rely on the Status field for tracking such things. We also no longer have an explicit "prune" step on state, since the state mutation methods themselves keep the structure pruned. To address this, here we introduce a new instance object status "planned", which is equivalent to having an empty ID value in the old world. We also introduce a new method on states.SyncState that deletes from the state any planned objects, which therefore replaces that portion of the old State.prune operation just for this refresh use-case. Finally, we are now expecting the expression evaluator to pull pending objects from the planned changeset rather than from the state directly, and so for correct results these placeholder resource creation changes must also be reported in a throwaway changeset during the refresh walk. The addition of states.ObjectPlanned also permits a previously-missing safety check in the expression evaluator to prevent us from relying on the incomplete value stored in state for a pending object, in the event that some bug prevents the real pending object from being written into the planned changeset.	7 years ago
Martin Atkins	20adb9d9b7	core: Evaluate resource references from plan where possible ... Our state representation is not able to preserve unknown values, so it's not suitable for retaining the transient incomplete values we produce during planning. Instead, we'll discard the unknown values when writing to state and have the expression evaluator prefer an object from the plan where possible. We still use the shape of the transient state to inform things like the resource's "each mode", so the plan only masks the object values themselves.	7 years ago
Martin Atkins	fb57801dfe	states: object Private data is []byte, not cty.Value ... We're going to allow the provider to encode whatever it wants in here, so a provider can use whatever is most convenient for its implementation language and to avoid some of the bugs we saw with the prior model where the forced round-trip through JSON and back into interface{} would cause some loss of fidelity, leading to bugs.	7 years ago
Martin Atkins	a3403f2766	terraform: Ugly huge change to weave in new State and Plan types ... Due to how often the state and plan types are referenced throughout Terraform, there isn't a great way to switch them out gradually. As a consequence, this huge commit gets us from the old world to a _compilable_ new world, but still has a large number of known test failures due to key functionality being stubbed out. The stubs here are for anything that interacts with providers, since we now need to do the follow-up work to similarly replace the old terraform.ResourceProvider interface with its replacement in the new "providers" package. That work, along with work to fix the remaining failing tests, will follow in subsequent commits. The aim here was to replace all references to terraform.State and its downstream types with states.State, terraform.Plan with plans.Plan, state.State with statemgr.State, and switch to the new implementations of the state and plan file formats. However, due to the number of times those types are used, this also ended up affecting numerous other parts of core such as terraform.Hook, the backend.Backend interface, and most of the CLI commands. Just as with 5861dbf3fc before, I apologize in advance to the person who inevitably just found this huge commit while spelunking through the commit history.	7 years ago
Martin Atkins	424afe0ace	states: separate types for encoded and decoded state objects ... The types here were originally written to allow us to defer decoding of object values until schemas are available, but it turns out that this was forcing us to defer decoding longer than necessary and potentially decode the same value multiple times. To avoid this, we create pairs of types to represent the encoded and decoded versions and methods for moving between them. These types are identical to one another apart from how the dynamic values are represented.	7 years ago
Martin Atkins	b975ada8db	states: New package with modern models for Terraform state ... Our previous state models in the "terraform" package had a few limitations that are addressed here: - Instance attributes were stored as map[string]string with dot-separated keys representing traversals through a data structure. Now that we have a full type system, it's preferable to store it as a real data structure. - The existing state structures skipped over the "resource" concept and went straight to resource instance, requiring heuristics to decide whether a particular resource should appear as a single object or as a list of objects when used in configuration expressions. - Related to the previous point, the state models also used incorrect terminology where "ResourceState" was really a resource instance state and "InstanceState" was really the state of a particular remote object associated with an instance. These new models use the correct names for each of these, introducing the idea of a "ResourceInstanceObject" as the local record of a remote object associated with an instance. This is a first pass at fleshing out a new model for state. Undoubtedly there will be further iterations of this as we work on integrating these new models into the "terraform" package. These new model types no longer serve double-duty as a description of the JSON state file format, since they are for in-memory use only. A subsequent commit will introduce a separate package that deals with persisting state to files and reloading those files later.	7 years ago