31 Commits (6440c839474f052d8997bb5cec19c2743e5108e6)

Author	SHA1	Message	Date
Kristin Laemmert	3933cbd491	remove LegacyProvider (#26433)	6 years ago
Pam Selle	0a02e7040f	Store sensitive attribute paths in state (#26338) ... * Add creation test and simplify in-place test * Add deletion test * Start adding marking from state Start storing paths that should be marked when pulled out of state. Implements deep copy for attr paths. This commit also includes some comment noise from investigations, and fixing the diff test * Fix apply stripping marks * Expand diff tests * Basic apply test * Update comments on equality checks to clarify current understanding * Add JSON serialization for sensitive paths We need to serialize a slice of cty.Path values to be used to re-mark the sensitive values of a resource instance when loading the state file. Paths consist of a list of steps, each of which may be either getting an attribute value by name, or indexing into a collection by string or number. To serialize these without building a complex parser for a compact string form, we render a nested array of small objects, like so: [ [ { type: "get_attr", value: "foo" }, { type: "index", value: { "type": "number", "value": 2 } } ] ] The above example is equivalent to a path `foo[2]`. * Format diffs with map types Comparisons need unmarked values to operate on, so create unmarked values for those operations. Additionally, change diff to cover map types * Remove debugging printing * Fix bug with marking non-sensitive values When pulling a sensitive value from state, we were previously using those marks to remark the planned new value, but that new value might *not* be sensitive, so let's not do that * Fix apply test Apply was not passing the second state through to the third pass at apply * Consistency in checking for length of paths vs inspecting into value * In apply, don't mark with before paths * AttrPaths test coverage for DeepCopy * Revert format changes Reverts format changes in format/diff for this branch so those changes can be discussed on a separate PR * Refactor name of AttrPaths to AttrSensitivePaths * Rename AttributePaths/attributePaths for naming consistency Co-authored-by: Alisdair McDiarmid <alisdair@users.noreply.github.com>	6 years ago
Martin Atkins	549aede792	Remove terraform.ResourceProvider, use providercache.Installer instead ... Back when we first introduced provider versioning in Terraform 0.10, we did the provider version resolution in terraform.NewContext because we weren't sure yet how exactly our versioning model was going to play out (whether different versions could be selected per provider configuration, for example) and because we were building around the limitations of our existing filesystem-based plugin discovery model. However, the new installer codepath is new able to do all of the selections up front during installation, so we don't need such a heavy inversion of control abstraction to get this done: the command package can select the exact provider versions and pass their factories directly to terraform.NewContext as a simple static map. The result of this commit is that CLI commands other than "init" are now able to consume the local cache directory and selections produced by the installation process in "terraform init", passing all of the selected providers down to the terraform.NewContext function for use in implementing the main operations. This commit is just enough to get the providers passing into the terraform.Context. There's still plenty more to do here, including to repair all of the tests this change has additionally broken.	6 years ago
James Bardin	e13eecbc5b	finish provider ModuleInstance replacement	6 years ago
Paddy	e6592dc710	Add support for provider metadata to modules. (#22583) ... Implement a new provider_meta block in the terraform block of modules, allowing provider-keyed metadata to be communicated from HCL to provider binaries. Bundled in this change for minimal protocol version bumping is the addition of markdown support for attribute descriptions and the ability to indicate when an attribute is deprecated, so this information can be shown in the schema dump. Co-authored-by: Paul Tyng <paul@paultyng.net>	6 years ago
Kristin Laemmert	47a16b0937	addrs: embed Provider in AbsProviderConfig instead of Type ... a large refactor to addrs.AbsProviderConfig, embedding the addrs.Provider instead of a Type string. I've added and updated tests, added some Legacy functions to support older state formats and shims, and added a normalization step when reading v4 (current) state files (not the added tests under states/statefile/roundtrip which work with both current and legacy-style AbsProviderConfig strings). The remaining 'fixme' and 'todo' comments are mostly going to be addressed in a subsequent PR and involve looking up a given local provider config's FQN. This is fine for now as we are only working with default assumption.	6 years ago
Martin Atkins	8b511524d6	Initial steps towards AbsProviderConfig/LocalProviderConfig separation (#23978) ... * Introduce "Local" terminology for non-absolute provider config addresses In a future change AbsProviderConfig and LocalProviderConfig are going to become two entirely distinct types, rather than Abs embedding Local as written here. This naming change is in preparation for that subsequent work, which will also include introducing a new "ProviderConfig" type that is an interface that AbsProviderConfig and LocalProviderConfig both implement. This is intended to be largely just a naming change to get started, so we can deal with all of the messy renaming. However, this did also require a slight change in modeling where the Resource.DefaultProviderConfig method has become Resource.DefaultProvider returning a Provider address directly, because this method doesn't have enough information to construct a true and accurate LocalProviderConfig -- it would need to refer to the configuration to know what this module is calling the provider it has selected. In order to leave a trail to follow for subsequent work, all of the changes here are intended to ensure that remaining work will become obvious via compile-time errors when all of the following changes happen: - The concept of "legacy" provider addresses is removed from the addrs package, including removing addrs.NewLegacyProvider and addrs.Provider.LegacyString. - addrs.AbsProviderConfig stops having addrs.LocalProviderConfig embedded in it and has an addrs.Provider and a string alias directly instead. - The provider-schema-handling parts of Terraform core are updated to work with addrs.Provider to identify providers, rather than legacy strings. In particular, there are still several codepaths here making legacy provider address assumptions (in order to limit the scope of this change) but I've made sure each one is doing something that relies on at least one of the above changes not having been made yet. * addrs: ProviderConfig interface In a (very) few special situations in the main "terraform" package we need to make runtime decisions about whether a provider config is absolute or local. We currently do that by exploiting the fact that AbsProviderConfig has LocalProviderConfig nested inside of it and so in the local case we can just ignore the wrapping AbsProviderConfig and use the embedded value. In a future change we'll be moving away from that embedding and making these two types distinct in order to represent that mapping between them requires consulting a lookup table in the configuration, and so here we introduce a new interface type ProviderConfig that can represent either AbsProviderConfig or LocalProviderConfig decided dynamically at runtime. This also includes the Config.ResolveAbsProviderAddr method that will eventually be responsible for that local-to-absolute translation, so that callers with access to the configuration can normalize to an addrs.AbsProviderConfig given a non-nil addrs.ProviderConfig. That's currently unused because existing callers are still relying on the simplistic structural transform, but we'll switch them over in a later commit. * rename LocalType to LocalName Co-authored-by: Kristin Laemmert <mildwonkey@users.noreply.github.com>	6 years ago
Kristin Laemmert	6541775ce4	addrs: roll back change to Type field in ProviderConfig (#23937)	6 years ago
Kristin Laemmert	e3416124cc	addrs: replace "Type string" with "Type Provider" in ProviderConfig ... * huge change to weave new addrs.Provider into addrs.ProviderConfig * terraform: do not include an empty string in the returned Providers / Provisioners - Fixed a minor bug where results included an extra empty string	6 years ago
Kristin Laemmert	9891d0354a	providers: use addrs.Provider as map keys for provider.Factory (#23548) ... * terraform/context: use new addrs.Provider as map key in provider factories * added NewLegacyProviderType and LegacyString funcs to make it explicit that these are temporary placeholders This PR introduces a new concept, provider fully-qualified name (FQN), encapsulated by the `addrs.Provider` struct.	6 years ago
James Bardin	dcab82e897	send and receive Private through ReadResource ... Send Private data blob through ReadResource as well. This will allow for extra flexibility for future providers that may want to pass data out of band through to their resource Read functions.	7 years ago
Martin Atkins	31299e688d	core: Allow legacy SDK to opt out of plan-time safety checks ... Due to the inprecision of our shimming from the legacy SDK type system to the new Terraform Core type system, the legacy SDK produces a number of inconsistencies that produce only minor quirky behavior or broken edge-cases. To retain compatibility with those existing weird behaviors, the legacy SDK opts out of our safety checks. The intent here is to allow existing providers to continue to do their previous unsafe behaviors for now, accepting that this will allow certain quirky bugs from previous releases to persist, and then gradually migrate away from the legacy SDK and remove this opt-out on a per-resource basis over time. As with the apply-time safety check opt-out, this is reserved only for the legacy SDK and must not be used in any new SDK implementations. We still include any inconsistencies as warnings in the logs as an aid to anyone debugging weird behavior, so that they can see situations where blame may be misplaced in the user-visible error messages.	7 years ago
Martin Atkins	1530fe52f7	core: Legacy SDK providers opt out of our new apply result check ... The shim layer for the legacy SDK type system is not precise enough to guarantee it will produce identical results between plan and apply. In particular, values that are null during plan will often become zero-valued during apply. To avoid breaking those existing providers while still allowing us to introduce this check in the future, we'll introduce a rather-hacky new flag that allows the legacy SDK to signal that it is the legacy SDK and thus disable the check. Once we start phasing out the legacy SDK in favor of one that natively understands our new type system, we can stop setting this flag and thus get the additional safety of this check without breaking any previously-released providers. No other SDK is permitted to set this flag, and we will remove it if we ever introduce protocol version 6 in future, assuming that any provider supporting that protocol will always produce consistent results.	7 years ago
Martin Atkins	72e279e6b2	providers: Consistently use int64 for schema versions ... The rest of Terraform is still using uint64 for this in various spots, but we'll update that gradually later. We use int64 here because that matches what's used in our protobuf definition, and unsigned integers are not portable across all of the protobuf target languages anyway.	7 years ago
James Bardin	c1303f8482	providers PrepareProviderConfig ... Change ValidateProviderConfig to PrepareProviderConfig. Providers have a concept of "required fields with defaults" that that was handled previously by helper/schema doing input and validation. Because Input and Validation is mostly now handled by core, the provider had no way of setting default values for missing required attributes. To achieve the same behavior with new providers, there will be a PrepareProviderConfig, which allow for manual validation, as well as alteration of the config. The provider is free to set whatever attributes necessary to create a valid config and return it to the caller. If a new config is returned, it will be used instead of the original in the subsequent Configure call, however core may still add missing required values during an optional Input phase.	8 years ago
Martin Atkins	44bc7519a6	terraform: More wiring in of new provider types ... This doesn't actually work yet, but it builds and then panics in a pretty satisfying way.	8 years ago
Martin Atkins	594a821ab3	providers: Resolver and Factory types ... These new types are adaptations of terraform.ProviderResolver and terraform.ResourceProviderFactory respectively. In a later commit we will update users of the old types to use these new types, possibly in a "gradual repair" sort of fashion by initially using type aliases, though that won't 100% solve the problem because the types themselves have changed in this move to return providers.Interface instead of terraform.ResourceProvider.	8 years ago
James Bardin	c39e9e107f	add Close methods to the plugin interfaces ... Close was previously an optional interface, because it's only applicable on the client side but the types were shared accross all packages. Since all plugins will now implement Close, it no longer needs to be optional.	8 years ago
Martin Atkins	a3403f2766	terraform: Ugly huge change to weave in new State and Plan types ... Due to how often the state and plan types are referenced throughout Terraform, there isn't a great way to switch them out gradually. As a consequence, this huge commit gets us from the old world to a _compilable_ new world, but still has a large number of known test failures due to key functionality being stubbed out. The stubs here are for anything that interacts with providers, since we now need to do the follow-up work to similarly replace the old terraform.ResourceProvider interface with its replacement in the new "providers" package. That work, along with work to fix the remaining failing tests, will follow in subsequent commits. The aim here was to replace all references to terraform.State and its downstream types with states.State, terraform.Plan with plans.Plan, state.State with statemgr.State, and switch to the new implementations of the state and plan file formats. However, due to the number of times those types are used, this also ended up affecting numerous other parts of core such as terraform.Hook, the backend.Backend interface, and most of the CLI commands. Just as with 5861dbf3fc before, I apologize in advance to the person who inevitably just found this huge commit while spelunking through the commit history.	8 years ago
James Bardin	f9fee83279	minor providers.Interface fixes ... Add a missing TerraformVersion field, and fix a TypeName field.	8 years ago
James Bardin	4fedf6c6f4	removing connection data from the providers ... Very few resources need connection info, and rather than relying on providers to set default for some resources, we are going to require that connection blocks be explicitly set in the configuration.	8 years ago
James Bardin	cd53b333f2	add config value to ApplyResourceChange ... Providers will also need the config for reference during apply, in case some of the config values were still unknown during plan.	8 years ago
James Bardin	f77dd12e5c	add Config to the PlanResourceChangeRequest ... In order for providers to determine if a computed value was unset in the configuration, the configuration must be passed along with the prior and proposed states. Terraform core will still handle the creation of the ProposedNewState to ensure correctness, but the raw configuration value will also be supplied for comparison.	8 years ago
Martin Atkins	479c6b2466	move "configschema" from "config" to "configs" ... The "config" package is no longer used and will be removed as part of the 0.12 release cleanup. Since configschema is part of the "new world" of configuration modelling, it makes more sense for it to live as a subdirectory of the newer "configs" package.	8 years ago
Martin Atkins	f77e7a61b0	various: helpers for collecting necessary provider types ... Since schemas are required to interpret provider, resource, and provisioner attributes in configs, states, and plans, these helpers intend to make it easier to gather up the the necessary provider types in order to preload all of the needed schemas before beginning further processing. Config.ProviderTypes returns directly the list of provider types, since at this level further detail is not useful: we've not yet run the provider allocation algorithm, and so the only thing we can reliably extract here is provider types themselves. State.ProviderAddrs and Plan.ProviderAddrs each return a list of absolute provider addresses, which can then be turned into a list of provider types using the new helper providers.AddressedTypesAbs. Since we're already using configs.Config throughout core, this also updates the terraform.LoadSchemas helper to use Config.ProviderTypes to find the necessary providers, rather than implementing its own discovery logic. states.State is not yet plumbed in, so we cannot yet use State.ProviderAddrs to deal with the state but there's a TODO comment to remind us to update that in a later commit when we swap out terraform.State for states.State. A later commit will probably refactor this further so that we can easily obtain schema for the providers needed to interpret a plan too, but that is deferred here because further work is required to make core work with the new plan types first. At that point, terraform.LoadSchemas may become providers.LoadSchemas with a different interface that just accepts lists of provider and provisioner names that have been gathered by the caller using these new helpers.	8 years ago
James Bardin	12521366e1	Add ImportedResource type ... The ImportResourceStateResponse command can return mutliple types. ImportedResource maps the state to the appropriate type name, and also allows providers to return Private data for each resource.	8 years ago
James Bardin	dc78269a3c	use cty.Value for connection info ... For now this will be an encoded map, but it allows for a more complex schema in the future.	8 years ago
James Bardin	960553e18b	rename UpgradedState	8 years ago
James Bardin	6a1b57a1e4	split PriorStateRaw into JSON and Flatmap ... This way the provider doesn't need to be concerned with determining the state encoding.	8 years ago
James Bardin	91b5bbbde0	add versions to the provider schemas ... We need to know the schema version for all resources. This is stored in order to allow providers to upgrade the state from a known previous version.	8 years ago
James Bardin	1473d09c50	rename provider and provisioner packages ... Using plural names to avoid collisions in existing code.	8 years ago