21967 Commits (dd04450c526a180e7108985dd06b3bd866d2dabc)

 

All Branches   

Search

Author	SHA1	Message	Date
Martin Atkins	dd04450c52	govendor fetch github.com/hashicorp/hcl2/... ... This includes updates to various diagnostic messages to improve precision and consistency of terminology. It also includes some other changes to portions of HCL API that Terraform isn't yet using.	8 years ago
Martin Atkins	3da6ba2634	govendor fetch github.com/zclconf/go-cty/... ... This update includes a new conversion from tuple types to set types and improvements to the error messages returned when conversions fail.	8 years ago
James Bardin	5ac28fef5d	Merge pull request #18484 from hashicorp/jbardin/planned-config ... add Config to the PlanResourceChangeRequest	8 years ago
James Bardin	0d843ac150	add Config to the PlanResourceChangeRequest ... In order for providers to determine if a computed value was unset in the configuration, the configuration must be passed along with the prior and proposed states. Terraform core will still handle the creation of the ProposedNewState to ensure correctness, but the raw configuration value will also be supplied for comparison.	8 years ago
Kristin Laemmert	37e60bbb7d	builtin/providers: terraform remote state datasource (#18446) ... * builtin/providers: implement terraform remote state datasource as providers.Interface * append and return diags separately (to match the idiomatic usage elsewhere in Terraform) * diagnostic summary style improvements * update tests to pass config to schema.CoerceValue * trust that the schema will be enforced and there is no need to check that a given attribute exists * added dataSourceRemoteStateGetSchema() (effectively replacing a function that was inappropriately removed) for consistency with other terraform providers * builtin/provider terraform test: added InternalValidate() test for dataSourceRemoteStateGetSchema	8 years ago
Martin Atkins	608b7ce8d8	statemgr: Backup file support for statemgr.Filesystem ... In the old state package we had this as a separate manager state.BackupState, but that doesn't work with our new interfaces because we handle lineage and serial within the state managers themselves and don't expose them to callers anymore. In practice it being built in to the filesystem manager is not a problem because we only use the backup functionality for local state anyway. This also slightly adjusts the behavior to be more intuitive. The old BackupState relied on the implementation detail that Terraform re-persists the original state early in an apply operation, which meant that by coincidence it would back up the right snapshot. In this new approach, we instead take an in-memory copy during State and then write _that_ to disk in WriteState if the new state seems different, so we're guaranteed that we'll always write out what we read before any changes were made. In future we may improve this further, such as keeping multiple generations of backups, etc. But for now this is intended to preserve the goals of the original implementation while making its behavior self-contained and not dependent on coincidences.	8 years ago
James Bardin	a4c034253a	Merge pull request #18437 from hashicorp/jbardin/schema-id ... Add required "id" field to resource schemas	8 years ago
James Bardin	2d0a028071	use a custom comparer for cty.Type ... Make sure we also compare cty.Types in `cmp.Equal`, even though they contain unexported fields.	8 years ago
James Bardin	58366af05b	SetId should set the attribute as well ... The update protocol shims will also check for this this, but eventually "id" will only be a normal attribute, and we shouldn't have to special case this.	8 years ago
James Bardin	6a86c76351	add implicit "id" to resource attribute schemas ... When converting a legacy schemaMap to a configschema, we need to add "id" as a required attribute to top-level resources if it's not declared. The "id" field will be required to interoperate with the legacy helper schema, since the presence of an id was used to indicate the existence of a resource.	8 years ago
James Bardin	2defe47d46	add go-cmp ... This was much easier to use when tracking down diffs in large structures.	8 years ago
Martin Atkins	eb13ffe269	plans: Include target addresses in the plan ... We shouldn't really need these because the plan is already filtered to include diffs only for targeted resources, but we currently rely on this to filter out non-resource items from the diff, and so we'll retain it for now to avoid reworking how the apply-time graph builder works.	8 years ago
Martin Atkins	e9f4e03062	plans: Include backend settings in plan and plan files ... On the first pass here we erroneously assumed that this was redundant with the backend settings embedded in the configuration itself. In practice, users can override backend configuration when running "terraform init" and so we need to record the _effective_ backend configuration. Along with this, we also return the selected workspace name at the time the plan was created so we'll later be able to produce a specialized error for the situation of having the wrong workspace selected. This isn't strictly required because we'll also check the lineage of the state, but the error message that would result from that failure would be relatively opaque and thus less helpful to the user.	8 years ago
Martin Atkins	d0a0f60738	govendor fetch github.com/zclconf/go-cty/... ... This adds an ImpliedType to the msgpack package, which serves the same purpose as ImpliedType in the json package.	8 years ago
James Bardin	7c0b31905c	Merge pull request #18423 from hashicorp/jbardin/shims ... fix shims to handle nil flatmaps	8 years ago
James Bardin	7243b5aa57	return a nil flatmap when presented with a NullVal ... A nil flatmap will be encoded as a NullVal of the correct type. When Converting a NullVal back to a flatmap, return nil immediately rather than attempting to build the values.	8 years ago
James Bardin	9a1e1226cd	return a NullVal when presented with a nil flatmap ... This allows us to decode the cty.Value back to a nil map, rather than trying to create a map with nil values.	8 years ago
Martin Atkins	a98d25f41d	move "configschema" from "config" to "configs" ... The "config" package is no longer used and will be removed as part of the 0.12 release cleanup. Since configschema is part of the "new world" of configuration modelling, it makes more sense for it to live as a subdirectory of the newer "configs" package.	8 years ago
Martin Atkins	cfdc951920	core: validate that depends_on addresses are for objects ... Since the "References" function on graph nodes can't return errors, we need to catch invalid depends_on references during the validation pass. In this case, we're checking that the address is exact, rather than being part of a traversal into an attribute of the object. In other words, aws_instance.example is valid but aws_instance.example.id is not.	8 years ago
James Bardin	cf7bc6f98c	Merge pull request #18368 from hashicorp/jbardin/tfdiags ... add tfdiags.GetAttribute	8 years ago
James Bardin	764d2663c1	add tfdiags.GetAttribute ... The only contextual information returned by the plugins in a diagnostic is an attribute path. In order to test the conversions, we need to be able to extract the cty.Path from a diagnostic if it exists.	8 years ago
Martin Atkins	0ad31dee79	configupgrade: Analysis of input configuration ... In order to properly migrate the contents of resource, data, provider and provisioner blocks we will need the provider's schema in order to understand what is expected, so we can resolve some ambiguities inherent in the legacy HCL AST. This includes an initial prototype of migrating the content of resource blocks just to verify that the information is being gathered correctly. As with the rest of the upgrade_native.go file, this will be reorganized significantly once the basic end-to-end flow is established and we can see how to organize this code better.	8 years ago
Martin Atkins	d12a3d6287	lang/funcs: never include the private key in error output ... This is based on c811440188 made against the old "config" package implementations, but also catches a few other cases where we would previously have printed the private key into the error messages.	8 years ago
Martin Atkins	816452d4c8	core: Assorted small test fixes ... Rebasing the v0.12 dev branch to master caught a few changes that didn't pass tests afterwards, so this is a catchup commit to fix those.	8 years ago
Martin Atkins	f1561a17f4	core: Don't DynamicExpand during validate ... Previously we would attempt to DynamicExpand during the validate walk and then validate each expanded instance separately. However, this meant that we would not be able to validate the contents of a block where count = 0 or if count is not yet known. Here we instead do a more static validation pass against the resource configuration itself, setting count.index to cty.UnknownVal(cty.Number) so we can type-check everything inside the block as being correct regardless of the final count. This is another step towards repairing the "validate" command for our changed assumptions in a world where we have a more sophisticated type checker. This doesn't yet address the remaining problem that the expression evaluator can't, with the current state structures, distinguish between a completed resource with count = 0 and a resource that doesn't exist at all (during validate), and so we'll still get errors if an expression elsewhere in configuration refers to a dynamic index of a resource with "count" set. That's a pre-existing condition that's no longer being masked by _this_ problem, but can't be addressed until we've introduced the new state types (states.State, etc) and thus we _can_ distinguish these two situations. That will therefore be addressed in a later commit.	8 years ago
Martin Atkins	b2334c095f	core: Handle count.index evaluation more explicitly ... Previously we had the evaluate methods accept directly an addrs.InstanceKey and had our evaluator infer a suitable value for count.index for it, but that prevents us from setting the index to be unknown in the validation scenario where we may not be able to predict the number of instances yet but we still want to be able to check that the configuration block is type-safe for all possible count values. To achieve this, we separate the concern of deciding on a value for count.index from the concern of evaluating it, which then allows for other implementations of this in future. For the purpose of this commit there is no change in behavior, with the count.index value being populated whenever the instance key is a number. This commit does a little more groundwork for the future implementation of the for_each feature (which'll support each.key and each.value) but still doesn't yet implement it, leaving it just stubbed out for the moment.	8 years ago
Martin Atkins	849293825b	command: validate must set values for root variables ... Since the intent of the validate command is to check config validity regardless of context (input variables, state, etc), we use unknown values of the requested type here, which will then allow us to complete type checking against the specified types of the variables without assuming any particular values.	8 years ago
Martin Atkins	3cbf433936	various: helpers for collecting necessary provider types ... Since schemas are required to interpret provider, resource, and provisioner attributes in configs, states, and plans, these helpers intend to make it easier to gather up the the necessary provider types in order to preload all of the needed schemas before beginning further processing. Config.ProviderTypes returns directly the list of provider types, since at this level further detail is not useful: we've not yet run the provider allocation algorithm, and so the only thing we can reliably extract here is provider types themselves. State.ProviderAddrs and Plan.ProviderAddrs each return a list of absolute provider addresses, which can then be turned into a list of provider types using the new helper providers.AddressedTypesAbs. Since we're already using configs.Config throughout core, this also updates the terraform.LoadSchemas helper to use Config.ProviderTypes to find the necessary providers, rather than implementing its own discovery logic. states.State is not yet plumbed in, so we cannot yet use State.ProviderAddrs to deal with the state but there's a TODO comment to remind us to update that in a later commit when we swap out terraform.State for states.State. A later commit will probably refactor this further so that we can easily obtain schema for the providers needed to interpret a plan too, but that is deferred here because further work is required to make core work with the new plan types first. At that point, terraform.LoadSchemas may become providers.LoadSchemas with a different interface that just accepts lists of provider and provisioner names that have been gathered by the caller using these new helpers.	8 years ago
Martin Atkins	f976a086a9	plans: Record provider config address in each resource change ... This allows us to record the resource-to-provider associations we made during the plan phase and ensure that we use the same config during apply. This isn't technically necessary, since we can in principle just repeat the same matching algorithm against the config, but that algorithm is relatively complicated and so if we execute it just once during plan we remove the risk of bugs causing different decisions to be made during the apply phase. This also includes updates to the plan file format to include the provider addresses. Since we've not actually shipped any version of Terraform using our protobuf schema yet, we renumber here all of the fields after the new one to keep them incrementing consecutively. This example should not be followed after we have a released version of Terraform using this, as a courtesy to anyone trying to parse these files (even though we're not yet guaranteeing compatibility between releases).	8 years ago
Martin Atkins	c2987cd2b4	command: 0.12upgrade command ... This is the frontend to the work-in-progress codepath for upgrading the source code for a module written for Terraform v0.11 or earlier to use the new syntax and idiom of v0.12. The underlying upgrade code is not yet complete as of this commit, and so the command is not yet very useful. We will continue to iterate on the upgrade code in subsequent commits.	8 years ago
Martin Atkins	ca4875128e	configupgrade: Basic expression formatting ... This covers all of the expression node types in HIL's AST, and also includes initial support for some of our top-level blocks so that we can easily test that. The initial implementations of the "variable" and "output" blocks are pretty redundant and messy, so we can hopefully improve on these in a later pass.	8 years ago
Martin Atkins	1c5d05b1a0	configupgrade: Beginnings of Upgrade function ... This function is the main functionality of this package. So far it just deals with detecting and renaming JSON files that are mislabeled as native syntax files. Other functionality will follow in later commits.	8 years ago
Martin Atkins	79b361160a	configupgrade: Load source code for a module and detect already upgraded ... This package will do all of its work in-memory so that it can avoid making partial updates and then failing, so we need to be able to load the sources files from a particular directory into memory. The upgrade process isn't idempotent, so we also attempt to detect heuristically whether an upgrade has already been performed (can parse with the new parser and has a version constraint that prevents versions earlier than 0.12) so that the CLI tool that will eventually wrap this will be able to produce a warning and prompt for confirmation in that case.	8 years ago
Martin Atkins	34b0a20ee5	configupgrade: new package for upgrading configs for 0.12 ... Although the new HCL implementation and configuration loader is broadly compatible with the prior implementation, it has a number of new idiomatic forms and it also cannot parse some more extreme non-idiomatic usages that were possible under the old parser. To help users migrate to the new implementation, this package will rewrite configuration to comply with the new idiom and fix as many cases as possible where the legacy parser was too liberal or exposed implementation details.	8 years ago
Martin Atkins	2b124a1d00	configload: Don't download the same module source multiple times ... It is common for the same module source package to be referenced multiple times in the same configuration, either because there are literally multiple instances of the same module source or because a single package (or repository) contains multiple modules in sub-directories and many of them are referenced. To optimize this, here we introduce a simple caching behavior where the module installer will detect if it's asked to install multiple times from the same source and produce the second and subsequent directories by copying the first, rather than by downloading again over the network. This optimization is applied once all of the go-getter detection has completed and sub-directory portions have been trimmed, so it is also able to normalize differently-specified source addresses that all ultimately detect to the same resolved address. When installing, we always extract the entire specified package (or repository) and then reference the specified sub-directory, so we can safely re-use existing directories when the base package is the same, even if the sub-directory is different. However, as a result we do not yet address the fact that the same package will be stored multiple times _on disk_, which may still be problematic when referencing large repositories multiple times in disk-storage-constrained environments. We could address this in a subsequent change by investigating the use of symlinks where possible. Since the Registry installer is implemented just as an extra resolution step in front of go-getter, this optimization applies to registry modules too. This does not apply to local relative references, which will continue to just resolve into the already-prepared directory of their parent module. The cache of previously installed paths lives only for the duration of one call to InstallModules, so we will never re-use directories that were created by previous runs of "terraform init" and there is no risk that older versions will pollute the cache when attempting an upgrade from a source address that doesn't explicitly specify a version. No additional tests are added here because the existing module installer tests (when TF_ACC=1) already cover the case of installing multiple modules from the same source.	8 years ago
James Bardin	119665ec6a	add GetSchema to provisioner interface	8 years ago
James Bardin	661848d30d	Add ImportedResource type ... The ImportResourceStateResponse command can return mutliple types. ImportedResource maps the state to the appropriate type name, and also allows providers to return Private data for each resource.	8 years ago
James Bardin	b385c86269	use cty.Value for connection info ... For now this will be an encoded map, but it allows for a more complex schema in the future.	8 years ago
James Bardin	483490f8c2	rename UpgradedState	8 years ago
James Bardin	d694076fb2	split PriorStateRaw into JSON and Flatmap ... This way the provider doesn't need to be concerned with determining the state encoding.	8 years ago
James Bardin	39650828c0	add versions to the provider schemas ... We need to know the schema version for all resources. This is stored in order to allow providers to upgrade the state from a known previous version.	8 years ago
James Bardin	b942eb7f7f	rename provider and provisioner packages ... Using plural names to avoid collisions in existing code.	8 years ago
Martin Atkins	3d304efd91	tfdiags: Sort order for diagnostics ... Because we gather together diagnostics from many different parts of the codebase, the list often ends up being in a non-ideal order. Here we define a partial ordering for diagnostics that should hopefully make them easier to scan when many are present, by grouping together diagnostics that are of the same severity and belong to the same file. We use sort.Stable here because we have a partial order and so we need to make sure that diagnostics that do not have a relative ordering will remain in their original order. This sorting is applied just in time before rendering the diagnostics in command.Meta.showDiagnostics.	8 years ago
James Bardin	2b5ae042b8	Add provider plugin interface and types ... This adds the new interface for the resource providers. The methods here closely follow the grpc service methods, using the types expected in terraform core	8 years ago
James Bardin	56a9af829f	Add provisioner interface and request types ... This adds the interface for the new resource provisioners. The methods closely follow the grpc service methods, using the type expected within terraform core.	8 years ago
Martin Atkins	4466754bb9	plans/planfile: Reading and writing the new plan format ... The new format is radically different in than the old in physical structure, but still has the same logical parts: the plan itself, a snapshot of the input configuration, and a snapshot of the state as it existed when the plan was created. Rather than creating plan-specific serializations of state and config, the new format instead leans on the existing file formats implemented elsewhere, wrapping the result up in a zip archive with some internal file naming conventions. The plan portion of the file is serialized with protobuf, consistent with our general strategy of replacing all use of encoding/gob with protobuf moving forward.	8 years ago
Martin Atkins	ab3dccff1f	plans: New package for in-memory plan models ... The types in this package are intended to replace plan- and diff-related types from the "terraform" package, although those older types must remain for now so that they can be used to implement shims for older codepaths. type "Changes" is approximately equivalent to terraform.Diff, but renamed since it now describes whole objects before and after rather than an attribute-level diff as before. The term "diff" is now reserved for the visual rendition of the changes we'll display to the user, although rendering of this new Changes model is not yet implemented.	8 years ago
Martin Atkins	628a846b24	configload: Configuration snapshots ... Here we introduce a new idea of a "configuration snapshot", which is an in-memory copy of the source code of each of the files that make up the configuration. The primary intended purpose for this is as an intermediate step before writing the configuration files into a plan file, and then reading them out when that plan file is later applied. During earlier configs package development we expected to use an afero vfs implementation to read directly from the zip file, but that doesn't work in practice because we need to preserve module paths from the source file system that might include parent directory traversals (../) while retaining the original path for use in error messages. The result, for now, is a bit of an abstraction inversion: we implement a specialized afero vfs implementation that makes the sparse filesystem representation from a snapshot appear like a normal filesystem just well enough that the config loader and parser can work with it. In future we may wish to rework the internals here so that the main abstraction is at a similar level to the snapshot and then that API is mapped to the native filesystem in the normal case, removing afero. For now though, this approach avoids the need for a significant redesign of the parser/loader internals, at the expense of some trickiness in the case where we're reading from a snapshot. This commit does not yet include the reading and writing of snapshots into plan files. That will follow in a subsequent commit.	8 years ago
Martin Atkins	f98ca58f75	govendor fetch github.com/zclconf/go-cty/cty/... ... This allows automatic conversions between different object types as long as the target type is a subset of the given type.	8 years ago
Martin Atkins	2a5d6348e8	govendor fetch github.com/zclconf/go-cty/cty/... ... The existing cty packages were already at the latest version, but we were not yet vendoring the msgpack package. This also imports some dependencies from: github.com/vmihailenco/msgpack	8 years ago