terraform

Commit Graph

Author	SHA1	Message	Date
Barrett Clark	d29532cfeb	tfc apply before init https://app.asana.com/0/1199201948575144/1201019450474843/f	5 years ago
Barrett Clark	e49f271eb5	Update copy for terraform init with cloud block	5 years ago
Chris Arcand	09d5b70b7a	command/meta_backend_migrate:Tweak multi-to-single To TFC copy	5 years ago
Omar Ismail	dfb4609be2	Backend State Migration from cloud to cloud. * Add test for tfc to tfc mgiration * Fix old tests, and remove unused code.	5 years ago
Barrett Clark	93bfcff61a	More Terraform Cloud UX changes * Update e2e tests to specify Terraform Cloud * Update more of the terraform command flow to specify Terraform Cloud	5 years ago
Barrett Clark	261a2b49d3	UX for terraform init that copies state to TFC	5 years ago
Barrett Clark	0cd9490838	PR Feedback: cleanup variable setter	5 years ago
Chris Arcand	f8256f6634	Update ux for "terraform init [-reconfigure]"	5 years ago
Omar Ismail	dc76bbee73	Backend State Migration: Add remote backend test	5 years ago
Omar Ismail	3fedd6898c	Backend State Migration to `cloud`: Multiple Workspaces * Handle when there are multiple workspaces migrating to cloud, using both the cloud name strategy and cloud tags strategy. * Add e2e tests	5 years ago
Barrett Clark	edbc84420c	Run apply -refresh-state instead of refresh When a user runs `terraform refresh` we give them an error message that tells them to run `terraform apply -refresh-state`. We could just run that command for them, though. That is what this PR does.	5 years ago
Omar Ismail	55fc590904	Teraform Cloud Backend State Migration * determining source or destination to cloud * handling single to single state migrations to cloud, using a name strategy or a tags strategy * Add end-to-end tests for state migration.	5 years ago
Chris Arcand	fccc873a3d	Remove ability to declare a 'cloud' backend With the alternative block introduced in 7bf9b2c7b, this removes the ability to explicitly declare the 'cloud' backend. The literal backend interface is an implementation detail and no longer a user-level concept when using Terraform Cloud.	5 years ago
Chris Arcand	a4c24e3147	Add cloud {} configuration block for Terraform Cloud This is a replacement declaration for using Terraform Cloud as a remote backend, leaving the literal backend as an implementation detail and not a user-level concept.	5 years ago
James Bardin	9b62dd6901	Merge pull request #29648 from hashicorp/jbardin/tfproto6.1 tfproto version 6.1	5 years ago
James Bardin	d03a037567	insert panic handlers	5 years ago
James Bardin	42742c173d	remove wrapped streams and readline	5 years ago
James Bardin	622c4df14c	remove the use of panicwrap Stop using panicwrap, and execute terraform in the main process.	5 years ago
Alisdair McDiarmid	ded4f1a0fd	Merge pull request #29805 from hashicorp/alisdair/fix-init-workspace-select-input-false command/init: Fail if -input=false but required	5 years ago
Alisdair McDiarmid	ecb98e1c43	command/init: Fail if -input=false but required When running `terraform init` against a backend with multiple workspaces, none of which are the currently indicated local workspace, Terraform prompts the user to choose a workspace from the list. In automation, using the `-input=false` argument should disable asking for input, but previously would hang instead.	5 years ago
Alisdair McDiarmid	6d0521e6be	Merge pull request #29803 from hashicorp/alisdair/init-backend-provider-errors cli: Fix backend init failure with deleted cache	5 years ago
Alisdair McDiarmid	22923f9873	Merge pull request #29793 from hashicorp/alisdair/fix-remote-backend-migrate-version-check backend/remote: Fix version check when migrating	5 years ago
Alisdair McDiarmid	d42d83572b	cli: Fix backend init failure with deleted cache When an explicit backend is configured with a configuration which has not yet been initialized, running `terraform init` performs a state migration to fetch the remotely stored state in order to operate on it. Like the previous bug introduced by the recent provider diagnostics change, this code path was not correctly configured to enable init mode for the backend, which resulted in a fatal error during init when the cache dir is deleted. Setting the `Init` backend option allows this code path to continue without error when first initializing the backend for state migration. The new e2e test fails without this change.	5 years ago
Alisdair McDiarmid	1729431520	backend/remote: Fix version check when migrating When migrating state to an existing Terraform Cloud workspace using the remote backend, we check the remote version is compatible with the local one by default. This commit fixes two bugs in this code: - If using the "name" strategy for the remote backend, the list of destination workspaces is empty. This resulted in no version checking of the remote workspace, and we fell back to the string equality check. - The user-specified CLI flag `-ignore-remote-version` was not being applied for the state migration version checking.	5 years ago
Alisdair McDiarmid	39de3ebec7	cli: Fix init failure with deleted cache The init command needs to initialize a backend, in order to access state, in turn to derive provider requirements from state. The backend initialization step requires building provider factories, which previously would fail if a lockfile was present without a corresponding local provider cache. This commit ensures that in this situation only, errors with the provider factories are temporarily ignored. This allows us to continue to initialize the backend, fetch providers, and then report any errors as necessary.	5 years ago
Alisdair McDiarmid	1190b95fe2	command/e2etest: Ensure init fixes missing cache We test that a deleted provider cache results in an error when running terraform plan, but previously did not test that running init (as instructed) would resolve the issue. This (failing) e2e test adds that step.	5 years ago
Martin Atkins	5b266dd5ca	command: Remove the experimental "terraform add" command We introduced this experiment to gather feedback, and the feedback we saw led to us deciding to do another round of design work before we move forward with something to meet this use-case. In addition to being experimental, this has only been included in alpha releases so far, and so on both counts it is not protected by the Terraform v1.0 Compatibility Promises.	5 years ago
Alisdair McDiarmid	cdd5ee6fb3	Merge pull request #29773 from hashicorp/alisdair/init-lock-flags cli: Restore -lock and -lock-timeout init flags	5 years ago
Alisdair McDiarmid	c587384dff	cli: Restore -lock and -lock-timeout init flags The -lock and -lock-timeout flags were removed prior to the release of 1.0 as they were thought to have no effect. This is not true in the case of state migrations when changing backends. This commit restores these flags, and adds test coverage for locking during backend state migration. Also update the help output describing other boolean flags, showing the argument as the user would type it rather than the default behavior.	5 years ago
Alisdair McDiarmid	fb58f9e6d2	cli: Fix flaky init cancel test There is a race between the MockSource and ShutdownCh which sometimes causes this test to fail. Add a HangingSource implementation of Source which hangs until the context is cancelled, so that there is always time for a user-initiated shutdown to trigger the cancellation code path under test.	5 years ago
James Bardin	ef3c98466d	Merge pull request #29755 from hashicorp/jbardin/first-plan-lineage Check for stale plan with no state metadata	5 years ago
James Bardin	9c80574417	test planfile may need to have a specific lineage In order to test applying a plan from an existing state, we need to be able to inject the state meta into the planfile.	5 years ago
Martin Atkins	bee7403f3e	command/workspace_delete: Allow deleting a workspace with empty husks Previously we would reject attempts to delete a workspace if its state contained any resources at all, even if none of the resources had any resource instance objects associated with it. Nowadays there isn't any situation where the normal Terraform workflow will leave behind resource husks, and so this isn't as problematic as it might've been in the v0.12 era, but nonetheless what we actually care about for this check is whether there might be any remote objects that this state is tracking, and for that it's more precise to look for non-nil resource instance objects, rather than whole resources. This also includes some adjustments to our error messaging to give more information about the problem and to use terminology more consistent with how we currently talk about this situation in our documentation and elsewhere in the UI. We were also using the old State.HasResources method as part of some of our tests. I considered preserving it to avoid changing the behavior of those tests, but the new check seemed close enough to the intent of those tests that it wasn't worth maintaining this method that wouldn't be used in any main code anymore. I've therefore updated those tests to use the new HasResourceInstanceObjects method instead.	5 years ago
Alisdair McDiarmid	b9f3dab035	json-output: Release format version 1.0	5 years ago
Omar Ismail	bea7e3ebce	Backend State Migration: change variable names from one/two to source/destination (#29699 )	5 years ago
Martin Atkins	01b22f4b76	command/e2etest: TestProviderTampering We have various mechanisms that aim to ensure that the installed provider plugins are consistent with the lock file and that the lock file is consistent with the provider requirements, and we do have existing unit tests for them, but all of those cases mock our fake out at least part of the process and in the past that's caused us to miss usability regressions, where we still catch the error but do so at the wrong layer and thus generate error message lacking useful additional context. Here we'll add some new end-to-end tests to supplement the existing unit tests, making sure things work as expected when we assemble the system together as we would in a release. These tests cover a number of different ways in which the plugin selections can grow inconsistent. These new tests all run only when we're in a context where we're allowed to access the network, because they exercise the real plugin installer codepath. We could technically build this to use a local filesystem mirror or other such override to avoid that, but the point here is to make sure we see the expected behavior in the main case, and so it's worth the small additional cost of downloading the null provider from the real registry.	5 years ago
Martin Atkins	d09510a8fb	command: Early error message for missing cache entries of locked providers In the original incarnation of Meta.providerFactories we were returning into a Meta.contextOpts whose signature didn't allow it to return an error directly, and so we had compromised by making the provider factory functions themselves return errors once called. Subsequent work made Meta.contextOpts need to return an error anyway, but at the time we neglected to update our handling of the providerFactories result, having it still defer the error handling until we finally instantiate a provider. Although that did ultimately get the expected result anyway, the error ended up being reported from deep in the guts of a Terraform Core graph walk, in whichever concurrently-visited graph node happened to try to instantiate the plugin first. This meant that the exact phrasing of the error message would vary between runs and the reporting codepath didn't have enough context to given an actionable suggestion on how to proceed. In this commit we make Meta.contextOpts pass through directly any error that Meta.providerFactories produces, and then make Meta.providerFactories produce a special error type so that Meta.Backend can ultimately return a user-friendly diagnostic message containing a specific suggestion to run "terraform init", along with a short explanation of what a provider plugin is. The reliance here on an implied contract between two functions that are not directly connected in the callstack is non-ideal, and so hopefully we'll revisit this further in future work on the overall architecture of the CLI layer. To try to make this robust in the meantime though, I wrote it to use the errors.As function to potentially unwrap a wrapped version of our special error type, in case one of the intervening layers is changed at some point to wrap the downstream error before returning it.	5 years ago
Martin Atkins	df578afd7e	backend/local: Check dependency lock consistency before any operations In historical versions of Terraform the responsibility to check this was inside the terraform.NewContext function, along with various other assorted concerns that made that function particularly complicated. More recently, we reduced the responsibility of the "terraform" package only to instantiating particular named plugins, assuming that its caller is responsible for selecting appropriate versions of any providers that _are_ external. However, until this commit we were just assuming that "terraform init" had correctly selected appropriate plugins and recorded them in the lock file, and so nothing was dealing with the problem of ensuring that there haven't been any changes to the lock file or config since the most recent "terraform init" which would cause us to need to re-evaluate those decisions. Part of the game here is to slightly extend the role of the dependency locks object to also carry information about a subset of provider addresses whose lock entries we're intentionally disregarding as part of the various little edge-case features we have for overridding providers: dev_overrides, "unmanaged providers", and the testing overrides in our own unit tests. This is an in-memory-only annotation, never included in the serialized plan files on disk. I had originally intended to create a new package to encapsulate all of this plugin-selection logic, including both the version constraint checking here and also the handling of the provider factory functions, but as an interim step I've just made version constraint consistency checks the responsibility of the backend/local package, which means that we'll always catch problems as part of preparing for local operations, while not imposing these additional checks on commands that _don't_ run local operations, such as "terraform apply" when in remote operations mode.	5 years ago
Martin Atkins	6a98e4720c	plans/planfile: Create takes most arguments via a struct type Previously the planfile.Create function had accumulated probably already too many positional arguments, and I'm intending to add another one in a subsequent commit and so this is preparation to make the callsites more readable (subjectively) and make it clearer how we can extend this function's arguments to include further components in a plan file. There's no difference in observable functionality here. This is just passing the same set of arguments in a slightly different way.	5 years ago
Martin Atkins	8d193ad268	core: Simplify and centralize plugin availability checks Historically the responsibility for making sure that all of the available providers are of suitable versions and match the appropriate checksums has been split rather inexplicably over multiple different layers, with some of the checks happening as late as creating a terraform.Context. We're gradually iterating towards making that all be handled in one place, but in this step we're just cleaning up some old remnants from the main "terraform" package, which is now no longer responsible for any version or checksum verification and instead just assumes it's been provided with suitable factory functions by its caller. We do still have a pre-check here to make sure that we at least have a factory function for each plugin the configuration seems to depend on, because if we don't do that up front then it ends up getting caught instead deep inside the Terraform runtime, often inside a concurrent graph walk and thus it's not deterministic which codepath will happen to catch it on a particular run. As of this commit, this actually does leave some holes in our checks: the command package is using the dependency lock file to make sure we have exactly the provider packages we expect (exact versions and checksums), which is the most crucial part, but we don't yet have any spot where we make sure that the lock file is consistent with the current configuration, and we are no longer preserving the provider checksums as part of a saved plan. Both of those will come in subsequent commits. While it's unusual to have a series of commits that briefly subtracts functionality and then adds back in equivalent functionality later, the lock file checking is the only part that's crucial for security reasons, with everything else mainly just being to give better feedback when folks seem to be using Terraform incorrectly. The other bits are therefore mostly cosmetic and okay to be absent briefly as we work towards a better design that is clearer about where that responsibility belongs.	5 years ago
Zach Whaley	c9a5fdb366	cliconfig: Fix error message about invalid credentials helper type	5 years ago
James Bardin	ab0322e406	remove debugging println	5 years ago
James Bardin	c2e0d265cf	LoadModule now always returns the module We don't need to load the configuration twice, since configload can return the module for us.	5 years ago
James Bardin	a53faf43f6	return partial config from LoadConfig with errors LoadConfig should return any parsed configuration in order for the caller to verify `required_version`.	5 years ago
James Bardin	625e768678	make sure required_version is checked before diags We must ensure that the terraform required_version is checked as early as possible, so that new configuration constructs don't cause init to fail without indicating the version is incompatible. The loadConfig call before the earlyconfig parsing seems to be unneeded, and we can delay that to de-tangle it from installing the modules which may have their own constraints. TODO: it seems that loadConfig should be able to handle returning the version constraints in the same manner as loadSingleModule.	5 years ago
Alisdair McDiarmid	b699391d04	json-output: Add change reasons to explain deletes The extra feedback information for why resource instance deletion is planned is now included in the streaming JSON UI output. We also add an explicit case for no-op actions to switch statements in this package to ensure exhaustiveness, for future linting.	5 years ago
James Bardin	9847eaa9cf	remove usage of MinItems/MaxItems MinItems and MaxItems are not used on nested types in the protocol, so remove their usage in Terraform to prevent future confusion.	5 years ago
Martin Atkins	04f9e7148c	command/format: Include deletion reasons in plan report The core runtime is now able to specify a reason for some situations when Terraform plans to delete a resource instance. This commit makes that information visible in the human-oriented UI. A previous commit already made the underlying data informing these new hints visible as part of the machine-oriented (JSON) plan output. This also removes the bold formatting from the existing "has moved to" hints, because subjectively it seemed like the result was emphasizing too many parts of the output and thus somewhat defeating the benefit of the emphasis in trying to create additional visual hierarchy for sighted users running Terraform in a terminal. Now only the first line containing the main action statement will be in bold, and all of the parenthesized follow-up notes will be unformatted.	5 years ago
Martin Atkins	a1a713cf28	core: Report ActionReasons when we plan to delete "orphans" There are a few different reasons why a resource instance tracked in the prior state might be considered an "orphan", but previously we reported them all identically in the planned changes. In order to help users understand the reason for a surprising planned delete, we'll now try to specify an additional reason for the planned deletion, covering all of the main reasons why that could happen. This commit only introduces the new detail to the plans.Changes result, though it also incidentally exposes it as part of the JSON plan result in order to keep that working without returning errors in these new cases. We'll expose this information in the human-oriented UI output in a subsequent commit.	5 years ago
Chris Arcand	171cdbbf93	command: Clean up testInputResponseMap before failing on unused answers If you don't, the unused answers will persist in the package-level var and bleed in to other tests.	5 years ago
Chris Arcand	60bc7aa05d	command: Auto-select single workspace if necessary When initializing a backend, if the currently selected workspace does not exist, the user is prompted to select from the list of workspaces the backend provides. Instead, we should automatically select the only workspace available _if_ that's all that's there. Although with being a nice bit of polish, this enables future improvments with Terraform Cloud in potentially removing the implicit depenency on always using the 'default' workspace when the current configuration is mapped to a single TFC workspace.	5 years ago
Chris Arcand	08a86b6c13	Merge pull request #29621 from hashicorp/error-on-unused-test-answers command: Ensure all answers were used in command.testInputResponseMap	5 years ago
Chris Arcand	8684a85e26	command: Ensure all answers were used in command.testInputResponseMap Remove answers from testInputResponse as they are given, and raise an error during cleanup if any answers remain unused. This enables tests to ensure that the expected mock answers are actually used in a test; previously, an entire branch of code including an input sequence could be omitted and the test(s) would not fail. The only test that had unused answers in this map is one leftover from legacy state migrations, a prompt that was removed in `7c93b2e5e6`	5 years ago
Alisdair McDiarmid	b59b057591	json-output: Config-driven move support in JSON UI Add previous address information to the `planned_change` and `resource_drift` messages for the streaming JSON UI output of plan and apply operations. Here we also add a "move" action value to the `change` object of these messages, to represent a move-only operation. As part of this work we also simplify this code to use the plan's DriftedResources values instead of recomputing the drift from state.	5 years ago
Alisdair McDiarmid	78c4a8c461	json-output: Previous address for resource changes Configuration-driven moves are represented in the plan file by setting the resource's `PrevRunAddr` to a different value than its `Addr`. For JSON plan output, we here add a new field to resource changes, `previous_address`, which is present and non-empty only if the resource is planned to be moved. Like the CLI UI, refresh-only plans will include move-only changes in the resource drift JSON output. In normal plan mode, these are elided to avoid redundancy with planned changes.	5 years ago
Martin Atkins	f0034beb33	core: refactoring.ImpliedMoveStatements replaces NodeCountBoundary Going back a long time we've had a special magic behavior which tries to recognize a situation where a module author either added or removed the "count" argument from a resource that already has instances, and to silently rename the zeroth or no-key instance so that we don't plan to destroy and recreate the associated object. Now we have a more general idea of "move statements", and specifically the idea of "implied" move statements which replicates the same heuristic we used to use for this behavior, we can treat this magic renaming rule as just another "move statement", special only in that Terraform generates it automatically rather than it being written out explicitly in the configuration. In return for wiring that in, we can now remove altogether the NodeCountBoundary graph node type and its associated graph transformer, CountBoundaryTransformer. We handle moves as a preprocessing step before building the plan graph, so we no longer need to include any special nodes in the graph to deal with that situation. The test updates here are mainly for the graph builders themselves, to acknowledge that indeed we're no longer inserting the NodeCountBoundary vertices. The vertices that NodeCountBoundary previously depended on now become dependencies of the special "root" vertex, although in many cases here we don't see that explicitly because of the transitive reduction algorithm, which notices when there's already an equivalent indirect dependency chain and removes the redundant edge. We already have plenty of test coverage for these "count boundary" cases in the context tests whose names start with TestContext2Plan_count and TestContext2Apply_resourceCount, all of which continued to pass here without any modification and so are not visible in the diff. The test functions particularly relevant to this situation are: - TestContext2Plan_countIncreaseFromNotSet - TestContext2Plan_countDecreaseToOne - TestContext2Plan_countOneIndex - TestContext2Apply_countDecreaseToOneCorrupted The last of those in particular deals with the situation where we have both a no-key instance _and_ a zero-key instance in the prior state, which is interesting here because to exercises an intentional interaction between refactoring.ImpliedMoveStatements and refactoring.ApplyMoves, where we intentionally generate an implied move statement that produces a collision and then expect ApplyMoves to deal with it in the same way as it would deal with all other collisions, and thus ensure we handle both the explicit and implied collisions in the same way. This does affect some UI-level tests, because a nice side-effect of this new treatment of this old feature is that we can now report explicitly in the UI that we're assigning new addresses to these objects, whereas before we just said nothing and hoped the user would just guess what had happened and why they therefore weren't seeing a diff. The backend/local plan tests actually had a pre-existing bug where they were using a state with a different instance key than the config called for but getting away with it because we'd previously silently fix it up. That's still fixed up, but now done with an explicit mention in the UI and so I made the state consistent with the configuration here so that the tests would be able to recognize _real_ differences where present, as opposed to the errant difference caused by that inconsistency.	5 years ago
Alisdair McDiarmid	638784b195	cli: Omit move-only drift, except for refresh-only The set of drifted resources now includes move-only changes, where the object value is identical but a move has been executed. In normal operation, we previousl displayed these moves twice: once as part of drift output, and once as part of planned changes. As of this commit we omit move-only changes from drift display, except for refresh-only plans. This fixes the redundant output.	5 years ago
Alisdair McDiarmid	9a7bbdab6f	Fix terraform add test failure due to bad merge	5 years ago
Alisdair McDiarmid	d425c26d77	Merge pull request #29589 from hashicorp/alisdair/planfile-drifted-resources core: Compute resource drift during plan phase, store in plan file	5 years ago
Alisdair McDiarmid	1f57b7a8bd	Merge pull request #29235 from magodo/terraform_add_output_append `terraform add`: `-out` option append to existing config & optionally check resource existance	5 years ago
Alisdair McDiarmid	d5b5407ccc	format: Fix incorrect nesting of Color/Sprintf Colorizing the result of an interpolated string can result in incorrect output, if the values used to generate the string happen to include color codes such as `[red]` or `[bold]`. Instead we should always colorize the format string before calling functions like `Sprintf`. This commit fixes all instances in this file.	5 years ago
Alisdair McDiarmid	f0cf4235f9	cli: Refactor resource drift rendering	5 years ago
Martin Atkins	718fa3895f	backend: Remove Operation.Parallelism field The presence of this field was confusing because in practice the local backend doesn't use it for anything and the remote backend was using it only to return an error if it's set to anything other than the default, under the assumption that it would always match ContextOpts.Parallelism. The "command" package is the one actually responsible for handling this option, and it does so by placing it into the partial ContextOpts which it passes into the backend when preparing for a local operation. To make that clearer, here we remove Operation.Parallelism and change the few uses of it to refer to ContextOpts.Parallelism instead, so that everyone is reading and writing this value from the same place.	5 years ago
James Bardin	d0993b0e80	fix temp directory handling in some tests Cleanup some more test fixtures to use t.TempDir Use EvalSymlinks with temp dir paths to help with MacOS errors from various terraform components.	5 years ago
Martin Atkins	65e0c448a0	workdir: Start of a new package for working directory state management Thus far our various interactions with the bits of state we keep associated with a working directory have all been implemented directly inside the "command" package -- often in the huge command.Meta type -- and not managed collectively via a single component. There's too many little codepaths reading and writing from the working directory and data directory to refactor it all in one step, but this is an attempt at a first step towards a future where everything that reads and writes from the current working directory would do so via an object that encapsulates the implementation details and offers a high-level API to read and write all of these session-persistent settings. The design here continues our gradual path towards using a dependency injection style where "package main" is solely responsible for directly interacting with the OS command line, the OS environment, the OS working directory, the stdio streams, and the CLI configuration, and then communicating the resulting information to the rest of Terraform by wiring together objects. It seems likely that eventually we'll have enough wiring code in package main to justify a more explicit organization of that code, but for this commit the new "workdir.Dir" object is just wired directly in place of its predecessors, without any significant change of code organization at that top layer. This first commit focuses on the main files and directories we use to find provider plugins, because a subsequent commit will lightly reorganize the separation of concerns for plugin launching with a similar goal of collecting all of the relevant logic together into one spot.	5 years ago
Martin Atkins	343279110a	core: Graph walk loads plugin schemas opportunistically Previously our graph walker expected to recieve a data structure containing schemas for all of the provider and provisioner plugins used in the configuration and state. That made sense back when terraform.NewContext was responsible for loading all of the schemas before taking any other action, but it no longer has that responsiblity. Instead, we'll now make sure that the "contextPlugins" object reaches all of the locations where we need schema -- many of which already had access to that object anyway -- and then load the needed schemas just in time. The contextPlugins object memoizes schema lookups, so we can safely call it many times with the same provider address or provisioner type name and know that it'll still only load each distinct plugin once per Context object. As of this commit, the Context.Schemas method is now a public interface only and not used by logic in the "terraform" package at all. However, that does leave us in a rather tenuous situation of relying on the fact that all practical users of terraform.Context end up calling "Schemas" at some point in order to verify that we have all of the expected versions of plugins. That's a non-obvious implicit dependency, and so in subsequent commits we'll gradually move all responsibility for verifying plugin versions into the caller of terraform.NewContext, which'll heal a long-standing architectural wart whereby the caller is responsible for installing and locating the plugin executables but not for verifying that what's installed is conforming to the current configuration and dependency lock file.	5 years ago
Martin Atkins	80b3fcf93e	core: Replace contextComponentFactory with contextPlugins In the v0.12 timeframe we made contextComponentFactory an interface with the expectation that we'd write mocks of it for tests, but in practice we ended up just always using the same "basicComponentFactory" implementation throughout. In the interests of simplification then, here we replace that interface and its sole implementation with a new concrete struct type contextPlugins. Along with the general benefit that this removes an unneeded indirection, this also means that we can add additional methods to the struct type without the usual restriction that interface types prefer to be small. In particular, in a future commit I'm planning to add methods for loading provider and provisioner schemas, working with the currently-unused new fields this commit has included in contextPlugins, as compared to its predecessor basicComponentFactory.	5 years ago
Alisdair McDiarmid	cd29c3e5fd	Revert "json-output: Release format version 1.0"	5 years ago
Laura Pacilio	a819d7db3a	Merge pull request #29502 from hashicorp/alisdair/json-format-version json-output: Release format version 1.0	5 years ago
Alisdair McDiarmid	7f3a29b46e	Merge pull request #29523 from hashicorp/alisdair/moved-ui command: Render "moved" annotations in plan UI	5 years ago
James Bardin	1ad0421e15	Merge pull request #29522 from hashicorp/jbardin/json-blocktoattr allow json output to marshal ConfigModeAttr blocks	5 years ago
Alisdair McDiarmid	29999c1d6f	command: Render "moved" annotations in plan UI For resources which are planned to move, render the previous run address as additional information in the plan UI. For the case of a move-only resource (which otherwise is unchanged), we also render that as a planned change, but without any corresponding action symbol. If all changes in the plan are moves without changes, the plan is no longer considered "empty". In this case, we skip rendering the action symbols in the UI.	5 years ago
James Bardin	ac2a870ea0	allow json output to marshal ConfigModeAttr blocks In order to marshal config blocks using ConfigModeAttr, we need to insert the fixup body to map hcl blocks to the attribute in the schema.	5 years ago
Alisdair McDiarmid	b60f201eed	json-output: Release format version 1.0	5 years ago
James Bardin	863963e7a6	de-linting	5 years ago
Martin Atkins	89b05050ec	core: Functional-style API for terraform.Context Previously terraform.Context was built in an unfortunate way where all of the data was provided up front in terraform.NewContext and then mutated directly by subsequent operations. That made the data flow hard to follow, commonly leading to bugs, and also meant that we were forced to take various actions too early in terraform.NewContext, rather than waiting until a more appropriate time during an operation. This (enormous) commit changes terraform.Context so that its fields are broadly just unchanging data about the execution context (current workspace name, available plugins, etc) whereas the main data Terraform works with arrives via individual method arguments and is returned in return values. Specifically, this means that terraform.Context no longer "has-a" config, state, and "planned changes", instead holding on to those only temporarily during an operation. The caller is responsible for propagating the outcome of one step into the next step so that the data flow between operations is actually visible. However, since that's a change to the main entry points in the "terraform" package, this commit also touches every file in the codebase which interacted with those APIs. Most of the noise here is in updating tests to take the same actions using the new API style, but this also affects the main-code callers in the backends and in the command package. My goal here was to refactor without changing observable behavior, but in practice there are a couple externally-visible behavior variations here that seemed okay in service of the broader goal: - The "terraform graph" command is no longer hooked directly into the core graph builders, because that's no longer part of the public API. However, I did include a couple new Context functions whose contract is to produce a UI-oriented graph, and _for now_ those continue to return the physical graph we use for those operations. There's no exported API for generating the "validate" and "eval" graphs, because neither is particularly interesting in its own right, and so "terraform graph" no longer supports those graph types. - terraform.NewContext no longer has the responsibility for collecting all of the provider schemas up front. Instead, we wait until we need them. However, that means that some of our error messages now have a slightly different shape due to unwinding through a differently-shaped call stack. As of this commit we also end up reloading the schemas multiple times in some cases, which is functionally acceptable but likely represents a performance regression. I intend to rework this to use caching, but I'm saving that for a later commit because this one is big enough already. The proximal reason for this change is to resolve the chicken/egg problem whereby there was previously no single point where we could apply "moved" statements to the previous run state before creating a plan. With this change in place, we can now do that as part of Context.Plan, prior to forking the input state into the three separate state artifacts we use during planning. However, this is at least the third project in a row where the previous API design led to piling more functionality into terraform.NewContext and then working around the incorrect order of operations that produces, so I intend that by paying the cost/risk of this large diff now we can in turn reduce the cost/risk of future projects that relate to our main workflow actions.	5 years ago
Alisdair McDiarmid	40dd4292b8	Merge pull request #29438 from hashicorp/alisdair/init-force-copy-multiple-workspaces command: Suppress prompt for init -force-copy	5 years ago
Alisdair McDiarmid	2762a940c0	command: Suppress prompt for init -force-copy The -force-copy flag to init should automatically migrate state. Previously this was not applied to one case: when migrating from a backend with multiple workspaces to another backend supporting multiple workspaces. I believe this was an oversight so this commit fixes that.	5 years ago
Alisdair McDiarmid	70a4f7a6b6	command: Fix stale lock after running add	5 years ago
Martin Atkins	43d753e727	command: "terraform add" is experimental We're aware of several quirks of this command's current design, which result from some existing architectural limitations that we can't address immediately. However, we do still want to make this command available in its current capacity as an incremental improvement, so as a compromise we'll document it as experimental. Our intent here is to exclude it from the Terraform 1.0 Compatibility Promises so that we can have the space to continue to improve the design as other parts of the overall Terraform system gain new capabilities. We don't currently have any concrete plan for this command to be stabilized and subject to compatibility promises. That decision will follow from ongoing discussions with other teams whose systems may need to change in order to support the final design of "terraform add".	5 years ago
James Bardin	68ed50616e	handle null and unknown values in attr diffs The code adopted from block diffs was not set to handle null and unknown values, as those are not allowed for blocks. We also revert the change to formatting nested object types as single attributes, because the attribute formatter cannot handle sensitive values from the schema. This presents some awkward syntax for diffs for now, but should suffice until the entire formatter can be refactored to better handle these new nested types.	5 years ago
Martin Atkins	383bbdeebc	Upgrade to Go 1.17 This includes the addition of the new "//go:build" comment form in addition to the legacy "// +build" notation, as produced by gofmt to ensure consistent behavior between Go versions. The new directives are all equivalent to what was present before, so there's no change in behavior. Go 1.17 continues to use the Unicode 13 tables as in Go 1.16, so this upgrade does not require also upgrading our Unicode-related dependencies. This upgrade includes the following breaking changes which will also appear as breaking changes for Terraform users, but that are consistent with the Terraform v1.0 compatibility promises. - On MacOS, Terraform now requires macOS 10.13 High Sierra or later. This upgrade also includes the following breaking changes which will appear as breaking changes for Terraform users that are inconsistent with our compatibility promises, but have justified exceptions as follows: - cidrsubnet, cidrhost, and cidrnetmask will now reject IPv4 CIDR addresses whose decimal components have leading zeros, where previously they would just silently ignore those leading zeros. This is a security-motivated exception to our compatibility promises, because some external systems interpret zero-prefixed octets as octal numbers rather than decimal, and thus the previous lenient parsing could lead to a different interpretation of the address between systems, and thus potentially allow bypassing policy when configuring firewall rules etc. This upgrade also includes the following breaking changes which could _potentially_ appear as breaking changes for Terraform users, but that do not in practice for the reasons given: - The Go net/url package no longer allows query strings with pairs separated by semicolons instead of ampersands. This primarily affects HTTP servers written in Go, and Terraform includes a special temporary HTTP server as part of its implementation of OAuth for "terraform login", but that server only needs to accept URLs created by Terraform itself and Terraform does not generate any URLs that would be rejected.	5 years ago
James Bardin	a94155d0ca	Merge pull request #29397 from hashicorp/jbardin/format-id-name-marks unmark object ID or Name for formatting	5 years ago
James Bardin	a48b024c0a	unmark object ID or Name for formatting	5 years ago
James Bardin	296a757ab4	check for null sets in diff rendering	5 years ago
James Bardin	fbfb14142e	render empty nested containers as attributes Don't try to break down containers that are empty to render the diff, so we can avoid having to check for empty vs null in all cases.	5 years ago
Alisdair McDiarmid	3b33dc1105	json-output: Add output changes to plan logs Extend the outputs JSON log message to support an `action` field (and make the `type` and `value` fields optional). This allows us to emit a useful output change summary as part of the plan, bringing the JSON log output into parity with the text output. While we do have access to the before/after values in the output changes, attempting to wedge those into a structured log message is not appropriate. That level of detail can be extracted from the JSON plan output from `terraform show -json`.	5 years ago
magodo	90e6a3dffb	modify test case	5 years ago
magodo	67afee693b	target resource in module check only when `-out` points to a module	5 years ago
magodo	dea645797a	`terraform add -out` append to existing config	5 years ago
Kristin Laemmert	0b827ab6b6	format/diff: fix panic with null map in NestedType attrs (#29206 )	5 years ago
Alisdair McDiarmid	a456d030db	Fix flapping JSON output test properly This commit makes the output order of the resource drift messages stable, by building a slice of changes and sorting it by address.	5 years ago
Alisdair McDiarmid	c51112a30c	Fix flapping JSON output test This test would previously fail randomly due to the use of multiple resource instances. Instance keys are iterated over as a map for presentation, which has intentionally inconsistent ordering. To fix this, I changed the test to use different resource addresses for the three drift cases. I also extracted them to a separate test, and tweaked the test helper functions to reduce the number of fatal exit points, to make failed test debugging easier.	5 years ago
Alisdair McDiarmid	5a34825fc1	Merge pull request #29131 from hashicorp/alisdair/sequence-diff-commentary Add comments explaining how ctySequenceDiff works	5 years ago
Alisdair McDiarmid	72a7c95353	Merge pull request #29072 from hashicorp/alisdair/json-ui-resource-drift json-output: Add resource drift to machine readable UI	5 years ago
Alisdair McDiarmid	ef0181cfbd	Add comments explaining how ctySequenceDiff works The logic behind this code took me a while to understand, so I wrote down what I understand to be the reasoning behind how it works. The trickiest part is rendering changing objects as updates. I think the other pieces are fairly common to LCS sequence diff rendering, so I didn't explain those in detail.	5 years ago
Martin Atkins	ab350289ab	addrs: Rename AbsModuleCallOutput to ModuleCallInstanceOutput The previous name didn't fit with the naming scheme for addrs types: The "Abs" prefix typically means that it's an addrs.ModuleInstance combined with whatever type name appears after "Abs", but this is instead a ModuleCallOutput combined with an InstanceKey, albeit structured the other way around for convenience, and so the expected name for this would be the suffix "Instance". We don't have an "Abs" type corresponding with this one because it would represent no additional information than AbsOutputValue.	5 years ago
Alisdair McDiarmid	71a067242d	json-output: Add resource drift to machine readable UI	5 years ago
Kristin Laemmert	35c19d7c9f	command/jsonstate: remove redundant remarking of resource instance (#29049 ) * command/jsonstate: remove redundant remarking of resource instance ResourceInstanceObjectSrc.Decode already handles marking values with any marks stored in ri.Current.AttrSensitivePaths, so re-applying those marks is not necessary. We've gotten reports of panics coming from this line of code, though I have yet to reproduce the panic in a test. * Implement test to reproduce panic on #29042 Co-authored-by: David Alger <davidmalger@gmail.com>	5 years ago
Martin Atkins	70bc432f85	command/views/json: Never generate invalid diagnostic snippet offsets Because our snippet generator is trying to select whole lines to include in the snippet, it has some edge cases for odd situations where the relevant source range starts or ends directly at a newline, which were previously causing this logic to return out-of-bounds offsets into the code snippet string. Although arguably it'd be better for the original diagnostics to report more reasonable source ranges, it's better for us to report a slightly-inaccurate snippet than to crash altogether, and so we'll extend our existing range checks to check both bounds of the string and thus avoid downstreams having to deal with out-of-bounds indices. For completeness here I also added some similar logic to the human-oriented diagnostic formatter, which consumes the result of the JSON diagnostic builder. That's not really needed with the additional checks in the JSON diagnostic builder, but it's nice to reinforce that this code can't panic (in this way, at least) even if its input isn't valid.	5 years ago
James Bardin	c687ebeaf1	Merge pull request #29039 from hashicorp/jbardin/sensitive New marks.Sensitive type, and audit of sensitive marks usage	5 years ago
James Bardin	55ebb2708c	remove IsMarked and ContainsMarked calls Make sure sensitivity checks are looking for specific marks rather than any marks at all.	5 years ago
James Bardin	d9dfd451ea	update to use typed sensitive marks	5 years ago
Kristin Laemmert	096010600d	terraform: use hcl.MergeBodies instead of configs.MergeBodies for pro… (#29000 ) * terraform: use hcl.MergeBodies instead of configs.MergeBodies for provider configuration Previously, Terraform would return an error if the user supplied provider configuration via interactive input iff the configuration provided on the command line was missing any required attributes - even if those attributes were already set in config. That error came from configs.MergeBody, which was designed for overriding valid configuration. It expects that the first ("base") body has all required attributes. However in the case of interactive input for provider configuration, it is perfectly valid if either or both bodies are missing required attributes, as long as the final body has all required attributes. hcl.MergeBodies works very similarly to configs.MergeBodies, with a key difference being that it only checks that all required attributes are present after the two bodies are merged. I've updated the existing test to use interactive input vars and a schema with all required attributes. This test failed before switching from configs.MergeBodies to hcl.MergeBodies. * add a command package test that shows that we can still have providers with dynamic configuration + required + interactive input merging This test failed when buildProviderConfig still used configs.MergeBodies instead of hcl.MergeBodies	5 years ago
Alisdair McDiarmid	3326ab7dae	json-output: Omit unchanged resource_drift entries Previously, if any resources were found to have drifted, the JSON plan output would include a drift entry for every resource in state. This commit aligns the JSON plan output with the CLI UI, and only includes those resources where the old value does not equal the new value---i.e. drift has been detected. Also fixes a bug where the "address" field was missing from the drift output, and adds some test coverage.	5 years ago
Kristin Laemmert	583859e510	commands: `terraform add` (#28874 ) * command: new command, terraform add, generates resource templates terraform add ADDRESS generates a resource configuration template with all required (and optionally optional) attributes set to null. This can optionally also pre-populate nonsesitive attributes with values from an existing resource of the same type in state (sensitive vals will be populated with null and a comment indicating sensitivity) * website: terraform add documentation	5 years ago
Kristin Laemmert	329585d07d	jsonconfig: properly unwind and enumerate references (#28884 ) The "references" included in the expression representation now properly unwrap for each traversal step, to match what was documented.	5 years ago
Kristin Laemmert	ac03d35997	jsonplan and jsonstate: include sensitive_values in state representations (#28889 ) * jsonplan and jsonstate: include sensitive_values in state representations A sensitive_values field has been added to the resource in state and planned values which is a map of all sensitive attributes with the values set to true. It wasn't entirely clear to me if the values in state would suffice, or if we also need to consult the schema - I believe that this is sufficient for state files written since v0.15, and if that's incorrect or insufficient, I'll add in the provider schema check as well. I also updated the documentation, and, since we've considered this before, bumped the FormatVersions for both jsonstate and jsonplan.	5 years ago
James Bardin	09c33fa449	account for noop deposed instances in json plan When rendering a json plan, we need to account for deposed instances that have become a noop rather than a destroy.	5 years ago
Alisdair McDiarmid	24ace6ae7d	Merge pull request #28864 from hashicorp/alisdair/fix-remote-backend-multi-workspace-state-migration Fix remote backend multi workspace state migration	5 years ago
Martin Atkins	1a8da65314	Refactoring of module source addresses and module installation It's been a long while since we gave close attention to the codepaths for module source address parsing and external module package installation. Due to their age, these codepaths often diverged from our modern practices such as representing address types in the addrs package, and encapsulating package installation details only in a particular location. In particular, this refactor makes source address parsing a separate step from module installation, which therefore makes the result of that parsing available to other Terraform subsystems which work with the configuration representation objects. This also presented the opportunity to better encapsulate our use of go-getter into a new package "getmodules" (echoing "getproviders"), which is intended to be the only part of Terraform that directly interacts with go-getter. This is largely just a refactor of the existing functionality into a new code organization, but there is one notable change in behavior here: the source address parsing now happens during configuration loading rather than module installation, which may cause errors about invalid addresses to be returned in different situations than before. That counts as backward compatible because we only promise to remain compatible with configurations that are _valid_, which means that they can be initialized, planned, and applied without any errors. This doesn't introduce any new error cases, and instead just makes a pre-existing error case be detected earlier. Our module registry client is still using its own special module address type from registry/regsrc for now, with a small shim from the new addrs.ModuleSourceRegistry type. Hopefully in a later commit we'll also rework the registry client to work with the new address type, but this commit is already big enough as it is.	5 years ago
Alisdair McDiarmid	3f0c6a2217	cli: Add -ignore-remote-version flag for init When performing state migration to a remote backend target, Terraform may fail due to mismatched remote and local Terraform versions. Here we add the `-ignore-remote-version` flag to allow users to ignore this version check when necessary.	5 years ago
Alisdair McDiarmid	6692336541	cli: Fix state migration version check When migrating multiple local workspaces to a remote backend target using the `prefix` argument, we need to perform the version check against all existing workspaces returned by the `Workspaces` method. Failing to do so will result in a version check error.	5 years ago
Alisdair McDiarmid	953738c128	command/views: Remove unused drift summary message This was dead code, and there is no clear way to retrieve this information, as we currently only derive the drift information as part of the rendering process.	5 years ago
James Bardin	3bf498422c	typo	5 years ago
James Bardin	45b5c289a0	no drift with only deposed changes Changes to deposed instances should not triggers any drift to be rendered, as they will have nothing to display.	5 years ago
James Bardin	57aa7c6025	skip drift rendering for deposed resources Deposed instances have no current state and are only scheduled for deletion, so there is no reason to try and render drift on these instances.	5 years ago
James Bardin	aae642fb07	fix schemas and add deposed test The schemas for provider and the resources didn't match, so the changes were not going to be rendered at all. Add a test which contains a deposed resource.	5 years ago
Kristin Laemmert	649095c602	providers subcommand tests (#28744 ) * getproviders ParsePlatform: add check for invalid platform strings with too many parts The existing logic would not catch things like a platform string containing multiple underscores. I've added an explicit check for exactly 2 parts and some basic tests to prove it. * command/providers-lock: add tests This commit adds some simple tests for the providers lock command. While adding this test I noticed that there was a mis-copied error message, so I replaced that with a more specific message. I also added .terraform.lock.hcl to our gitignore for hopefully obvious reasons. getproviders.ParsePlatform: use parts in place of slice range, since it's available * command: Providers mirror tests The providers mirror command is already well tested in e2e tests, so this includes only the most absolutely basic test case.	5 years ago
Kristin Laemmert	4928e1dd01	terraform: use ProtocolVersion from unmanaged providers' reattachConfig to chose the correct PluginClient (#28190 ) * add/use ProtocolVersion with unmanaged providers reattach config	5 years ago
Alisdair McDiarmid	5fc3ba37a6	cli: Improve sensitivity change warning output When an attribute value changes in sensitivity, we previously rendered this in the diff with a `~` update action and a note about the consequence of the sensitivity change. Since we also suppress the attribute value, this made it impossible to know if the underlying value was changing, too, which has significant consequences on the meaning of the plan. This commit adds an equality check of the old/new underlying values. If these are unchanged, we add a note to the sensitivity warning to clarify that only sensitivity is changing.	5 years ago
Martin Atkins	d35bc05312	Update various non-code references to renamed Go packages Previous commits have moved all of our Go packages under "internal", so this is a retroactive update of non-Go references to those same packages.	5 years ago
Martin Atkins	36d0a50427	Move terraform/ to internal/terraform/ This is part of a general effort to move all of Terraform's non-library package surface under internal in order to reinforce that these are for internal use within Terraform only. If you were previously importing packages under this prefix into an external codebase, you could pin to an earlier release tag as an interim solution until you've make a plan to achieve the same functionality some other way.	5 years ago
Martin Atkins	f40800b3a4	Move states/ to internal/states/ This is part of a general effort to move all of Terraform's non-library package surface under internal in order to reinforce that these are for internal use within Terraform only. If you were previously importing packages under this prefix into an external codebase, you could pin to an earlier release tag as an interim solution until you've make a plan to achieve the same functionality some other way.	5 years ago
Martin Atkins	b40a4fb741	Move plugin/ and plugin6/ to internal/plugin{,6}/ This is part of a general effort to move all of Terraform's non-library package surface under internal in order to reinforce that these are for internal use within Terraform only. If you were previously importing packages under this prefix into an external codebase, you could pin to an earlier release tag as an interim solution until you've make a plan to achieve the same functionality some other way.	5 years ago
Martin Atkins	034e944070	Move plans/ to internal/plans/ This is part of a general effort to move all of Terraform's non-library package surface under internal in order to reinforce that these are for internal use within Terraform only. If you were previously importing packages under this prefix into an external codebase, you could pin to an earlier release tag as an interim solution until you've make a plan to achieve the same functionality some other way.	5 years ago
Martin Atkins	cdd9464f9a	Move lang/ to internal/lang/ This is part of a general effort to move all of Terraform's non-library package surface under internal in order to reinforce that these are for internal use within Terraform only. If you were previously importing packages under this prefix into an external codebase, you could pin to an earlier release tag as an interim solution until you've make a plan to achieve the same functionality some other way.	5 years ago
Martin Atkins	31349a9c3a	Move configs/ to internal/configs/ This is part of a general effort to move all of Terraform's non-library package surface under internal in order to reinforce that these are for internal use within Terraform only. If you were previously importing packages under this prefix into an external codebase, you could pin to an earlier release tag as an interim solution until you've make a plan to achieve the same functionality some other way.	5 years ago
Martin Atkins	ffe056bacb	Move command/ to internal/command/ This is part of a general effort to move all of Terraform's non-library package surface under internal in order to reinforce that these are for internal use within Terraform only. If you were previously importing packages under this prefix into an external codebase, you could pin to an earlier release tag as an interim solution until you've make a plan to achieve the same functionality some other way.	5 years ago

1 2 3 4 5

229 Commits (356cee7b89c99ea1f77ea002bb2249b2950ad331)