127 Commits (3c8a4e6e05afefb7639bf30efa852cf59f14ade6)

Author	SHA1	Message	Date
Martin Atkins	3c8a4e6e05	command+backend/local: -refresh-only and drift detection ... This is a light revamp of our plan output to make use of Terraform core's new ability to report both the previous run state and the refreshed state, allowing us to explicitly report changes made outside of Terraform. Because whether a plan has "changes" or not is no longer such a straightforward matter, this now merges views.Operation.Plan with views.Operation.PlanNoChanges to produce a single function that knows how to report all of the various permutations. This was also an opportunity to fill some holes in our previous logic which caused it to produce some confusing messages, including a new tailored message for when "terraform destroy" detects that nothing needs to be destroyed. This also allows users to request the refresh-only planning mode using a new -refresh-only command line option. In that case, Terraform _only_ performs drift detection, and so applying a refresh-only plan only involves writing a new state snapshot, without changing any real infrastructure objects.	5 years ago
Kristin Laemmert	5f30efe857	command tests: plan and init (#28616) ... * command/init: add test for reconfigure * command/plan: adding tests * command/apply: tests * command: show and refresh tests	5 years ago
Martin Atkins	1d3e34e35e	command: New -replace=... planning option ... This allows a similar effect to pre-tainting an object but does the action within the context of a normal plan and apply, avoiding the need for an intermediate state where the old object still exists but is marked as tainted. The core functionality for this was already present, so this commit is just the UI-level changes to make that option available for use and to explain how it contributed to the resulting plan in Terraform's output.	5 years ago
Alisdair McDiarmid	f72730a02b	cli: Add JSON logs for operations commands	5 years ago
Alisdair McDiarmid	43028c4912	cli: Migrate apply to command views	5 years ago
Alisdair McDiarmid	68558ccd54	backend/local: Replace CLI with view instance ... This commit extracts the remaining UI logic from the local backend, and removes access to the direct CLI output. This is replaced with an instance of a `views.Operation` interface, which codifies the current requirements for the local backend to interact with the user. The exception to this at present is interactivity: approving a plan still depends on the `UIIn` field for the backend. This is out of scope for this commit and can be revisited separately, at which time the `UIOut` field can also be removed. Changes in support of this: - Some instances of direct error output have been replaced with diagnostics, most notably in the emergency state backup handler. This requires reformatting the error messages to allow the diagnostic renderer to line-wrap them; - The "in-automation" logic has moved out of the backend and into the view implementation; - The plan, apply, refresh, and import commands instantiate a view and set it on the `backend.Operation` struct, as these are the only code paths which call the `local.Operation()` method that requires it; - The show command requires the plan rendering code which is now in the views package, so there is a stub implementation of a `views.Show` interface there. Other refactoring work in support of migrating these commands to the common views code structure will come in follow-up PRs, at which point we will be able to remove the UI instances from the unit tests for those commands.	5 years ago
Kristin Laemmert	f6505870cc	Mildwonkey/providers interface renaming (#27805) ... * providers.Interface: huge renamification This commit renames a handful of functions in the providers.Interface to match changes made in protocol v6. The following commit implements this change across the rest of the codebase; I put this in a separate commit for ease of reviewing and will squash these together when merging. One noteworthy detail: protocol v6 removes the config from the ValidateProviderConfigResponse, since it's never been used. I chose to leave that in place in the interface until we deprecate support for protocol v5 entirely. Note that none of these changes impact current providers using protocol v5; the protocol is unchanged. Only the translation layer between the proto and terraform have changed.	5 years ago
Pam Selle	b582b19359	Remove duplicate declaration	5 years ago
Alisdair McDiarmid	6375c6ce6b	Merge pull request #27787 from hashicorp/alisdair/command-views-state-locker ... clistate: Update clistate.Locker for command views	5 years ago
Alisdair McDiarmid	2d1976bbda	clistate: Update clistate.Locker for command views ... The clistate package includes a Locker interface which provides a simple way for the local backend to lock and unlock state, while providing feedback to the user if there is a delay while waiting for the lock. Prior to this commit, the backend was responsible for initializing the Locker, passing through direct access to the cli.Ui instance. This structure prevented commands from implementing different implementations of the state locker UI. In this commit, we: - Move the responsibility of creating the appropriate Locker to the source of the Operation; - Add the ability to set the context for a Locker via a WithContext method; - Replace the Locker's cli.Ui and Colorize members with a StateLocker view; - Implement views.StateLocker for human-readable UI; - Update the Locker interface to return detailed diagnostics instead of errors, reducing its direct interactions with UI; - Add a Timeout() method on Locker to allow the remote backend to continue to misuse the -lock-timeout flag to cancel pending runs. When an Operation is created, the StateLocker field must now be populated with an implementation of Locker. For situations where locking is disabled, this can be a no-op locker. This change has no significant effect on the operation of Terraform, with the exception of slightly different formatting of errors when state locking or unlocking fails.	5 years ago
Alisdair McDiarmid	a7b7cd29fc	cli: Migrate Terraform UI hook to command views ... Move the code which renders Terraform hook callbacks as UI into the views package, backed by a views.View instead of a cli.Ui. Update test setup accordingly. To allow commands to control this hook, we add a hooks member on the backend Operation struct. This supersedes the hooks in the Terraform context, which is not directly controlled by the command logic. This commit should not change how Terraform works, and is refactoring in preparation for more changes which move UI code out of the backend.	5 years ago
Alisdair McDiarmid	c5a6aa31d3	cli: Add initial command views abstraction ... Terraform supports multiple output formats for several sub-commands. The default format is user-readable text, but many sub-commands support a `-json` flag to output a machine-readable format for the result. The output command also supports a `-raw` flag for a simpler, scripting- focused machine readable format. This commit adds a "views" abstraction, intended to help ensure consistency between the various output formats. This extracts the render specific code from the command package, and moves it into a views package. Each command is expected to create an interface for its view, and one or more implementations of that interface. By doing so, we separate the concerns of generating the sub-command result from rendering the result in the specified output format. This should make it easier to ensure that all output formats will be updated together when changes occur in the result-generating phase. There are some other consequences of this restructuring: - Views now directly access the terminal streams, rather than the now-redundant cli.Ui instance; - With the reorganization of commands, parsing CLI arguments is now the responsibility of a separate "arguments" package. For now, views are added only for the output sub-command, as an example. Because this command uses code which is shared with the apply and refresh commands, those are also partially updated.	5 years ago
Alisdair McDiarmid	4991cc4835	cli: Improve error for invalid -target flags ... Errors encountered when parsing flags for apply, plan, and refresh were being suppressed. This resulted in a generic usage error when using an invalid `-target` flag. This commit makes several changes to address this. First, these commands now output the flag parse error before exiting, leaving at least some hint about the error. You can verify this manually with something like: terraform apply -invalid-flag We also change how target attributes are parsed, moving the responsibility from the flags instance to the command. This allows us to customize the diagnostic output to be more user friendly. The diagnostics now look like: ```shellsession $ terraform apply -no-color -target=foo Error: Invalid target "foo" Resource specification must include a resource type and name. ``` Finally, we add test coverage for both parsing of target flags, and at the command level for successful use of resource targeting. These tests focus on the UI output (via the change summary and refresh logs), as the functionality of targeting is covered by the context tests in the terraform package.	5 years ago
Alisdair McDiarmid	d7613a0aac	cli: Better diagnostics for apply positional args ... The previous changes removing support for using the trailing positional argument as a working directory missed a spot in the apply/destroy command implementation. We still support this argument for applying a saved plan: terraform apply foo.tfplan However, if you pass a positional path which doesn't "look like" a plan (for example, the path to a configuration directory), Terraform would silently ignore it and continue. This commit fixes that by adding an error message if the user specifies a path which the plan loader rejects as not "looking like" a plan. This message includes a reference to the `-chdir` flag as a pointer about what to do next. We also rearrange the error message when calling `terraform destroy` with a plan file argument, and add test coverage for the above. While we're here, update the destroy tests to copy the fixture directory, chdir, and defer cleanup.	5 years ago
Alisdair McDiarmid	b2ba650c21	cli: Remove deprecated destroy -force flag ... This dramatically simplifies the logic around auto-approve, which is nice. Also add test coverage for the manual approve step, for both apply and destroy, answering both yes and no.	5 years ago
James Bardin	1e5a8e4dae	fix concurrent test relying on sleep ... Make an old concurrent test deterministic, and not rely on sleep for any synchronization.	5 years ago
Alisdair McDiarmid	ca23a096d8	cli: Remove legacy positional path arguments ... Several commands continued to support the legacy positional path argument to specify a working directory. This functionality has been replaced with the global -chdir flag, which is specified before any other arguments, including the sub-command name. This commit removes support for the trailing path parameter from most commands. The only command which still supports a path argument is fmt, which also supports "-" to indicate receiving configuration from standard input. Any invocation of a command with an invalid trailing path parameter will result in a short error message, pointing at the -chdir alternative. There are many test updates in this commit, almost all of which are migrations from using positional arguments to specify a working directory. Because of the layer at which these tests run, we are unable to use the -chdir argument, so the churn in test files is larger than ideal. Sorry!	5 years ago
James Bardin	3bc7d77230	update MockProvider usage	5 years ago
James Bardin	6f577abc72	unused	5 years ago
James Bardin	e7b2d98ca3	Use prepared config in provider.Configure ... Core is only using the PrepareProviderConfig call for the validation part of the method, but we should be re-validating the final config immediately before Configure. This change elects to not start using the PreparedConfig here, since there is no useful reason for it at this point, and it would introduce a functional difference between terraform releases that can be avoided.	5 years ago
James Bardin	f987b69777	interrupted execution now exits with an error	6 years ago
James Bardin	027a6c2cf6	update command tests to use new mock provider	6 years ago
Alisdair McDiarmid	8b2b569d6e	repl: Improved value renderer for console outputs ... Use a slightly modified value renderer from terraform-provider-testing to display values in the console REPL, as well as outputs from the apply and outputs subcommands. Derived from code in this repository, MIT licensed: https://github.com/apparentlymart/terraform-provider-testing Note that this is technically a breaking change for the console subcommand, which would previously error if the user attempted to render an unknown value (such as an unset variable). This was marked as an unintentional side effect, with the goal being the new behaviour of rendering "(unknown)", which is why I changed the behaviour in this commit.	6 years ago
Kristin Laemmert	6621501ae3	state: remove deprecated state package (#25490) ... Most of the state package has been deprecated by the states package. This PR replaces all the references to the old state package that can be done simply - the low-hanging fruit. * states: move state.Locker to statemgr The state.Locker interface was a wrapper around a statemgr.Full, so moving this was relatively straightforward. * command: remove unnecessary use of state package for writing local terraform state files * move state.LocalState into terraform package state.LocalState is responsible for managing terraform.States, so it made sense (to me) to move it into the terraform package. * slight change of heart: move state.LocalState into clistate instead of terraform	6 years ago
Kristin Laemmert	0af09b23ca	command: apply and most of import tests passing	6 years ago
Martin Atkins	549aede792	Remove terraform.ResourceProvider, use providercache.Installer instead ... Back when we first introduced provider versioning in Terraform 0.10, we did the provider version resolution in terraform.NewContext because we weren't sure yet how exactly our versioning model was going to play out (whether different versions could be selected per provider configuration, for example) and because we were building around the limitations of our existing filesystem-based plugin discovery model. However, the new installer codepath is new able to do all of the selections up front during installation, so we don't need such a heavy inversion of control abstraction to get this done: the command package can select the exact provider versions and pass their factories directly to terraform.NewContext as a simple static map. The result of this commit is that CLI commands other than "init" are now able to consume the local cache directory and selections produced by the installation process in "terraform init", passing all of the selected providers down to the terraform.NewContext function for use in implementing the main operations. This commit is just enough to get the providers passing into the terraform.Context. There's still plenty more to do here, including to repair all of the tests this change has additionally broken.	6 years ago
James Bardin	e13eecbc5b	finish provider ModuleInstance replacement	6 years ago
Kristin Laemmert	47a16b0937	addrs: embed Provider in AbsProviderConfig instead of Type ... a large refactor to addrs.AbsProviderConfig, embedding the addrs.Provider instead of a Type string. I've added and updated tests, added some Legacy functions to support older state formats and shims, and added a normalization step when reading v4 (current) state files (not the added tests under states/statefile/roundtrip which work with both current and legacy-style AbsProviderConfig strings). The remaining 'fixme' and 'todo' comments are mostly going to be addressed in a subsequent PR and involve looking up a given local provider config's FQN. This is fine for now as we are only working with default assumption.	6 years ago
Martin Atkins	8b511524d6	Initial steps towards AbsProviderConfig/LocalProviderConfig separation (#23978) ... * Introduce "Local" terminology for non-absolute provider config addresses In a future change AbsProviderConfig and LocalProviderConfig are going to become two entirely distinct types, rather than Abs embedding Local as written here. This naming change is in preparation for that subsequent work, which will also include introducing a new "ProviderConfig" type that is an interface that AbsProviderConfig and LocalProviderConfig both implement. This is intended to be largely just a naming change to get started, so we can deal with all of the messy renaming. However, this did also require a slight change in modeling where the Resource.DefaultProviderConfig method has become Resource.DefaultProvider returning a Provider address directly, because this method doesn't have enough information to construct a true and accurate LocalProviderConfig -- it would need to refer to the configuration to know what this module is calling the provider it has selected. In order to leave a trail to follow for subsequent work, all of the changes here are intended to ensure that remaining work will become obvious via compile-time errors when all of the following changes happen: - The concept of "legacy" provider addresses is removed from the addrs package, including removing addrs.NewLegacyProvider and addrs.Provider.LegacyString. - addrs.AbsProviderConfig stops having addrs.LocalProviderConfig embedded in it and has an addrs.Provider and a string alias directly instead. - The provider-schema-handling parts of Terraform core are updated to work with addrs.Provider to identify providers, rather than legacy strings. In particular, there are still several codepaths here making legacy provider address assumptions (in order to limit the scope of this change) but I've made sure each one is doing something that relies on at least one of the above changes not having been made yet. * addrs: ProviderConfig interface In a (very) few special situations in the main "terraform" package we need to make runtime decisions about whether a provider config is absolute or local. We currently do that by exploiting the fact that AbsProviderConfig has LocalProviderConfig nested inside of it and so in the local case we can just ignore the wrapping AbsProviderConfig and use the embedded value. In a future change we'll be moving away from that embedding and making these two types distinct in order to represent that mapping between them requires consulting a lookup table in the configuration, and so here we introduce a new interface type ProviderConfig that can represent either AbsProviderConfig or LocalProviderConfig decided dynamically at runtime. This also includes the Config.ResolveAbsProviderAddr method that will eventually be responsible for that local-to-absolute translation, so that callers with access to the configuration can normalize to an addrs.AbsProviderConfig given a non-nil addrs.ProviderConfig. That's currently unused because existing callers are still relying on the simplistic structural transform, but we'll switch them over in a later commit. * rename LocalType to LocalName Co-authored-by: Kristin Laemmert <mildwonkey@users.noreply.github.com>	6 years ago
Kristin Laemmert	6541775ce4	addrs: roll back change to Type field in ProviderConfig (#23937)	6 years ago
Kristin Laemmert	e3416124cc	addrs: replace "Type string" with "Type Provider" in ProviderConfig ... * huge change to weave new addrs.Provider into addrs.ProviderConfig * terraform: do not include an empty string in the returned Providers / Provisioners - Fixed a minor bug where results included an extra empty string	6 years ago
Kristin Laemmert	9891d0354a	providers: use addrs.Provider as map keys for provider.Factory (#23548) ... * terraform/context: use new addrs.Provider as map key in provider factories * added NewLegacyProviderType and LegacyString funcs to make it explicit that these are temporary placeholders This PR introduces a new concept, provider fully-qualified name (FQN), encapsulated by the `addrs.Provider` struct.	6 years ago
Martin Atkins	e21f0fa61e	backend/local: Handle interactive prompts for variables in UI layer ... During the 0.12 work we intended to move all of the variable value collection logic into the UI layer (command package and backend packages) and present them all together as a unified data structure to Terraform Core. However, we didn't quite succeed because the interactive prompts for unset required variables were still being handled _after_ calling into Terraform Core. Here we complete that earlier work by moving the interactive prompts for variables out into the UI layer too, thus allowing us to handle final validation of the variables all together in one place and do so in the UI layer where we have the most context still available about where all of these values are coming from. This allows us to fix a problem where previously disabling input with -input=false on the command line could cause Terraform Core to receive an incomplete set of variable values, and fail with a bad error message. As a consequence of this refactoring, the scope of terraform.Context.Input is now reduced to only gathering provider configuration arguments. Ideally that too would move into the UI layer somehow in a future commit, but that's a problem for another day.	7 years ago
Radek Simko	5b9f2fafc8	Standardise directory name for test data	7 years ago
James Bardin	e71e3d85a9	Merge pull request #21611 from hashicorp/jbardin/private-data-read ... Make sure resource private data is carried through the entire resource lifecycle	7 years ago
James Bardin	ac2219ba6e	don't lose Private state data during copy ... Fix the scope of the private data copy in DeepCopy. Make sure Dependencies matches nil vs empty so that Equal compares correctly between copied states	7 years ago
Ivan Kalita	5b6b1663ef	backend/http: implement retries for the http backend (#19702) ... Fixes #19619	7 years ago
Martin Atkins	0b7179c363	command: Apply tests with realistic mock providers ... Now that we're actually verifying correct behavior of providers during plan and apply, our mock providers need to behave like real providers, properly propagating any configured values through the plan and into the final state. For most of these it was simpler to just switch over to using the newer PlanResourceChangeFn mock interface, away from the legacy DiffFn approach, because then we can just return the ProposedNewState verbatim because our schema for these tests does not require any default values to be populated.	7 years ago
Sander van Harmelen	79a9a15879	command/state: lock when pushing state ... Next to adding the locking for the `state push` command, this commit also fixes a small bug where the lock would not be propertly released when running the `state show` command. And finally it renames some variables in the `[un]taint` code in order to try to standardize the var names of a few frequently used variables (e.g. statemgr.Full, states.State, states.SyncState).	7 years ago
Martin Atkins	592850e22c	command: Fix TestApply_plan_remoteState ... Some over-zealous bulk updating of this test file caused this test to be producing a remote state config cache file on disk when it doesn't actually need one: the backend config comes from the plan file when applying a saved plan.	8 years ago
Sander van Harmelen	b846c5f653	Make the test compile again ... Not sure if these checks still make sense, but without this change the test don’t compile for the related packages.	8 years ago
James Bardin	fe9ed37dfc	minor fixes for command Apply tests	8 years ago
Martin Atkins	73318a436b	command: go fmt	8 years ago
Martin Atkins	741d334ee4	command: Even more fixes for "apply" command tests	8 years ago
Martin Atkins	34a29315f7	command: More fixes (but still not all) for "apply" command tests	8 years ago
Martin Atkins	fc2614c939	command: Fix some tests for the "apply" command	8 years ago
Martin Atkins	b0016e9cf6	command: Allow tests to run to completion without panics or hangs ... There are still 160 test failures as of this commit, but at least the test program can run to completion and list out all the failures.	8 years ago
Martin Atkins	d446d32d96	command: Fix TestApply for new mock provider interface ... It must now provide a basic implementation of plan and apply for its mock provider, which in this case can just pass through the proposed value generated by core because there are no computed attributes in this schema.	8 years ago
Martin Atkins	65341b0973	command: Update apply_test.go for new provider types ... This only makes it compile. It does not yet ensure that the tests pass.	8 years ago
Martin Atkins	a3403f2766	terraform: Ugly huge change to weave in new State and Plan types ... Due to how often the state and plan types are referenced throughout Terraform, there isn't a great way to switch them out gradually. As a consequence, this huge commit gets us from the old world to a _compilable_ new world, but still has a large number of known test failures due to key functionality being stubbed out. The stubs here are for anything that interacts with providers, since we now need to do the follow-up work to similarly replace the old terraform.ResourceProvider interface with its replacement in the new "providers" package. That work, along with work to fix the remaining failing tests, will follow in subsequent commits. The aim here was to replace all references to terraform.State and its downstream types with states.State, terraform.Plan with plans.Plan, state.State with statemgr.State, and switch to the new implementations of the state and plan file formats. However, due to the number of times those types are used, this also ended up affecting numerous other parts of core such as terraform.Hook, the backend.Backend interface, and most of the CLI commands. Just as with 5861dbf3fc before, I apologize in advance to the person who inevitably just found this huge commit while spelunking through the commit history.	8 years ago