88 Commits (fa8a87fab58166de33b7ceeb63bb906bc20bfbb2)

Author	SHA1	Message	Date
Krista LaFentres (she/her)	6dcf00aefc	Merge pull request #30344 from hashicorp/lafentres/refactor-show-command ... cli: Refactor show command & migrate to command arguments and views	4 years ago
Krista LaFentres	64e1241ae3	backend/local: Remove unused DisablePlanFileStateLineageChecks flag ... Now that show command has been refactored to remove its dependence on a local backend and local run, this flag is no longer needed to fix #30195.	4 years ago
Martin Atkins	36c4d4c241	core and backend: remove redundant handling of default variable values ... Previously we had three different layers all thinking they were responsible for substituting a default value for an unset root module variable: - the local backend, via logic in backend.ParseVariableValues - the context.Plan function (and other similar functions) trying to preprocess the input variables using terraform.mergeDefaultInputVariableValues . - the newer prepareFinalInputVariableValue, which aims to centralize all of the variable preparation logic so it can be common to both root and child module variables. The second of these was also trying to handle type constraint checking, which is also the responsibility of the central function and not something we need to handle so early. Only the last of these consistently handles both root and child module variables, and so is the one we ought to keep. The others are now redundant and are causing prepareFinalInputVariableValue to get a slightly corrupted view of the caller's chosen variable values. To rectify that, here we remove the two redundant layers altogether and have unset root variables pass through as cty.NilVal all the way to the central prepareFinalInputVariableValue function, which will then handle them in a suitable way which properly respects the "nullable" setting. This commit includes some test changes in the terraform package to make those tests no longer rely on the mergeDefaultInputVariableValues logic we've removed, and to instead explicitly set cty.NilVal for all unset variables to comply with our intended contract for PlanOpts.SetVariables, and similar. (This is so that we can more easily catch bugs in callers where they _don't_ correctly handle input variables; it allows us to distinguish between the caller explicitly marking a variable as unset vs. not describing it at all, where the latter is a bug in the caller.)	4 years ago
Alisdair McDiarmid	768741c0f7	command/show: Disable plan state lineage checks ... When showing a saved plan, we do not need to check the state lineage against current state, because the plan cannot be applied. This is relevant when plan and apply specify a `-state` argument to choose a non-default state file. In this case, the stored prior state in the plan will not match the default state file, so a lineage check will always error.	4 years ago
James Bardin	8c4031ef15	don't persist a nil state from Apply ... Apply should not return a nil state to be persisted.	4 years ago
Chris Arcand	f521ba6cd7	Remove 'enhanced' backend type distinction ... As explained in the changes: The 'enhanced' backend terminology, which only truly pertains to the 'remote' backend with a single API (Terraform Cloud/Enterprise's), has been found to be a confusing vestige which need only be explained in the context of the 'remote' backend. These changes reorient the explanation(s) of backends to pertain more directly to their primary purpose, which is storage of state snapshots (and not implementing operations). That Terraform operations are still _implemented_ by the literal `Backend` and `Enhanced` interfaces is inconsequential a user of Terraform, an internal detail.	4 years ago
Katy Moe	b1c071bb57	pass hclog logger to retryable client	4 years ago
Tom Harvey	a1c7c0dfa4	Update internal/backend/remote-state/azure/backend.go ... Co-authored-by: Tom Bamford <tbamford@hashicorp.com>	4 years ago
tombuildsstuff	ffe9a41468	internal/backend/remote-state/azure: adding tests covering MSAL	4 years ago
tombuildsstuff	a2cad7d579	internal/backend/remote-state/azure: prefixing the existing tests with ADAL	4 years ago
tombuildsstuff	9f710558ce	internal/backend/remote-state/azure: adding a feature-toggle to use MSAL	4 years ago
tombuildsstuff	27a34d6c2c	internal/backend/remote-state/azure: updating the link to the client secret ... This redirects to the registry so we should point there directly	4 years ago
tombuildsstuff	5133d30b71	internal/backend/remote-state/azure: code changes for go-azure-helpers v0.18.0	4 years ago
Jared Ledvina	61a5808680	k8s-backend - Initial context support for library update ... Signed-off-by: Jared Ledvina <jared.ledvina@datadoghq.com>	4 years ago
Chris Arcand	779c958fbf	cloud: Add streamlined 'remote' backend state migration path ... For Terraform Cloud users using the 'remote' backend, the existing 'pattern' prompt should work just fine - but because their workspaces are already present in TFC, the 'migration' here is really just realigning their local workspaces with Terraform Cloud. Instead of forcing users to do the mental gymnastics of what it means to migrate from 'prefix' - and because their remote workspaces probably already exist and already conform to Terraform Cloud's naming concerns - streamline the process for them and calculate the necessary pattern to migrate as-is, without any user intervention necessary.	5 years ago
xiaozhu36	e0a2716601	backend/oss: Removes the ConflictWith tag which on the attributes assume_role* to fix the incompatible error	5 years ago
xiaozhu36	5d19a34f88	backend/oss: Supports the new attribute sts_endpoint	5 years ago
James Bardin	2ebdc099ac	Merge pull request #29307 from hayorov/main ... Flatten `assume_role` block for OSS backend	5 years ago
James Bardin	81e709d185	Merge pull request #29825 from hashicorp/jbardin/no-panicwrap ... Remove the use of panicwrap in the Terraform CLI	5 years ago
Brandon Croft	dbbfae5a1c	refactor ParseVariableValues into separate operations ... 1. ParseDeclaredValues: parses unparsed variables into terraform.InputValues 2. ProbeUndeclaredVariableValues: compares variable declarations with unparsed values to warn/error about undeclared variables	5 years ago
Chris Arcand	1791b71196	cloud: TestCloud_backendWithTags ... Implementing this test was quite a rabbithole, as in order to satisfy backendTestBackendStates() the workspaces returned from backend.Workspaces() must match exactly, and the shortcut taken to test pagination in 3cc58813f0 created an impossible circumstance that got plastered over with the fact that prefix filtering is done clientside, not by the API as it should be. Tagging does not rely on clientside filtering, and expects that the request made to the TFC API returns exactly those workspaces with the given tags. These changes include a better way to test pagination, wherein we actually create over a page worth of valid workspaces in the mock client and implement a simplified pagination behavior to match how the TFC API actually works.	5 years ago
Chris Arcand	471dd479e8	Update go-tfe to 26689e ... These changes include additions to fulfill the interface for the client mock, plus moving all that logic (which needn't be duplicated across both the remote and cloud packages) over to the cloud package under a dedicated mock client file.	5 years ago
Chris Arcand	ea8ad0b15a	Initial commit of 'cloud' package ... The cloud package intends to implement a new integration for Terraform Cloud/Enterprise. The purpose of this integration is to better support TFC users; it will shed some overly generic UX and architecture, behavior changes that are otherwise backwards incompatible in the remote backend, and technical debt - all of which are vestiges from before Terraform Cloud existed. This initial commit is largely a porting of the existing 'remote' backend, which will serve as an underlying implementation detail and not be a typical user-level backend. This is because to re-implement the literal backend interface is orthogonal to the purpose of this integration, and can always be migrated away from later. As this backend is considered an implementation detail, it will not be registered as a declarable backend. Within these changes it is, for easy of initial development and a clean diff.	5 years ago
James Bardin	d03a037567	insert panic handlers	5 years ago
Alex Khaerov	6f1eceb0e2	Revert autoupdate	5 years ago
xiaozhu36	7afaea4cf2	backend/oss: Fixes the nil pointer panic error when missing access key or secret key	5 years ago
Alex Khaerov	14f366dbf4	Update documentation	5 years ago
Alex Khaerov	f32702c5c2	Support deprecated assume_role block	5 years ago
Alex Khaerov	de8810cdd9	Merge branch 'hashicorp:main' into main	5 years ago
Martin Atkins	39779e7021	backend/remote-state/cos: Don't use github.com/likexian/gokit ... We don't use this library anywhere else in Terraform, and this backend was using it only for trivial helpers that are easy to express inline anyway. The new direct code is also type-checkable, whereas these helper functions seem to be written using reflection. This gives us one fewer dependency to worry about and makes the test code for this backend follow a similar assertions style as the rest of this codebase.	5 years ago
James Bardin	ef3c98466d	Merge pull request #29755 from hashicorp/jbardin/first-plan-lineage ... Check for stale plan with no state metadata	5 years ago
James Bardin	5ffa0839f9	only check serial when applying the first plan ... Ensure that we still check for a stale plan even when it was created with no previous state. Create separate errors for incorrect lineage vs incorrect serial. To prevent confusion when applying a first plan multiple times, only report it as a stale plan rather than different lineage.	5 years ago
Martin Atkins	bee7403f3e	command/workspace_delete: Allow deleting a workspace with empty husks ... Previously we would reject attempts to delete a workspace if its state contained any resources at all, even if none of the resources had any resource instance objects associated with it. Nowadays there isn't any situation where the normal Terraform workflow will leave behind resource husks, and so this isn't as problematic as it might've been in the v0.12 era, but nonetheless what we actually care about for this check is whether there might be any remote objects that this state is tracking, and for that it's more precise to look for non-nil resource instance objects, rather than whole resources. This also includes some adjustments to our error messaging to give more information about the problem and to use terminology more consistent with how we currently talk about this situation in our documentation and elsewhere in the UI. We were also using the old State.HasResources method as part of some of our tests. I considered preserving it to avoid changing the behavior of those tests, but the new check seemed close enough to the intent of those tests that it wasn't worth maintaining this method that wouldn't be used in any main code anymore. I've therefore updated those tests to use the new HasResourceInstanceObjects method instead.	5 years ago
Megan Bang	9b9b26a3cd	update error message for invalid json	5 years ago
megan07	7dda3366a6	Update internal/backend/remote-state/gcs/backend.go ... Co-authored-by: appilon <apilon@hashicorp.com>	5 years ago
Megan Bang	81201d69a3	check valid json	5 years ago
Megan Bang	348c761bea	add better error if credentials are invalid	5 years ago
Martin Atkins	df578afd7e	backend/local: Check dependency lock consistency before any operations ... In historical versions of Terraform the responsibility to check this was inside the terraform.NewContext function, along with various other assorted concerns that made that function particularly complicated. More recently, we reduced the responsibility of the "terraform" package only to instantiating particular named plugins, assuming that its caller is responsible for selecting appropriate versions of any providers that _are_ external. However, until this commit we were just assuming that "terraform init" had correctly selected appropriate plugins and recorded them in the lock file, and so nothing was dealing with the problem of ensuring that there haven't been any changes to the lock file or config since the most recent "terraform init" which would cause us to need to re-evaluate those decisions. Part of the game here is to slightly extend the role of the dependency locks object to also carry information about a subset of provider addresses whose lock entries we're intentionally disregarding as part of the various little edge-case features we have for overridding providers: dev_overrides, "unmanaged providers", and the testing overrides in our own unit tests. This is an in-memory-only annotation, never included in the serialized plan files on disk. I had originally intended to create a new package to encapsulate all of this plugin-selection logic, including both the version constraint checking here and also the handling of the provider factory functions, but as an interim step I've just made version constraint consistency checks the responsibility of the backend/local package, which means that we'll always catch problems as part of preparing for local operations, while not imposing these additional checks on commands that _don't_ run local operations, such as "terraform apply" when in remote operations mode.	5 years ago
Martin Atkins	6a98e4720c	plans/planfile: Create takes most arguments via a struct type ... Previously the planfile.Create function had accumulated probably already too many positional arguments, and I'm intending to add another one in a subsequent commit and so this is preparation to make the callsites more readable (subjectively) and make it clearer how we can extend this function's arguments to include further components in a plan file. There's no difference in observable functionality here. This is just passing the same set of arguments in a slightly different way.	5 years ago
Martin Atkins	8d193ad268	core: Simplify and centralize plugin availability checks ... Historically the responsibility for making sure that all of the available providers are of suitable versions and match the appropriate checksums has been split rather inexplicably over multiple different layers, with some of the checks happening as late as creating a terraform.Context. We're gradually iterating towards making that all be handled in one place, but in this step we're just cleaning up some old remnants from the main "terraform" package, which is now no longer responsible for any version or checksum verification and instead just assumes it's been provided with suitable factory functions by its caller. We do still have a pre-check here to make sure that we at least have a factory function for each plugin the configuration seems to depend on, because if we don't do that up front then it ends up getting caught instead deep inside the Terraform runtime, often inside a concurrent graph walk and thus it's not deterministic which codepath will happen to catch it on a particular run. As of this commit, this actually does leave some holes in our checks: the command package is using the dependency lock file to make sure we have exactly the provider packages we expect (exact versions and checksums), which is the most crucial part, but we don't yet have any spot where we make sure that the lock file is consistent with the current configuration, and we are no longer preserving the provider checksums as part of a saved plan. Both of those will come in subsequent commits. While it's unusual to have a series of commits that briefly subtracts functionality and then adds back in equivalent functionality later, the lock file checking is the only part that's crucial for security reasons, with everything else mainly just being to give better feedback when folks seem to be using Terraform incorrectly. The other bits are therefore mostly cosmetic and okay to be absent briefly as we work towards a better design that is clearer about where that responsibility belongs.	5 years ago
James Bardin	8ce2254ad6	Merge pull request #29647 from hashicorp/jbardin/test-temp-cleanup ... temp path clean for some backend tests	5 years ago
Alisdair McDiarmid	57318ef561	backend/remote: Support interop from 0.14 to 1.1 ... The previous conservative guarantee that we would not make backwards incompatible changes to the state file format until at least Terraform 1.1 can now be extended. Terraform 0.14 through 1.1 will be able to interoperably use state files, so we can update the remote backend version compatibility check accordingly.	5 years ago
James Bardin	9c078c27cf	temp path clean for some backend tests	5 years ago
Martin Atkins	f0034beb33	core: refactoring.ImpliedMoveStatements replaces NodeCountBoundary ... Going back a long time we've had a special magic behavior which tries to recognize a situation where a module author either added or removed the "count" argument from a resource that already has instances, and to silently rename the zeroth or no-key instance so that we don't plan to destroy and recreate the associated object. Now we have a more general idea of "move statements", and specifically the idea of "implied" move statements which replicates the same heuristic we used to use for this behavior, we can treat this magic renaming rule as just another "move statement", special only in that Terraform generates it automatically rather than it being written out explicitly in the configuration. In return for wiring that in, we can now remove altogether the NodeCountBoundary graph node type and its associated graph transformer, CountBoundaryTransformer. We handle moves as a preprocessing step before building the plan graph, so we no longer need to include any special nodes in the graph to deal with that situation. The test updates here are mainly for the graph builders themselves, to acknowledge that indeed we're no longer inserting the NodeCountBoundary vertices. The vertices that NodeCountBoundary previously depended on now become dependencies of the special "root" vertex, although in many cases here we don't see that explicitly because of the transitive reduction algorithm, which notices when there's already an equivalent indirect dependency chain and removes the redundant edge. We already have plenty of test coverage for these "count boundary" cases in the context tests whose names start with TestContext2Plan_count and TestContext2Apply_resourceCount, all of which continued to pass here without any modification and so are not visible in the diff. The test functions particularly relevant to this situation are: - TestContext2Plan_countIncreaseFromNotSet - TestContext2Plan_countDecreaseToOne - TestContext2Plan_countOneIndex - TestContext2Apply_countDecreaseToOneCorrupted The last of those in particular deals with the situation where we have both a no-key instance _and_ a zero-key instance in the prior state, which is interesting here because to exercises an intentional interaction between refactoring.ImpliedMoveStatements and refactoring.ApplyMoves, where we intentionally generate an implied move statement that produces a collision and then expect ApplyMoves to deal with it in the same way as it would deal with all other collisions, and thus ensure we handle both the explicit and implied collisions in the same way. This does affect some UI-level tests, because a nice side-effect of this new treatment of this old feature is that we can now report explicitly in the UI that we're assigning new addresses to these objects, whereas before we just said nothing and hoped the user would just guess what had happened and why they therefore weren't seeing a diff. The backend/local plan tests actually had a pre-existing bug where they were using a state with a different instance key than the config called for but getting away with it because we'd previously silently fix it up. That's still fixed up, but now done with an explicit mention in the UI and so I made the state consistent with the configuration here so that the tests would be able to recognize _real_ differences where present, as opposed to the errant difference caused by that inconsistency.	5 years ago
Martin Atkins	332ea1f233	backend/local: TestLocal_plan_context_error to fail terraform.NewContext ... The original intent of this test was to verify that we properly release the state lock if terraform.NewContext fails. This was in response to a bug in an earlier version of Terraform where that wasn't true. In the recent refactoring that made terraform.NewContext no longer responsible for provider constraint/checksum verification, this test began testing a failed plan operation instead, which left the error return path from terraform.NewContext untested. An invalid parallelism value is the one remaining case where terraform.NewContext can return an error, so as a localized fix for this test I've switched it to just intentionally set an invalid parallelism value. This is still not ideal because it's still testing an implementation detail, but I've at least left a comment inline to try to be clearer about what the goal is here so that we can respond in a more appropriate way if future changes cause this test to fail again. In the long run I'd like to move this last remaining check out to be the responsibility of the CLI layer, with terraform.NewContext either just assuming the value correct or panicking when it isn't, but the handling of this CLI option is currently rather awkwardly spread across the command and backend packages so we'll save that refactoring for a later date.	5 years ago
Martin Atkins	718fa3895f	backend: Remove Operation.Parallelism field ... The presence of this field was confusing because in practice the local backend doesn't use it for anything and the remote backend was using it only to return an error if it's set to anything other than the default, under the assumption that it would always match ContextOpts.Parallelism. The "command" package is the one actually responsible for handling this option, and it does so by placing it into the partial ContextOpts which it passes into the backend when preparing for a local operation. To make that clearer, here we remove Operation.Parallelism and change the few uses of it to refer to ContextOpts.Parallelism instead, so that everyone is reading and writing this value from the same place.	5 years ago
James Bardin	d0993b0e80	fix temp directory handling in some tests ... Cleanup some more test fixtures to use t.TempDir Use EvalSymlinks with temp dir paths to help with MacOS errors from various terraform components.	5 years ago
James Bardin	8ed9a270e5	Merge pull request #29559 from hashicorp/jbardin/optional-attrs ... Prevent `ObjectWithOptionalAttrs` from escaping	5 years ago
James Bardin	53a73a8ab6	configs: add ConstraintType to config.Variable ... In order to handle optional attributes, the Variable type needs to keep track of the type constraint for decoding and conversion, as well as the concrete type for creating values and type comparison. Since the Type field is referenced throughout the codebase, and for future refactoring if the handling of optional attributes changes significantly, the constraint is now loaded into an entirely new field called ConstraintType. This prevents types containing ObjectWithOptionalAttrs from escaping the decode/conversion codepaths into the rest of the codebase.	5 years ago
Martin Atkins	343279110a	core: Graph walk loads plugin schemas opportunistically ... Previously our graph walker expected to recieve a data structure containing schemas for all of the provider and provisioner plugins used in the configuration and state. That made sense back when terraform.NewContext was responsible for loading all of the schemas before taking any other action, but it no longer has that responsiblity. Instead, we'll now make sure that the "contextPlugins" object reaches all of the locations where we need schema -- many of which already had access to that object anyway -- and then load the needed schemas just in time. The contextPlugins object memoizes schema lookups, so we can safely call it many times with the same provider address or provisioner type name and know that it'll still only load each distinct plugin once per Context object. As of this commit, the Context.Schemas method is now a public interface only and not used by logic in the "terraform" package at all. However, that does leave us in a rather tenuous situation of relying on the fact that all practical users of terraform.Context end up calling "Schemas" at some point in order to verify that we have all of the expected versions of plugins. That's a non-obvious implicit dependency, and so in subsequent commits we'll gradually move all responsibility for verifying plugin versions into the caller of terraform.NewContext, which'll heal a long-standing architectural wart whereby the caller is responsible for installing and locating the plugin executables but not for verifying that what's installed is conforming to the current configuration and dependency lock file.	5 years ago