97 Commits (dbanck/variable-const-attribute)

Author	SHA1	Message	Date
Sarah French	0dfa115d7d	First step to enable use of TF_REATTACH_PROVIDERS with PSS (#37634) ... * Add test coverage for `parseReattachProviders` * Add unhappy path test cases to parseReattachProviders test * Add `isProviderReattached` function, to help identify when a reattached provider is being used for PSS * Move reattach config-related code into its own package * Make calling code not need to know what the ENV is for reattach providers * Refactor IsProviderReattached to use pre-existing logic * Add test case, make spell check happy with US English * Add headers * Make calling code responsible for reading in the related ENV * Add godoc comment to TF_REATTACH_PROVIDERS const * Update internal/getproviders/reattach/reattach.go	5 months ago
Martin Atkins	48df3121af	go.mod: go get github.com/mattn/go-shellwords@v1.0.12 ... We use this library only for interpreting the "TF_CLI_ARG_..." environment variables as additional command line arguments, so the potential impact of this is very limited. The upstream changes here expand on the supported dynamic behavior around backtick command execution and nested environment variable expansion. We don't use either of those features, but just to make sure I changed the code to force them off (since otherwise another package in the program could change the package's global configuration) and added test cases that will fail if they end up turned on.	2 years ago
Jamie Finnigan	05b0bdac99	adopt hashicorp/cli fork of mitchellh/cli (#34429)	2 years ago
hashicorp-copywrite[bot]	53c34ff49c	Update copyright file headers to BUSL-1.1	3 years ago
Martin Atkins	7bee77bdd3	command: Start of propagating OpenTelemetry context ... Several times over the years we've considered adding tracing instrumentation to Terraform, since even when running in isolation as a CLI program it has a "distributed system-like" structure, with lots of concurrent internal work and also some work delegated to provider plugins that are essentially temporarily-running microservices. However, it's always felt a bit overwhelming to do it because much of Terraform predates the Go context.Context idiom and so it's tough to get a clean chain of context.Context values all the way down the stack without disturbing a lot of existing APIs. This commit aims to just get that process started by establishing how a context can propagate from "package main" into the command package, focusing initially on "terraform init" and some other commands that share some underlying functions with that command. OpenTelemetry has emerged as a de-facto industry standard and so this uses its API directly, without any attempt to hide it behind an abstraction. The OpenTelemetry API is itself already an adapter layer, so we should be able to swap in any backend that uses comparable concepts. For now we just discard the tracing reports by default, and allow users to opt in to delivering traces over OTLP by setting an environment variable when running Terraform (the environment variable was established in an earlier commit, so this commit builds on that.) When tracing collection is enabled, every Terraform CLI run will generate at least one overall span representing the command that was run. Some commands might also create child spans, but most currently do not.	3 years ago
Martin Atkins	a7807dac16	main: Optionally configure an OpenTelemetry OTLP exporter ... Terraform CLI is sometimes used as part of a larger distributed system, in which case it would be helpful to be able to gather telemetry from it as part of the larger request it's being run in response to. We'll now support optionally enabling an OTLP exporter by setting the environment variable OTEL_TRACES_EXPORTER=otlp (a standard OpenTelemetry convention). As of this commit there isn't actually anything emitting traces to the specified collector, but we'll gradually add tracing instrumentation to parts of Terraform CLI and Core in later commits.	3 years ago
hashicorp-copywrite[bot]	325d18262e	[COMPLIANCE] Add Copyright and License Headers	3 years ago
Martin Atkins	fda0579537	Experiments supported only in alpha/dev builds ... We originally introduced the idea of language experiments as a way to get early feedback on not-yet-proven feature ideas, ideally as part of the initial exploration of the solution space rather than only after a solution has become relatively clear. Unfortunately, our tradeoff of making them available in normal releases behind an explicit opt-in in order to make it easier to participate in the feedback process had the unintended side-effect of making it feel okay to use experiments in production and endure the warnings they generate. This in turn has made us reluctant to make use of the experiments feature lest experiments become de-facto production features which we then feel compelled to preserve even though we aren't yet ready to graduate them to stable features. In an attempt to tweak that compromise, here we make the availability of experiments _at all_ a build-time flag which will not be set by default, and therefore experiments will not be available in most release builds. The intent (not yet implemented in this PR) is for our release process to set this flag only when it knows it's building an alpha release or a development snapshot not destined for release at all, which will therefore allow us to still use the alpha releases as a vehicle for giving feedback participants access to a feature (without needing to install a Go toolchain) but will not encourage pretending that these features are production-ready before they graduate from experimental. Only language experiments have an explicit framework for dealing with them which outlives any particular experiment, so most of the changes here are to that generalized mechanism. However, the intent is that non-language experiments, such as experimental CLI commands, would also in future check Meta.AllowExperimentalFeatures and gate the use of those experiments too, so that we can be consistent that experimental features will never be available unless you explicitly choose to use an alpha release or a custom build from source code. Since there are already some experiments active at the time of this commit which were not previously subject to this restriction, we'll pragmatically leave those as exceptions that will remain generally available for now, and so this new approach will apply only to new experiments started in the future. Once those experiments have all concluded, we will be left with no more exceptions unless we explicitly choose to make an exception for some reason we've not imagined yet. It's important that we be able to write tests that rely on experiments either being available or not being available, so here we're using our typical approach of making "package main" deal with the global setting that applies to Terraform CLI executables while making the layers below all support fine-grain selection of this behavior so that tests with different needs can run concurrently without trampling on one another. As a compromise, the integration tests in the terraform package will run with experiments enabled _by default_ since we commonly need to exercise experiments in those tests, but they can selectively opt-out if they need to by overriding the loader setting back to false again.	4 years ago
Martin Atkins	5ac1074c54	main: Report version information for "interesting" dependencies ... We have a few dependencies that are such a significant part of Terraform's behavior that they will often be the root cause of or the solution to a bug reported against Terraform. As a small quality-of-life improvement to help with diagnosing those, we'll now report the selected versions for each of these so-called "interesting" dependencies as part of our initial trace log output during Terraform startup. The goal here is that when someone opens a bug report, and includes the trace log as our bug report template requests, we'll be able to see at a glance which versions of these dependencies were involved, instead of having to manually cross-reference in the go.mod file of the reported main Terraform CLI version. This does slightly grow the general overhead of the logs, but as long as we keep this set of interesting dependencies relatively small it shouldn't present any significant problem in typical usage.	4 years ago
James Bardin	d2d2508f5c	write provider panics to error ... We no longer have to hide these from panicwrap	4 years ago
James Bardin	d03a037567	insert panic handlers	4 years ago
James Bardin	622c4df14c	remove the use of panicwrap ... Stop using panicwrap, and execute terraform in the main process.	4 years ago
Izaak Lauer	231175204d	Small comment typo	5 years ago
Kristin Laemmert	4928e1dd01	terraform: use ProtocolVersion from unmanaged providers' reattachConfig to chose the correct PluginClient (#28190) ... * add/use ProtocolVersion with unmanaged providers reattach config	5 years ago
Martin Atkins	ffe056bacb	Move command/ to internal/command/ ... This is part of a general effort to move all of Terraform's non-library package surface under internal in order to reinforce that these are for internal use within Terraform only. If you were previously importing packages under this prefix into an external codebase, you could pin to an earlier release tag as an interim solution until you've make a plan to achieve the same functionality some other way.	5 years ago
Martin Atkins	b9a93a0fe7	Move addrs/ to internal/addrs/ ... This is part of a general effort to move all of Terraform's non-library package surface under internal in order to reinforce that these are for internal use within Terraform only. If you were previously importing packages under this prefix into an external codebase, you could pin to an earlier release tag as an interim solution until you've make a plan to achieve the same functionality some other way.	5 years ago
Martin Atkins	4c254cc2be	Move httpclient/ to internal/httpclient/ ... This is part of a general effort to move all of Terraform's non-library package surface under internal in order to reinforce that these are for internal use within Terraform only. If you were previously importing packages under this prefix into an external codebase, you could pin to an earlier release tag as an interim solution until you've make a plan to achieve the same functionality some other way.	5 years ago
Martin Atkins	73dda868cc	Move backend/ to internal/backend/ ... This is part of a general effort to move all of Terraform's non-library package surface under internal in order to reinforce that these are for internal use within Terraform only. If you were previously importing packages under this prefix into an external codebase, you could pin to an earlier release tag as an interim solution until you've make a plan to achieve the same functionality some other way.	5 years ago
James Bardin	270ab97418	don't error when processing autocomplete commands ... The shell autocomplete command will use the binary name as the first argument which does not show up under the list of subcommands.	5 years ago
Jonathan Hall	49ee3d3ef8	Grammar nit: "setup" as a verb should be spelled "set up"	5 years ago
Martin Atkins	15c0645bd5	main: initialize the terminal (if any) using internal/terminal ... We need to call into terminal.Init in early startup to make sure that we either have a suitable Terminal or that we disable attempts to use virtual terminal escape sequences. This commit gets the terminal initialized but doesn't do much with it after that. Subsequent commits will make more use of this.	5 years ago
Pam Selle	83e6703bf7	Remove revision from version command ... The revision field is only populated on dev builds so this means most releases of Terraform have an empty "terraform_revision" field in the JSON output. Since we recommend developers use go tooling to `go build` this tool when developing, the revision is not useful data and so it is removed.	5 years ago
James Bardin	0c3bb316ea	redirect warnigns to stdout ... The default cli Warn calls always write to the error writer (stderr by default), however the output is intended to be viewed in the UI by the user, rather than in a separate stream. Terraform also generally does not consider warnings to be errors from the cli point of view, and does not need to output the warning text to stderr. By redirecting Warn calls to Output calls at the lowest level in the main package, we can eliminate the chance that Warn and Output messages are interleaved, while still allowing the internal `cli.Ui` implementations to format `Warn` and `Output` calls separately.	5 years ago
Martin Atkins	28d2cb55fd	main: Special error message for invalid top-level command ... Previously Terraform would react to an invalid top-level command the same way as for typing no command at all: just printing out the long top-level help directory. If someone's tried to type a command, it's more helpful to respond to that request by explaining directly that the command is invalid, rather than leaving the user to puzzle that out themselves by referring to the help text. As a bonus, this also allows us to use our "didyoumean" package to suggest possible alternatives if it seems like the user made a typo.	5 years ago
James Bardin	3225d9ac11	record all plugin panics, and print on main exit ... Create a logger that will record any apparent crash output for later processing. If the cli command returns with a non-zero exit status, check for any recorded crashes and add those to the output.	5 years ago
James Bardin	f8893785f0	separate core and provider loggers ... Now that hclog can independently set levels on related loggers, we can separate the log levels for different subsystems in terraform. This adds the new environment variables, `TF_LOG_CORE` and `TF_LOG_PROVIDER`, which each take the same set of log level arguments, and only applies to logs from that subsystem. This means that setting `TF_LOG_CORE=level` will not show logs from providers, and `TF_LOG_PROVIDER=level` will not show logs from core. The behavior of `TF_LOG` alone does not change. While it is not necessarily needed since the default is to disable logs, there is also a new level argument of `off`, which reflects the associated level in hclog.	5 years ago
James Bardin	c2af5333e8	use a log sink to capture logs for panicwrap ... Use a separate log sink to always capture trace logs for the panicwrap handler to write out in a crash log. This requires creating a log file in the outer process and passing that path to the child process to log to.	5 years ago
James Bardin	fd4f7eb0b9	remove prefixed io ... The main process is now handling what output to print, so it doesn't do any good to try and run it through prefixedio, which is only adding extra coordination to echo the same data.	5 years ago
James Bardin	1d9d82973b	move panicwrap handler to logging package	5 years ago
James Bardin	211edf5d75	use hclog as the default logger ... Inject hclog as the default logger in the main binary.	5 years ago
James Bardin	6ca477f042	move helper/logging to internal ... remove a dead code file too	5 years ago
Martin Atkins	30204ecded	command/cliconfig: Allow development overrides for providers ... For normal provider installation we want to associate each provider with a selected version number and find a suitable package for that version that conforms to the official hashes for that release. Those requirements are very onerous for a provider developer currently testing a not-yet-released build, though. To allow for that case this new CLI configuration feature allows overriding specific providers to refer to give local filesystem directories. Any provider overridden in this way is not subject to the usual restrictions about selected versions or checksum conformance, and activating an override won't cause any changes to the selections recorded in the lock file because it's intended to be a temporary setting for one developer only. This is, in a sense, a spiritual successor of an old capability we had to override specific plugins in the CLI configuration file. There were some vestiges of that left in the main package and CLI config package but nothing has actually been honoring them for several versions now and so this commit removes them to avoid confusion with the new mechanism.	5 years ago
Pam Selle	328baaad84	Fix un-saved error on chdir	5 years ago
Martin Atkins	efe78b2910	main: new global option -chdir ... This new option is intended to address the previous inconsistencies where some older subcommands supported partially changing the target directory (where Terraform would use the new directory inconsistently) where newer commands did not support that override at all. Instead, now Terraform will accept a -chdir command at the start of the command line (before the subcommand) and will interpret it as a request to direct all actions that would normally be taken in the current working directory into the target directory instead. This is similar to options offered by some other similar tools, such as the -C option in "make". The new option is only accepted at the start of the command line (before the subcommand) as a way to reflect that it is a global command (not specific to a particular subcommand) and that it takes effect _before_ executing the subcommand. This also means it'll be forced to appear before any other command-specific arguments that take file paths, which hopefully communicates that those other arguments are interpreted relative to the overridden path. As a measure of pragmatism for existing uses, the path.cwd object in the Terraform language will continue to return the _original_ working directory (ignoring -chdir), in case that is important in some exceptional workflows. The path.root object gives the root module directory, which will always match the overriden working directory unless the user simultaneously uses one of the legacy directory override arguments, which is not a pattern we intend to support in the long run. As a first step down the deprecation path, this commit adjusts the documentation to de-emphasize the inconsistent old command line arguments, including specific guidance on what to use instead for the main three workflow commands, but all of those options remain supported in the same way as they were before. In a later commit we'll make those arguments produce a visible deprecation warning in Terraform's output, and then in an even later commit we'll remove them entirely so that -chdir is the single supported way to run Terraform from a directory other than the one containing the root module configuration.	6 years ago
Alisdair McDiarmid	f1f24df7ff	main: Pass untyped nil for missing creds source ... If we are unable to create a credentials source for some reason, we can rely on the disco object to nil-check it before calling any of its methods. However to do this we must ensure that we pass untyped nil. This commit rearranges the initialization to ensure that this happens. The user-facing bug that triggered this work is that running init when the HOME environment variable is unset would result in a panic on macOS.	6 years ago
Paddy	5127f1ef8b	command: Unmanaged providers ... This adds supports for "unmanaged" providers, or providers with process lifecycles not controlled by Terraform. These providers are assumed to be started before Terraform is launched, and are assumed to shut themselves down after Terraform has finished running. To do this, we must update the go-plugin dependency to v1.3.0, which added support for the "test mode" plugin serving that powers all this. As a side-effect of not needing to manage the process lifecycle anymore, Terraform also no longer needs to worry about the provider's binary, as it won't be used for anything anymore. Because of this, we can disable the init behavior that concerns itself with downloading that provider's binary, checking its version, and otherwise managing the binary. This is all managed on a per-provider basis, so managed providers that Terraform downloads, starts, and stops can be used in the same commands as unmanaged providers. The TF_REATTACH_PROVIDERS environment variable is added, and is a JSON encoding of the provider's address to the information we need to connect to it. This change enables two benefits: first, delve and other debuggers can now be attached to provider server processes, and Terraform can connect. This allows for attaching debuggers to provider processes, which before was difficult to impossible. Second, it allows the SDK test framework to host the provider in the same process as the test driver, while running a production Terraform binary against the provider. This allows for Go's built-in race detector and test coverage tooling to work as expected in provider tests. Unmanaged providers are expected to work in the exact same way as managed providers, with one caveat: Terraform kills provider processes and restarts them once per graph walk, meaning multiple times during most Terraform CLI commands. As unmanaged providers can't be killed by Terraform, and have no visibility into graph walks, unmanaged providers are likely to have differences in how their global mutable state behaves when compared to managed providers. Namely, unmanaged providers are likely to retain global state when managed providers would have reset it. Developers relying on global state should be aware of this.	6 years ago
Martin Atkins	94b87e056b	fixup main.go comment about providersource	6 years ago
Martin Atkins	5af1e6234a	main: Honor explicit provider_installation CLI config when present ... If the CLI configuration contains a provider_installation block then we'll use the source configuration it describes instead of the implied one we'd build otherwise.	6 years ago
Martin Atkins	8c928e8358	main: Consult local directories as potential mirrors of providers ... This restores some of the local search directories we used to include when searching for provider plugins in Terraform 0.12 and earlier. The directory structures we are expecting in these are different than before, so existing directory contents will not be compatible without restructuring, but we need to retain support for these local directories so that users can continue to sideload third-party provider plugins until the explicit, first-class provider mirrors configuration (in CLI config) is implemented, at which point users will be able to override these to whatever directories they want. This also includes some new search directories that are specific to the operating system where Terraform is running, following the documented layout conventions of that platform. In particular, this follows the XDG Base Directory specification on Unix systems, which has been a somewhat-common request to better support "sideloading" of packages via standard Linux distribution package managers and other similar mechanisms. While it isn't strictly necessary to add that now, it seems ideal to do all of the changes to our search directory layout at once so that our documentation about this can cleanly distinguish "0.12 and earlier" vs. "0.13 and later", rather than having to document a complex sequence of smaller changes. Because this behavior is a result of the integration of package main with package command, this behavior is verified using an e2etest rather than a unit test. That test, TestInitProvidersVendored, is also fixed here to create a suitable directory structure for the platform where the test is being run. This fixes TestInitProvidersVendored.	6 years ago
Martin Atkins	e9d0822b2a	command: Accept a "provider source" from the main package ... Following the same approach we use for other CLI-Config-able objects like the service discovery system, the main package is responsible for producing a suitable implementation of this interface which the command package can then use. When unit testing in the command package we can then substitute mocks as necessary, following the dependency inversion principle.	6 years ago
Pam Selle	78b1220558	Remove config.go and update things using its aliases	6 years ago
Radek Simko	32f9722d9d	Replace import paths & set UA string where necessary	6 years ago
Martin Atkins	cfc1c4900d	command/login: Use Cli.Ask to request confirmation ... This is more straightforward than using readline because it already works properly with panicwrap.	7 years ago
Martin Atkins	22a2580e93	main: Use the new cliconfig package credentials source ... This should not cause any change in behavior yet, but using this new implementation will allow the "terraform login" and "terraform logout" commands to store and forget credentials when they are implemented in subsequent commits.	7 years ago
Martin Atkins	e1590d0a70	command/cliconfig: Factor out CLI config handling ... This is just a wholesale move of the CLI configuration types and functions from the main package into its own package, leaving behind some type aliases and wrappers for now to keep existing callers working. This commit alone doesn't really achieve anything, but in future commits we'll expand the functionality in this package.	7 years ago
Matthew Sanabria	be04e70a0f	Adding documentation for TF_CLI_CONFIG_FILE environment variable (#20834) ... Fixes #15849	7 years ago
Sander van Harmelen	52a1b22f7a	Implement the remote enhanced backend ... This is a refactored version of the `remote` backend that was initially added to Terraform v0.11.8 which should now be compatible with v0.12.0.	7 years ago
Martin Atkins	a3403f2766	terraform: Ugly huge change to weave in new State and Plan types ... Due to how often the state and plan types are referenced throughout Terraform, there isn't a great way to switch them out gradually. As a consequence, this huge commit gets us from the old world to a _compilable_ new world, but still has a large number of known test failures due to key functionality being stubbed out. The stubs here are for anything that interacts with providers, since we now need to do the follow-up work to similarly replace the old terraform.ResourceProvider interface with its replacement in the new "providers" package. That work, along with work to fix the remaining failing tests, will follow in subsequent commits. The aim here was to replace all references to terraform.State and its downstream types with states.State, terraform.Plan with plans.Plan, state.State with statemgr.State, and switch to the new implementations of the state and plan file formats. However, due to the number of times those types are used, this also ended up affecting numerous other parts of core such as terraform.Hook, the backend.Backend interface, and most of the CLI commands. Just as with 5861dbf3fc before, I apologize in advance to the person who inevitably just found this huge commit while spelunking through the commit history.	7 years ago
Martin Atkins	bd10b84a8e	command/format: include source snippets in diagnostics ... If we get a diagnostic message that references a source range, and if the source code for the referenced file is available, we'll show a snippet of the source code with the source range highlighted. At the moment we have no cache of source code, so in practice this codepath can never be visited. Callers to format.Diagnostic will be gradually updated in subsequent commits.	7 years ago
Sander van Harmelen	7fb2d1b8de	Implement the Enterprise enhanced remote backend	8 years ago