terraform

Commit Graph

Author	SHA1	Message	Date
James Bardin	8242c773b8	missed the local state backend	8 years ago
James Bardin	57cbcbe99d	update remote state tests to use new test helpers	8 years ago
James Bardin	112c37458f	report the proper lock id from a state lock error Fix the now failing state unlock test by reporting the correct ID. The ID used by GCS is the generation number of the info object, which isn't known until the info is already written out. While we can't get the correct ID from the info data for the error rmessage, we can update it with the generation number after it's read.	8 years ago
James Bardin	1fee5ae172	update gcs to use the new backend tests	8 years ago
James Bardin	85d6b1d9cc	add test for lock error and force-unlock This adds a general test to verify that a remote state backend returns the expected error type when it cannot lock a state. It then extracts the ID reported in the error, and attempts to unlock the state using that ID, which simulated the force-unlock scenario. This is a separate test, since not all backends have persistent locks that can be unlocked later. We also split out the backend test to be called individually as needed.	8 years ago
James Bardin	ef8ed1e275	coalesce the backened interrupt code Moves the nested select statements for backend operations into a single function. The only difference in this part was that apply called PersistState, which should be harmless regardless of the type of operation being run.	8 years ago
James Bardin	7cba68326a	always wait for a RunningOperation to return If the user wishes to interrupt the running operation, only the first interrupt was communicated to the operation by canceling the provided context. A second interrupt would start the shutdown process, but not communicate this to the running operation. This order of event could cause partial writes of state. What would happen is that once the command returns, the plugin system would stop the provider processes. Once the provider processes dies, all pending Eval operations would return return with an error, and quickly cause the operation to complete. Since the backend code didn't know that the process was shutting down imminently, it would continue by attempting to write out the last known state. Under the right conditions, the process would exit part way through the writing of the state file. Add Stop and Cancel CancelFuncs to the RunningOperation, to allow it to easily differentiate between the two signals. The backend will then be able to detect a shutdown and abort more gracefully. In order to ensure that the backend is not in the process of writing the state out, the command will always attempt to wait for the process to complete after cancellation.	8 years ago
Laura Martin	6e1e614a56	Change -force to -auto-approve when destroying Since an early version of Terraform, the `destroy` command has always had the `-force` flag to allow an auto approval of the interactive prompt. 0.11 introduced `-auto-approve` as default to `false` when using the `apply` command. The `-auto-approve` flag was introduced to reduce ambiguity of it's function, but the `-force` flag was never updated for a destroy. People often use wrappers when automating commands in Terraform, and the inconsistency between `apply` and `destroy` means that additional logic must be added to the wrappers to do similar functions. Both commands are more or less able to run with similar syntax, and also heavily share their code. This commit updates the command in `destroy` to use the `-auto-approve` flag making working with the Terraform CLI a more consistent experience. We leave in `-force` in `destroy` for the time-being and flag it as deprecated to ensure a safe switchover period.	8 years ago
James Bardin	a5ed7d0ae4	Merge pull request #17086 from hashicorp/jbardin/GH-17083 only split the prefix string once from the s3 key	8 years ago
stack72	e9476c6765	backend/manta: Update manta dependencies Internally, triton-go has changed how it handles errors. We can now get rid of checking strings for errors, and we have introduced an errors library that wraps some of the major errors we encounter and test for	8 years ago
Masayuki Morita	f440dba137	Standardize on log level "WARN" rather than "WARNING"	8 years ago
James Bardin	43d000625d	only split the prefix string once from the s3 key Ensure that the prefix is only split off a single time when the workspace_key_prefix is a substring of the workspace or key name.	8 years ago
Stefan Schmidt	c200c170ad	Handle refresh errors.	8 years ago
Paddy	e4cdbd6c9f	Merge pull request #16936 from negz/gcskeys Support 'customer supplied encryption keys' in the GCS backend	8 years ago
James Bardin	7c703f2ab2	Merge pull request #16932 from rv-jmaggio/master Fixing issues with workspace_key_prefix for s3 backend	8 years ago
James Bardin	dd5882e5b2	Merge pull request #16992 from hashicorp/jbardin/go1.10-prep go1.10 prep	8 years ago
goraxe	fb6b349e58	backend/s3: Allow forcing use of the S3 path-style API This is useful when the underlying API is an S3-workalike service, rather than official S3.	8 years ago
Paul Stack	191cf283d5	backend/manta: Support Triton RBAC Triton Manta allows an account other than the main triton account to be used via RBAC. Here we expose the SDC_USER / TRITON_USER options to the backend so that a user can be specified.	8 years ago
James Bardin	4b49a323c3	go fmt slight change to go fmt coming in 0.10	8 years ago
rv-jmaggio	b313ce80c4	Changing prefix for empty workspace prefix	8 years ago
Henry Bell	56357c0ab7	Fix typo ('depracted' -> 'deprecated')	8 years ago
rv-jmaggio	bef64cfe91	Fixing implementation for empty string and making acceptance test work	8 years ago
rv-jmaggio	b02a1c8a46	clarifying tests and using SplitN in implementation	8 years ago
Nic Cope	011841124b	Support 'customer supplied encryption keys' in the GCS backend https://cloud.google.com/storage/docs/encryption#customer-supplied GCS state created using customer supplied encryption keys can only be read or modified using the same key.	8 years ago
rv-jmaggio	7f8d686074	refactor and add a test	8 years ago
rv-jmaggio	0889c118a8	Fixing issues with workspace_key_prefix	8 years ago
Rob Campbell	5daeee5f6d	Update various files for new version of "stringer" The latest version of stringer now uses strconv instead of fmt.	8 years ago
James Bardin	52eced589f	accept a path or contents for credentials Match the operation of the google provider, by accepting either a file path or contents for both `credentials` and `GOOGLE_CREDENTIALS`	8 years ago
James Bardin	2932203492	verify that a state can be read even when locked This should only happen when a state is loaded via the backend, as well as a remote state.	8 years ago
James Bardin	9dea2f78d4	create unique buckets for each test, and clean up This creates a unique bucket name for each test, so that the tests in parallel don't collide, and buckets left over from interrupted tests don't cause future failures. Also make sure that buckets are removed, regardless of content.	8 years ago
James Bardin	aec45e6967	backends should never create unmanaged resources The backend was creating bucket named in the configuration if it didn't exist. We don't allow other backends to do this, because these are not managed resources that terraform can control.	8 years ago
Kaveh Mousavi Zamani	7507e3cd21	backend/gcs: fix locking issue when used with terraform_remote_state Previously there was a problem with double-locking when using the GCS backend with the terraform_remote_state data source. Here we adjust the locking methodology to avoid that problem.	8 years ago
James Bardin	12b7dac124	Merge pull request #16833 from hashicorp/jbardin/plan-shutdown Fully enable shutdown for plan and refresh in the local backend	8 years ago
Sander van Harmelen	aaedf255c3	backend/s3: allow skipping the region check (#16757 ) Without the possibility to skip this check, it’s not possible to use a custom region with a third-party service that mimicks the S3 API.	8 years ago
James Bardin	85295e5c23	watch for cancellation in plan and refresh Cancellation in the local backend was only implemented for apply.	8 years ago
Martin Atkins	ba0514106a	return tfdiags.Diagnostics from validation methods Validation is the best time to return detailed diagnostics to the user since we're much more likely to have source location information, etc than we are in later operations. This change doesn't actually add any detail to the messages yet, but it changes the interface so that we can gradually introduce more detailed diagnostics over time. While here there are some minor adjustments to some of the messages to improve their consistency with terminology we use elsewhere.	9 years ago
James Bardin	00b7715710	Merge pull request #16586 from hashicorp/jbardin/providers Store resolved providers in state	9 years ago
Nic Cope	27ba7de77d	backend/gcs: automatically add trailing slash to prefix The backend doesn't function correctly if a trailing slash is not present, due to how workspaces are enumerated.	9 years ago
James Bardin	09180a10ff	cannot destroy without a config	9 years ago
James Bardin	d62e9217ae	update test state strings for backend/local	9 years ago
Malik Bougacha	ed0d57d1b7	Get the enviroment from the environment variable (#16326 ) * Get the enviroment from the enviroment variable * typo	9 years ago
stack72	1fd0f803e4	Migrate Manta Remote state to be a backend This PR changes manta from being a legacy remote state client to a new backend type. This also includes creating a simple lock within manta This PR also unifies the way the triton client is configured (the schema) and also uses the same env vars to set the backend up It is important to note that if the remote state path does not exist, then the backend will create that path. This means the user doesn't need to fall into a chicken and egg situation of creating the directory in advance before interacting with it	9 years ago
James Bardin	cf54ca3b0f	update tests for new consul packages Reuse the running consul server for all tests. Update the lostLockConnection package, since the api client should no longer lose a lock immediately on network errors.	9 years ago
James Bardin	3a03d3683e	update consul api packages This is from a commit just after the v1.0.0 release, because it removes the Porter service dependency for tests. The client api package was not changed.	9 years ago
Martin Atkins	671aace8ec	backend/local: disable local backup of remote state Previously we forced all remote state backends to be wrapped in a BackupState wrapper that generates a local "terraform.tfstate.backup" file before updating the remote state. This backup mechanism was motivated by allowing users to recover a previous state if user error caused an undesirable change such as loss of the record of one or more resources. However, it also has the downside of flushing a possibly-sensitive state to local disk in a location where users may not realize its purpose and accidentally check it into version control. Those using remote state would generally prefer that state never be flushed to local disk at all. The use-case of recovering older states can be dealt with for remote backends by selecting a backend that has preservation of older versions as a first-class feature, such as S3 versioning or Terraform Enterprise's first-class historical state versioning mechanism. There remains still one case where state can be flushed to local disk: if a write to the remote backend fails during "terraform apply" then we will still create the "errored.tfstate" file to allow the user to recover. This seems like a reasonable compromise because this is done only in an _exceptional_ case, and the console output makes it very clear that this file has been created. Fixes #15339.	9 years ago
James Bardin	55089e472d	Merge pull request #16484 from hashicorp/f-gcloud-backend Convert gcloud backend	9 years ago
Florian Forster	afa13a3d8e	backend/remote-state/gcs: Move toBucketName to the tests.	9 years ago
Florian Forster	3023ae2813	backend/remote-state/gcs: Include project ID in bucket names when testing. Since bucket names must be globally unique. By including the project ID in the bucket name we ensure that people don't step on each other's feet when testing.	9 years ago
Florian Forster	dcb84ee2c2	backend/remote-state/gcs: Sanitize bucket names.	9 years ago
Florian Forster	454d6bbe2a	backend/remote-state/gcs: Delete test buckets after tests complete. This way tests clean up after themselves and don't leak buckets.	9 years ago
Florian Forster	a6669c9e99	backend/remote-state/gcs: Don't enable versioning on new buckets. Enabling versioning without setting up lifecycle management leads to every lock file being archived, slowly accruing useless data.	9 years ago
Florian Forster	a84823f597	backend/remote-state/gcs: Require TF_ACC for tests using the network.	9 years ago
Florian Forster	72ccf22a92	backend/remote-state/gcs: Implement additional tests. This calls backend.TestBackend() and remote.TestRemoteLocks() for standardized acceptance tests. It removes custom listing tests since those are performed by backend.TestBackend(), too. Since each tests uses its own bucket, all tests can be run in parallel.	9 years ago
Florian Forster	927085289d	backend/remote-state/gcs: Implement the "region" config option. This allows to select the region in which a bucket is created. This copies behavior from the Google Cloud provider.	9 years ago
Florian Forster	9583d0945c	backend/remote-state/gcs: Add support for the GOOGLE_PROJECT environment variable. This copies behavior from the Google Cloud provider.	9 years ago
Florian Forster	37dc95158d	backend/remote-state/gcs: Enable versioning on automatically created buckets.	9 years ago
Florian Forster	df386d3133	backend/remote-state/gcs: Automatically create the bucket if needed. This resurrects the previously documented but unused "project" option. This option is required to create buckets (so they are associated with the right cloud project) but not to access the buckets later on (because their names are globally unique).	9 years ago
Florian Forster	14263223e7	backend/remote-state/gcs: Simplify initialization of the GCS client. This also implements the (already documented) behavior of checking the GOOGLE_CREDENTIALS environment variable.	9 years ago
Florian Forster	816c98f387	backend/remote-state/gcs: Read credentials with ioutil.ReadFile(). We never expect the raw JSON to appear in the config, so pathorcontents is not the right package here.	9 years ago
Florian Forster	c00e929ee5	backend/remote-state/gcs: Mark the "path" option as deprecated.	9 years ago
Florian Forster	91b1a39a40	backend/remote-state/gcs: Implement an end-to-end test. The code is loosely based on state/remote/gcs_test.go. If the GOOGLE_PROJECT environment variable is set, this test will 1) create a new bucket; error out if the bucket already exists. 2) create a new state 3) list states and ensure that the newly created state is listed 4) ensure that an object with the expected name exists 5) rum "state/remote".TestClient() 6) delete the state The bucket is deleted when the test exits, though this may fail if the bucket it not empty.	9 years ago
Florian Forster	14bfbf0617	backend/remote-state/gcs: Document the "prefix" option. "state_dir" has been renamed to "prefix" to better fix the GCS terminology.	9 years ago
Florian Forster	c054bd0939	backend/remote-state/gcs: Rename "gcloud" to "gcs" for backwards compatibility.	9 years ago
Florian Forster	93a55f15e9	backend/remote-state/gcloud: Add test for Backend.{state,lock}File().	9 years ago
Florian Forster	5a4e2076e9	backend/remote-state/gcloud: Add the "path" config option. This config option was used by the legacy "gcs" client. If set, we're using it for the default state -- all other states still use the "state_dir" setting.	9 years ago
Florian Forster	f80b872bc3	backend/remote-state/gcloud: Unify on the "context" package. We don't need to use the legacy package here.	9 years ago
Florian Forster	5d4e25ada4	backend/remote-state/gcloud: Make gcsBackend private. This class is only used via the "backend".Backend interface, so there is no need to export this type beyond the gcloud package.	9 years ago
Florian Forster	9ec39573ee	backend/remote-state/gcloud: Make remoteClient private. This class is only used via the "state/remote".State interface, so there is no need to export this type beyond the gcloud package.	9 years ago
Florian Forster	42e8441a2b	backend/remote-state/gcloud: Refactor Backend.State(). Fixes: * https://github.com/golang/go/wiki/CodeReviewComments#doc-comments * https://github.com/golang/go/wiki/CodeReviewComments#error-strings * https://github.com/golang/go/wiki/CodeReviewComments#initialisms	9 years ago
Florian Forster	5cb574035a	backend/remote-state/gcloud: Refactor Backend.remoteClient(). This replaces stateFileName() and lockFileName() with path.Join(). Fixes: * https://github.com/golang/go/wiki/CodeReviewComments#doc-comments * https://github.com/golang/go/wiki/CodeReviewComments#doc-comments	9 years ago
Florian Forster	9ae45e320f	backend/remote-state/gcloud: Refactor Backend.DeleteState(). Fixes: * https://github.com/golang/go/wiki/CodeReviewComments#doc-comments * https://github.com/golang/go/wiki/CodeReviewComments#error-strings	9 years ago
Florian Forster	fabba5c0c8	backend/remote-state/gcloud: Refactor Backend.States(). The previous code listed all objects in the bucket and used local filtering (using regular expressions) to find .tfstate objects. This new code sets the delimiter to "/", which causes GCS to only return objects directly in the given prefix, but not any sub"directories". Fixes: * https://github.com/golang/go/wiki/CodeReviewComments#doc-comments * https://github.com/golang/go/wiki/CodeReviewComments#error-strings	9 years ago
Florian Forster	97e1aa7ce9	backend/remote-state/gcloud: Use the lock file's generation as lock ID. This allows Unlock() to call Delete() without reading the lock file's content first.	9 years ago
Florian Forster	edf2096e28	backend/remote-state/gcloud: Use the context provided to configure(). Calling context.Background() from outside the main() function is discouraged. The configure functions are only called from "…/helper/schema".Backend.Configure which provides the Background context, i.e. a long-living context we can use for backend communication.	9 years ago
Florian Forster	2e5fca78c2	backend/remote-state/gcloud: Coding style changes. Refactorings to make the code more idiomatic.	9 years ago
Florian Forster	f33005faba	backend/remote-state/gcloud: Handle errors returned by Write(), too. Many GCS errors are returned by Close(), but not all.	9 years ago
Florian Forster	1ee194986d	backend/remote-state/gcloud: Add the RemoteClient.{state,lock}File() methods.	9 years ago
Florian Forster	e54d36b489	backend/remote-state/gcloud: Sort standard library imports before other imports.	9 years ago
Florian Forster	52ac764036	backend/remote-state/gcloud: Rename Url -> URL	9 years ago
Florian Forster	5313e15e07	backend/remote-state/gcloud: Move the definition of the Backend struct.	9 years ago
Florian Forster	def3279ec7	backend/remote-state/gcloud: Use package provided OAuth scope.	9 years ago
Piotrek Bzdyl	5854373018	Implemented GCloud backend supporting remote locking and multiple workspaces.	9 years ago
James Bardin	b040cd0837	dynamoDB reads are not fully consisten by default Use fully consistent reads for backend operations.	9 years ago
James Bardin	31912956ce	Merge pull request #16290 from hashicorp/jbardin/s3-errors retry on s3 state upload errors	9 years ago
James Bardin	611b1ced59	retry on s3 state upload errors While #16243 added the ability to retry getting a state from S3, Put can return the same InternalError status. Use the same retry logic when uploading state to S3.	9 years ago
James Bardin	f5e9a20c66	reset testLockHook	9 years ago
James Bardin	25a8227291	add broken test for lock lost on connection error Add a way to inject network errors by setting an immediate deadline on open consul connections. The consul client currently doesn't retry on some errors, and will force us to lose our lock. Once the consul api client is fixed, this test will fail.	9 years ago
James Bardin	fd9adcdb36	only init one consul client, and lower keepalive The consul Client is analogous to an http.Client, and we really don't need more than 1. Configure a single client and store it in the backend. Replace the default Transport's Dialer to reduce the KeepAlive setting from 30s to 17s. This avoids racing with the common network timeout value of 30s, and is also coprime to other common intervals.	9 years ago
James Bardin	d0ecb232ae	record consul session ID in lock info This can help correlate TF and consul logs	9 years ago
James Bardin	b20ab9f0bb	Merge pull request #16243 from hashicorp/jbardin/s3-errors retry on s3 backend internal errors	9 years ago
bclodius	fba02f0bea	retry on s3 backend internal errors Internal errors from S3 are usually transient, and can be immediately retried. Make 2 attempts at retreiving the state object before returning an error.	9 years ago
James Bardin	91442b7146	Merge pull request #15680 from brunomcustodio/etcdv3-backend [WIP] etcd v3 backend with lock support.	9 years ago
James Bardin	d477d1f6d4	Merge pull request #15553 from bonifaido/custom_s3_backend Allow non-AWS S3 backends	9 years ago
James Bardin	e0ee1e4d6e	Merge pull request #16070 from octo/defaultdatadir backend/local: Remove unused const DefaultDataDir.	9 years ago
Martin Atkins	0fe43c8977	cli: allow disabling "next steps" message in terraform plan In #15884 we adjusted the plan output to give an explicit command to run to apply a plan, whereas before this command was just alluded to in the prose. Since releasing that, we've got good feedback that it's confusing to include such instructions when Terraform is running in a workflow automation tool, because such tools usually abstract away exactly what commands are run and require users to take different actions to proceed through the workflow. To accommodate such environments while retaining helpful messages for normal CLI usage, here we introduce a new environment variable TF_IN_AUTOMATION which, when set to a non-empty value, is a hint to Terraform that it isn't being run in an interactive command shell and it should thus tone down the "next steps" messaging. The documentation for this setting is included as part of the "...in automation" guide since it's not generally useful in other cases. We also intentionally disclaim comprehensive support for this since we want to avoid creating an extreme number of "if running in automation..." codepaths that would increase the testing matrix and hurt maintainability. The focus is specifically on the output of the three commands we give in the automation guide, which at present means the following two situations: * "terraform init" does not include the final paragraphs that suggest running "terraform plan" and tell you in what situations you might need to re-run "terraform init". * "terraform plan" does not include the final paragraphs that either warn about not specifying "-out=..." or instruct to run "terraform apply" with the generated plan file.	9 years ago
Florian Forster	ce85600cd4	backend/local: Remove unused const DefaultDataDir. Not to be confused with the const of the same name in the "command" package.	9 years ago
Bruno Miguel Custodio	524c3934c6	Add TLS support.	9 years ago
Bruno Miguel Custodio	c8ff10f603	Reuse 'etcd' client.	9 years ago
Bruno Miguel Custodio	6daf1d9d84	Allow for username and password to be read from envvars.	9 years ago
Bruno Miguel Custodio	8f7b315037	Add documentation.	9 years ago

1 2 3 4 5 ...

334 Commits (b41f08d79d2a2e8dea1fa690c19bb6dff249e141)