60 Commits (ba7679b6791a306a73062ddd5a00359452e152fc)

Author	SHA1	Message	Date
James Bardin	10d94fb764	Merge pull request #21967 from williams-brian/SSE-C_Remote_State ... Add support for SSE-C to S3 backend	7 years ago
Alex Pilon	4bf43efcfd	move hcl2shim package to configs	7 years ago
Brian Williams	5e3c3bafb8	Add support for SSE-C to S3 backend ... These changes add support for encrypting terraform remote-state in S3 using customer-supplied encryption keys (SSE-C).	7 years ago
hasheddan	961d43142c	s3 backend: minor typo in putMD5 comment ... Signed-off-by: hasheddan <georgedanielmangum@gmail.com>	7 years ago
Justin Downing	1e32ae243c	grammatical updates to comments and docs (#20195)	7 years ago
Brian Flad	e34a97abca	backend/s3: Add debug logging and user agent ... Porting over previous behavior from terraform-provider-aws.	7 years ago
James Bardin	dd16455a79	fix s3 default region to match provider	7 years ago
James Bardin	c814f2da37	Change backend.ValidateConfig to PrepareConfig ... This mirrors the change made for providers, so that default values can be inserted into the config by the backend implementation. This is only the interface and method name changes, it does not yet add any default values.	7 years ago
James Bardin	31ca293777	fix slash handling around workspace_key_prefix ... The handling of slashes was broken around listing workspaces in workspace_key_prefix. While it worked in most places by splitting an extra time around the spurious slashes, it failed in the case that the prefix ended with a slash of its own. A test was temporarily added to verify that the backend works with the unusual keys, but rather than risking silent breakage around prefixes with trailing slashes, we also add validation to prevent users from entering keys with trailing slashes at all.	7 years ago
James Bardin	f6af61f990	fix test that never worked ... The 0.12 update left this test broken. Make sure we're working with the correct objects here.	7 years ago
Brian Flad	a41e545198	backend/s3: Reinstate region validation and update copy for skip_requesting_account_id deprecation message	7 years ago
Brian Flad	43f12bbfe0	backend/s3: Update copy for skip_get_ec2_platforms and skip_requesting_account_id argument deprecations	7 years ago
Brian Flad	1aaac172b0	backend/s3: Switch from github.com/terraform-providers/terraform-provider-aws to github.com/hashicorp/aws-sdk-go-base ... Output from acceptance testing (no new failures): ``` --- PASS: TestBackend_impl (0.00s) --- PASS: TestBackendConfig (0.37s) --- PASS: TestBackendConfig_invalidKey (0.00s) --- PASS: TestBackend (3.26s) --- PASS: TestBackendLocked (6.80s) --- FAIL: TestBackendExtraPaths (2.32s) --- PASS: TestBackendPrefixInWorkspace (2.06s) --- PASS: TestKeyEnv (8.20s) --- PASS: TestRemoteClient_impl (0.00s) --- PASS: TestRemoteClient (2.42s) --- PASS: TestRemoteClientLocks (6.33s) --- PASS: TestForceUnlock (13.31s) --- PASS: TestRemoteClient_clientMD5 (11.75s) --- PASS: TestRemoteClient_stateChecksum (10.07s) ```	7 years ago
Brian Flad	ed37d07632	backend/s3: Configure AWS Client MaxRetries and provide enhanced S3 NoSuchBucket error message ... The AWS Go SDK automatically provides a default request retryer with exponential backoff that is invoked via setting `MaxRetries` or leaving it `nil` will default to 3. The terraform-aws-provider `config.Client()` sets `MaxRetries` to 0 unless explicitly configured above 0. Previously, we were not overriding this behavior by setting the configuration and therefore not invoking the default request retryer. The default retryer already handles HTTP error codes above 500, including S3's InternalError response, so the extraneous handling can be removed. This will also start automatically retrying many additional cases, such as temporary networking issues or other retryable AWS service responses. Changes: * s3/backend: Add `max_retries` argument * s3/backend: Enhance S3 NoSuchBucket error to include additional information	7 years ago
Brian Flad	9a3b02cd6c	backend/s3: Support DynamoDB, IAM, and STS endpoint configurations ... This change enables a few related use cases: * AWS has partitions outside Commercial, GovCloud (US), and China, which are the only endpoints automatically handled by the AWS Go SDK. DynamoDB locking and credential verification can not currently be enabled in those regions. * Allows usage of any DynamoDB-compatible API for state locking * Allows usage of any IAM/STS-compatible API for credential verification	7 years ago
Sander van Harmelen	48ef7ecfa6	Updates after running `make fmt` with Go v1.11.1	8 years ago
Martin Atkins	aaf405b662	backend/remote-state: Get all the backend tests building again ... The state manager refactoring in an earlier commit was reflected in the implementations of these backends, but not in their tests. This gets us back to a state where the backend tests will compile, and gets _most_ of them passing again, with a few exceptions that will be addressed in a subsequent commit.	8 years ago
Martin Atkins	a3403f2766	terraform: Ugly huge change to weave in new State and Plan types ... Due to how often the state and plan types are referenced throughout Terraform, there isn't a great way to switch them out gradually. As a consequence, this huge commit gets us from the old world to a _compilable_ new world, but still has a large number of known test failures due to key functionality being stubbed out. The stubs here are for anything that interacts with providers, since we now need to do the follow-up work to similarly replace the old terraform.ResourceProvider interface with its replacement in the new "providers" package. That work, along with work to fix the remaining failing tests, will follow in subsequent commits. The aim here was to replace all references to terraform.State and its downstream types with states.State, terraform.Plan with plans.Plan, state.State with statemgr.State, and switch to the new implementations of the state and plan file formats. However, due to the number of times those types are used, this also ended up affecting numerous other parts of core such as terraform.Hook, the backend.Backend interface, and most of the CLI commands. Just as with 5861dbf3fc before, I apologize in advance to the person who inevitably just found this huge commit while spelunking through the commit history.	8 years ago
Martin Atkins	5782357c28	backend: Update interface and implementations for new config loader ... The new config loader requires some steps to happen in a different order, particularly in regard to knowing the schema in order to decode the configuration. Here we lean directly on the configschema package, rather than on helper/schema.Backend as before, because it's generally sufficient for our needs here and this prepares us for the helper/schema package later moving out into its own repository to seed a "plugin SDK".	8 years ago
Sean Chittenden	d749420a25	Fix drift caused from gofmt when running make dev and go 1.11. ... A fresh checkout of `origin/master` does not build atm using the `dev` target because `master` has not been formatted using `gofmt` from Go 1.11 (tis has been the case for a while if you've been running devel). None of the drift in question is especially new but now that Go 1.11 has been released and gofmt's formatting guidelines have been updated, it would be *really* nice if the code in `master` reflected the current tooling in order to avoid having to fight this drift locally. * 8mo: https://github.com/hashicorp/terraform/blame/master/backend/remote-state/s3/backend_test.go#L260-L261 * 6mo: https://github.com/hashicorp/terraform/blame/master/builtin/provisioners/chef/linux_provisioner_test.go#L124 * 1yr: 7cfeffe36b/command/init.go (L75-L76) * 12d: 7cfeffe36b/command/meta_backend_test.go (L1437) * 2yr: 7cfeffe36b/helper/schema/resource_timeout_test.go (L26) * 4yr: 7cfeffe36b/helper/schema/schema_test.go (L2059) * 1yr: 7cfeffe36b/plugin/discovery/get_test.go (L151)	8 years ago
James Bardin	57cbcbe99d	update remote state tests to use new test helpers	8 years ago
James Bardin	a5ed7d0ae4	Merge pull request #17086 from hashicorp/jbardin/GH-17083 ... only split the prefix string once from the s3 key	8 years ago
Masayuki Morita	f440dba137	Standardize on log level "WARN" rather than "WARNING"	8 years ago
James Bardin	43d000625d	only split the prefix string once from the s3 key ... Ensure that the prefix is only split off a single time when the workspace_key_prefix is a substring of the workspace or key name.	8 years ago
James Bardin	7c703f2ab2	Merge pull request #16932 from rv-jmaggio/master ... Fixing issues with workspace_key_prefix for s3 backend	8 years ago
goraxe	fb6b349e58	backend/s3: Allow forcing use of the S3 path-style API ... This is useful when the underlying API is an S3-workalike service, rather than official S3.	8 years ago
rv-jmaggio	b313ce80c4	Changing prefix for empty workspace prefix	8 years ago
Henry Bell	56357c0ab7	Fix typo ('depracted' -> 'deprecated')	8 years ago
rv-jmaggio	bef64cfe91	Fixing implementation for empty string and making acceptance test work	8 years ago
rv-jmaggio	b02a1c8a46	clarifying tests and using SplitN in implementation	8 years ago
rv-jmaggio	7f8d686074	refactor and add a test	8 years ago
rv-jmaggio	0889c118a8	Fixing issues with workspace_key_prefix	8 years ago
Sander van Harmelen	aaedf255c3	backend/s3: allow skipping the region check (#16757) ... Without the possibility to skip this check, it’s not possible to use a custom region with a third-party service that mimicks the S3 API.	8 years ago
James Bardin	b040cd0837	dynamoDB reads are not fully consisten by default ... Use fully consistent reads for backend operations.	9 years ago
James Bardin	611b1ced59	retry on s3 state upload errors ... While #16243 added the ability to retry getting a state from S3, Put can return the same InternalError status. Use the same retry logic when uploading state to S3.	9 years ago
James Bardin	b20ab9f0bb	Merge pull request #16243 from hashicorp/jbardin/s3-errors ... retry on s3 backend internal errors	9 years ago
bclodius	fba02f0bea	retry on s3 backend internal errors ... Internal errors from S3 are usually transient, and can be immediately retried. Make 2 attempts at retreiving the state object before returning an error.	9 years ago
James Bardin	d477d1f6d4	Merge pull request #15553 from bonifaido/custom_s3_backend ... Allow non-AWS S3 backends	9 years ago
James Bardin	4c7cd549cd	don't allow leading slashes in s3 remote state key ... S3 accepts objects with a leading slash and strips them off. This works fine except in our workspace hierarchy, which then can no longer find suffixes matching the full key name.	9 years ago
Nandor Kracser	f6c77339f2	Allow non-AWS S3 backends ... This commit makes sts, metadata and other AWS related API calls optional, thus the backend initialization will not send non-AWS API tokens to AWS APIs	9 years ago
James Bardin	42cbb716b7	Merge pull request #15370 from rowleyaj/s3_backend_workspaces ... Add configurable workspace prefix for S3 Backend	9 years ago
Alex Rowley	2dd464cf8f	Clarify field description	9 years ago
James Bardin	1fa7667ad4	s3 backend should use Client.Delete to DeleteState ... The s3.Backend was using it's own code for DeleteState, but the dynamo entries are only handled through the RemoteClient. Have DeleteState use a RemoteClient for delete.	9 years ago
Alex Rowley	db40dc06ab	Add configurable workspace prefix for S3 Backend ... Fixes #13184	9 years ago
James Bardin	ac959af6b4	rename aws provider import in s3 backend	9 years ago
James Bardin	6b700ff1fb	replace lock_table with dynamodb_table in s3 cfg ... Since the DynamoDB table used by the S3 backend is no longer only used for locks, rename it in the config to remove any confusion about it being lock-specific.	9 years ago
James Bardin	e7502454b4	check for nil state in s3 client Get ... The S3 client can return (nil, nil) when the remote state doesn't exist. The caused a nil pointer dereference when checking the payload.MD5 against the expected value. This can happen if the remote state was manually removed, but the digest entry was left in the DynamoDB table.	9 years ago
James Bardin	ef1d53934c	Merge pull request #14746 from hashicorp/jbardin/s3-consistency ... store and verify s3 remote state checksum to avoid consistency issues.	9 years ago
James Bardin	91be40a577	add detailed error message s3 checksum mismatch ... Have the s3 RemoteClient return a detailed error message to the user in the case of a mismatch state checksum.	9 years ago
James Bardin	0022d224e8	store and verify s3 remote state checksum ... Updates to objects in S3 are only eventually consistent. If the RemoteClient has a DynamoDB table available, use that to store a checksum of the last written state, so the object can be verified by the next client to call Get. Terraform currently doesn't have any sort of user feedback around RefreshState/Get, so we poll only for a short time before returning an error.	9 years ago