213 Commits (1e2da4f776bdf74058154ed31c4cd62b052d3f2d)

Author	SHA1	Message	Date
Pam Selle	9631e4c73d	Merge pull request #20571 from sergkondr/fix_misspelling ... fix misspelling	7 years ago
Martin Atkins	85eda8a059	state/remote: Don't persist snapshot for unchanged state ... Previously we would write to the backend for every call to PersistState, even if nothing changed since the last write, but update the serial only if the state had changed. The Terraform Cloud & Enterprise state storage have a simple safety check that any future write with an already-used lineage and serial must be byte-for-byte identical. StatesMarshalEqual is intended to detect that, but it only actually detects changes the state itself, and not changes to the snapshot metadata. Because we write the current Terraform version into the snapshot metadata during serialization, we'd previously have an issue where if the first state write after upgrading Terraform to a new version happened to change nothing about the state content then we'd write a new snapshot that differed only by Terraform version, and Terraform Cloud/Enterprise would then reject it. The snapshot header is discarded immediately after decoding, so we can't use information from it when deciding whether to increment the serial. The next best thing is to skip sending no-op snapshot updates to the state client in the first place. These writes are unnecessary anyway, and state storage owners have asked us in the past to elide these to avoid generating noise in their version logs, so we'll also finally meet those requests as a nice side-effect of this change. We didn't previously have tests for the full flow of retrieving and then successively updating persisted state snapshots, so this includes a test which covers that logic and includes an assertion that a no-op update does not get written to the state client.	7 years ago
Martin Atkins	2124089e14	state/remote: Switch to statemgr interfaces in test ... These statemgr interfaces are the new names for the older interfaces in the "state" package. These types alias each other so it doesn't really matter which we use, but the "state" package is deprecated and we intend to eventually remove it, so this is a further step in that direction.	7 years ago
Sergey Kondrashov	43e7a7b552	fix misspelling	7 years ago
James Bardin	43b2c9a1d0	remove single rand source to prevent races ... This shouldn't really be an issue in normal usage, but some of the backend tests will trigger a race here.	7 years ago
Martin Atkins	94510bc1b9	states/statemgr: Migrate, Import, and Export functions ... In our recent refactoring of the state manager interfaces we made serial and lineage management the responsibility of the state managers themselves, not exposing them at all to most callers, and allowing for simple state managers that don't implement them at all. However, we do have some specific cases where we need to preserve these properly when available, such as migration between backends, and the "terraform state push" and "terraform state pull" commands. These new functions and their associated optional interface allow the logic here to be captured in one place and access via some simple calls. Separating this from the main interface leaves things simple for the normal uses of state managers. Since these functions are mostly just thin wrappers around other functionality, they are not yet well-tested directly, but will be indirectly tested through the tests of their callers. A subsequent commit will add more unit tests here.	7 years ago
Sander van Harmelen	a7b8cc8fe3	Do not clear out a previous set state when refreshing ... In the case when no state existed remotely and a new one was created locally, we don’t want to blow away the new local state when refreshing.	7 years ago
Sander van Harmelen	af1a471a05	command/state: update and fix the state list command	7 years ago
Martin Atkins	72a7947cb7	state/remote: Un-stub TestStateRace ... The underlying bug that was causing this test to hang has since been fixed.	7 years ago
Martin Atkins	78f1d1d1c0	state/remote: Don't hang in PersistState ... We were calling from PersistState into RefreshState, but RefreshState is protected by the same lock as PersistState and so the call would deadlock. Instead, we introduce a new entry point refreshState which can be used when already holding the lock.	7 years ago
Martin Atkins	477ea0d360	state/remote: Make tests compile and run to completion ... One of the tests was hanging, so for now it's stubbed out until we can get to the bottom of the hang on a subsequent pass.	7 years ago
Martin Atkins	4bdaffb586	state: Remove tests for obsolete components ... Most of the functionality in this package has been obsoleted by equivalent symbols in states/statemgr. We're keeping this package around for the moment mainly just to house the type aliases in state.go. The tests in here no longer work because they are written against the old APIs. We will eventually remove the components these are testing too, but we're keeping them around for the moment so that we don't break the build for some leftover callers that are still depending on these.	7 years ago
Martin Atkins	aaf405b662	backend/remote-state: Get all the backend tests building again ... The state manager refactoring in an earlier commit was reflected in the implementations of these backends, but not in their tests. This gets us back to a state where the backend tests will compile, and gets _most_ of them passing again, with a few exceptions that will be addressed in a subsequent commit.	7 years ago
Martin Atkins	a3403f2766	terraform: Ugly huge change to weave in new State and Plan types ... Due to how often the state and plan types are referenced throughout Terraform, there isn't a great way to switch them out gradually. As a consequence, this huge commit gets us from the old world to a _compilable_ new world, but still has a large number of known test failures due to key functionality being stubbed out. The stubs here are for anything that interacts with providers, since we now need to do the follow-up work to similarly replace the old terraform.ResourceProvider interface with its replacement in the new "providers" package. That work, along with work to fix the remaining failing tests, will follow in subsequent commits. The aim here was to replace all references to terraform.State and its downstream types with states.State, terraform.Plan with plans.Plan, state.State with statemgr.State, and switch to the new implementations of the state and plan file formats. However, due to the number of times those types are used, this also ended up affecting numerous other parts of core such as terraform.Hook, the backend.Backend interface, and most of the CLI commands. Just as with 5861dbf3fc before, I apologize in advance to the person who inevitably just found this huge commit while spelunking through the commit history.	7 years ago
James Bardin	979faa5dbe	move artifactory remote state to backend	7 years ago
James Bardin	178eb6076e	remove legacy remote file code ... This code is not referenced at all	7 years ago
James Bardin	18ef072325	move legacy http remote state to a backend	7 years ago
James Bardin	3595bd3f81	remove dead legacy gcs code	7 years ago
James Bardin	fe527ec9d7	move legacy etcd remote state to a backend	7 years ago
Sander van Harmelen	7fb2d1b8de	Implement the Enterprise enhanced remote backend	8 years ago
James Bardin	9bf9280c03	Merge pull request #17431 from oscr/use-seekstart ... Use io.SeekStart instead of deprecated os.SEEK_SET	8 years ago
James Bardin	a84b4a669a	use the open state file for refresh when possible ... Only open a new file descriptor for RefreshState if we haven't written a state, and don't have the correct state open already. This prevents windows from failing to refresh a locked state.	8 years ago
James Bardin	e10a7917e6	add failing test for windows state locks ... Refreshing a locked state on windows could return nil if the read path was locked, no state was yet written, and the read path is the same as the write path. Add a test that locks then refreshes a newly initialized state struct.	8 years ago
Paul Tyng	c868092d2d	Standardize http.Client creation with User-Agent	8 years ago
Oscar Utbult	7434cd0a81	Use io.SeekStart instead of deprecated os.SEEK_SET	8 years ago
jwa	3d4642719f	simulate a webdav backend	8 years ago
jwa	b3d432e6bc	allow HTTP 201 & 204 when storing remote state ... Apache mod_dav returns 201 (created) and 204 (no content) during PUTs; treat these as valid responses when using Apache as a http backend.	8 years ago
stack72	1fd0f803e4	Migrate Manta Remote state to be a backend ... This PR changes manta from being a legacy remote state client to a new backend type. This also includes creating a simple lock within manta This PR also unifies the way the triton client is configured (the schema) and also uses the same env vars to set the backend up It is important to note that if the remote state path does not exist, then the backend will create that path. This means the user doesn't need to fall into a chicken and egg situation of creating the directory in advance before interacting with it	8 years ago
Florian Forster	b09f121f86	state/remote: The "gcs" client has been superseeded by the "gcs" backend.	8 years ago
James Bardin	36b8be43e8	use the new version package ... Update all references to the version values to use the new package. The VersionString function was left in the terraform package specifically for the aws provider, which is vendored. We can remove that last call once the provider is updated.	8 years ago
Martin Atkins	c12d64f340	Use t.Helper() in our test helpers ... Go 1.9 adds this new function which, when called, marks the caller as being a "helper function". Helper function stack frames are then skipped when trying to find a line of test code to blame for a test failure, so that the code in the main test function appears in the test failure output rather than a line within the helper function itself. This covers many -- but probaly not all -- of our test helpers across various packages.	9 years ago
Martin Atkins	2e0c1d07ae	Merge #15793: Locking support in HTTP remote backend	9 years ago
Ross McFarland	d889ac38b0	Change remote/http store to update, more consistent with doc	9 years ago
Ross McFarland	510563b67f	Fully test conf handling in httpFactory	9 years ago
Ross McFarland	69546c4b33	Pass at much more flexible remote/http backend ... - Configurable Put (store) method, default POST to preserve behavior - Configurable Lock method & address - Configurable Unlock method & address More thorough testing still needed, but this if functional	9 years ago
James Bardin	68da0390b7	remove legacy azure remote state code ... The implementation has been moved to a backend.	9 years ago
Peter McAtominey	f9e8e54835	backend: convert Azure remote state to a backend ... Added locking support via blob leasing (requires that an empty state is created before any lock can be acquired. Added support for "environments" in much the same way as the S3 backend.	9 years ago
Ross McFarland	6cdea5af5d	Clean up code and make the state/http behavior more consistent	9 years ago
Ross McFarland	ce4d9fb3c2	Add tests for state/http with locking	9 years ago
Ross McFarland	1d38569c91	Add Lock/Unlock support to remote/http	9 years ago
James Bardin	32ae05c342	fix strict remote.State lineage check ... We can't check lineage in the remote state instance, because we may need to overwrite a state with a new lineage. Whil it's tempting to add an optional interface for this, like OverwriteState(), optional interfaces are never _really_ optional, and will have to be implemented by any wrapper types as well. Another solution may be to add a State.Supersedes field to indicate that we intend to replace an existing state, but that may not be worth the extra check either.	9 years ago
James Bardin	fba5decae5	update TestState helper ... In practice, States must all implement the full interface, so checking for each method set only leaves gaps where tests could be skipped. Change the helper to only accept a full state.State implementation. Add some Lineage, Version, and TFVersion checks to TestState to avoid regressions. Compare the copy test against the immediate State returnedm rather than our previous "current" state. Check that the states round-trip and still marhsal identically via MarshalEqual.	9 years ago
Martin Atkins	4d53eaa6df	state: more robust handling of state Serial ... Previously we relied on a constellation of coincidences for everything to work out correctly with state serials. In particular, callers needed to be very careful about mutating states (or not) because many different bits of code shared pointers to the same objects. Here we move to a model where all of the state managers always use distinct instances of state, copied when WriteState is called. This means that they are truly a snapshot of the state as it was at that call, even if the caller goes on mutating the state that was passed. We also adjust the handling of serials so that the state managers ignore any serials in incoming states and instead just treat each Persist as the next version after what was most recently Refreshed. (An exception exists for when nothing has been refreshed, e.g. because we are writing a state to a location for the first time. In that case we _do_ trust the caller, since the given state is either a new state or it's a copy of something we're migrating from elsewhere with its state and lineage intact.) The intent here is to allow the rest of Terraform to not worry about serials and state identity, and instead just treat the state as a mutable structure. We'll just snapshot it occasionally, when WriteState is called, and deal with serials _only_ at persist time. This is intended as a more robust version of #15423, which was a quick hotfix to an issue that resulted from our previous slopping handling of state serials but arguably makes the problem worse by depending on an additional coincidental behavior of the local backend's apply implementation.	9 years ago
Gavin Williams	96e6bf89ec	remote/swift: Remove un-used code	9 years ago
Gavin Williams	ee2e390f85	remote/swift: Migrate Swift remote state to a backend ... Move the Swift State from a legacy remote state to an official backend. Add `container` and `archive_container` configuration variables, and deprecate `path` and `archive_path` variables. Future improvements: Add support for locking and environments.	9 years ago
James Bardin	808b504bcf	rename openstack provider for swift remote state	9 years ago
Tom Harvey	13583b4b8b	provider/azurerm: Upgrading to AutoRest v8 (#15006) ... * Upgrading to AutoRest 8 * Upgrading to AutoRest v8 * Updating the Remote State to v8	9 years ago
Tom Harvey	df2875045d	provider/azurerm: Upgrading to v10 of the Azure SDK (#14004) ... provider/azurerm: Upgrading to v10.0.2 of the Azure SDK	9 years ago
James Bardin	4866f35645	add mutexes to Local, Backup, and InmemState	9 years ago
James Bardin	f0f2220abb	add mutexes to remote.State	9 years ago