boundary

Commit Graph

Author	SHA1	Message	Date
Irena Rindos	9733a250fa	refact(event): move observability/event to event (#3704 )	3 years ago
Jim	a16033b73a	fix (kms): move oplog DEK into its own internal kms (#3665 ) * fix (auth/ldap): fix which DEK is used for oplog during authenticate * chore: update go-kms-wrapping dep for new extras/kms changes The latest version supports setting a table name prefix for a kms which is needed to fix our oplog DEK problem * chore: update extras/kms dep This latest version supports an optional TableNamePrefix which is required to fix our existing oplog DEK issue. * fix (oplog): schema oplog DEK fix These changes will allow use to store the oplog DEK in it's own set of tables, so they won't get deleted when a scope is deleted * fix (kms): move oplog DEK into its own internal kms * tests: update tests to be compatible with oplog fix * tests: remove migration tests which are incompatible with oplog fix These tests no longer work because they are missing the required schema changes for the oplog fix	3 years ago
Jim	d5411a26f6	chore: update protobuf dep (#3669 )	3 years ago
Lars Lehtonen	ee6803d106	internal/server: fix dropped testing error (#3015 )	3 years ago
hashicorp-copywrite[bot]	29da0bcb92	[COMPLIANCE] License changes (#3567 ) * Adding explicit MPL license for sub-package. This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository. * Adding explicit MPL license for sub-package This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository. * Updating the license from MPL to Business Source License Going forward, this project will be licensed under the Business Source License v1.1. Please see our blog post for more details at https://hashi.co/bsl-blog, FAQ at www.hashicorp.com/licensing-faq, and details of the license at www.hashicorp.com/bsl. * Update copyright file headers to BUS-1.1 * Rerun make gen This will pick up the last of the license changes * Revert "Rerun make gen" This reverts commit `dbe0968bff`. * Remove copywrite from generation script The current version of copywrite is not sophisticated enough to only apply the copywrite changes to certain subdirectories, so to avoid accidentally relicensing our api and sdk modules, we will stop running the copywrite script until such time that it can do the right thing. * License protobuf API as MPL The protobuf API files are used to generate the SDK, since we want the SDK to be MPL, the protobuf files must also be MPL. This extends to all protobuf files whose sources end up in the SDK directory, including: - internal/proto/controller/api - internal/proto/controller/custom_options - internal/proto/plugin It also includes the generated service directory in internal/gen/controller/api/services, since they are generated from the protobuf sources used for the SDK. * Change website license file name This makes the naming scheme consistent with the rest of the repository. * Add note on licensing to README * Set copywrite license to BUSL * Revert "Add note on licensing to README" This reverts commit `4d865a91a7`. --------- Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com> Co-authored-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>	3 years ago
Johan Brandhorst-Satzkorn	c644396916	server: allow release version to be set in test (#3517 ) The old logic made it impossible for the caller to set the release version with WithReleaseVersion. Only set the release version when it's not explicitly set already.	3 years ago
Irena Rindos	0fcf0e21d9	chore(server): Update worker auth repository to handle dupe nodes (#3410 ) * chore(server): Update worker auth repository to use nodee DuplicateRecordError	3 years ago
Irena Rindos	a8e363888a	bug(workerAuth): allow duplicate workerAuth inserts if records match (#3389 ) * bug(workerAuth): allow duplicate workerAuth inserts if records match	3 years ago
Jeff Mitchell	8322916fb4	Remove all deprecated Err/Wrap/E calls (#3286 )	3 years ago
Jeff Mitchell	18e59998e9	Add storage bucket worker filter handling	3 years ago
Jim	6f78889947	refact: add servers.ListWorkers(...) refactor servers.(repository).ListWorkers(...) into just servers.ListWorkers(...) used independently of the repo. The method func is refactor to use the new bits as well.	3 years ago
Michael Li	357e835fa5	doc: Add missing copyright headers (#3238 )	3 years ago
Jeff Mitchell	5716a30813	Update nodeenrollment to fix TLS issue; add debug (#3231 ) * Add more logging to worker auth rotation and test flags This makes it much easier to figure out when and why rotation is happening and control it for tests/debugging. * Update auth rotation logic Because we were looking at validity period of client certs, which may not have matched the actual rotation interval, it was possible (mostly in test scenarios) for the rotation to happen multiple times on the roots before we actually hit the worker rotation interval. This changes how the interval is calculated by looking at the difference between not-after values on the certs to understand the rotation interval being used by the controller. We then use that as the starting point for calculating a next-rotation time.	3 years ago
Jeff Mitchell	77fd51fb12	Add more logging to worker auth rotation and test flags This makes it much easier to figure out when and why rotation is happening and control it for tests/debugging. Remove cert skew Cherry-picked from release/0.12.x	3 years ago
Jeff Mitchell	bdf7bf7702	Do not allow PKI-KMS workers to have names updated via API (#3143 ) Do not allow PKI-KMS workers to have names updated via API This would cause their public IDs to change, which we do not want. Descriptions are also disallowed, since those come from the config file. This also adds logic to allow upgrading from old KMS method to new, with a test. Co-authored-by: Irena Rindos <irenarindos@users.noreply.github.com>	3 years ago
Jeff Mitchell	163ce184b8	KMS-PKI Workers (#3101 ) This adds a new mechanism for worker authentication that uses the third-party arbitration of the original KMS flow by leveraging recently-added capabilities in nodeenrollment to support a registration wrapper. This allows for workers to be registered to the system with just a name and a matching wrapper, while at the same time allowing these workers to be used for private Vault access and multi-hop as they contain per-worker encryption keys and support rotation. Nodes using this mode will generate and rotate credentials in-memory (using another recently added nodeenrollment capability) so will have a new set of keys on every startup. This has the side effect of ensuring that there is never a conflict or re-use of these credentials. Note that the normal test worker flow was using the now-deprecated KMS method; there is still a test that uses this (explicitly) but all other tests using test workers that did not specify a specific PKI flow now actually use this mechanism. This means there is fairly decent test coverage in a general sense, plus the specific tests updated/added for this feature. This also adds a new KMS type that can be used to have distinct KMSes for upstream authentication vs. accepting from downstream.	3 years ago
Jim	63a2b62b85	chore (deps): update GKW and protobuf; and run "make gen" (#3111 )	3 years ago
Jeff Mitchell	b76b24a4ad	Move prefixes for many packages into the globals package (#3069 ) This is a prerequisite for some enhancements to grant validation	3 years ago
Todd	1e89be1b1a	Downstream worker connections are tracked by worker id (#2949 ) When possible downstream workers will include their worker id information in the connections they establish with the upstream using the nodeenrollment library. The tracking listener now tracks connections that are connected in this same way instead of just connections of type *tls.Conn.	3 years ago
Johan Brandhorst-Satzkorn	3c29308673	chore: Add license headers to all files	3 years ago
Todd	d8aea569c5	Only return live workers in ListHcpbWorkersResponse (#2876 )	3 years ago
Todd	bc409bc7d4	Graceful shutdown blocks on running proxy handler connections (#2789 )	3 years ago
Irena Rindos	b7f0f455db	Add options for directly connected worker (#2762 )	3 years ago
Irena Rindos	895aecb58c	Do not emit error on auth not found (#2737 )	3 years ago
Jim	38b12394fd	refactor (kms): add GetWrapperer for RewrapFn parameter (#2701 ) * refactor (kms): add GetWrapperer for RewrapFn parameter * tests (kms): add MockWrapper and MockGetWrapperer Make it a bit easier to write unhappy path tests which depend on a Wrapper or GetWrapperer	3 years ago
Irena Rindos	5b162a4433	add worker version to network graph (#2672 ) * add worker version to network graph	3 years ago
Todd	35e5bcd8e5	Fixing missed error handling when merging from 0.11.1 into main (#2634 )	3 years ago
Johan Brandhorst-Satzkorn	edd323b73a	Key Rotation/Destruction (#2477 ) (#2607 ) * Key Rotation/Destruction (#2477) * fix: Correct kms.CreateKeys comment This method does not return anything. * feat(kms): Add new ListKeys method The new ListKeys method wraps the underlying KMS wrapper, returning all the keys in the scope specified. * feat(scopes): Add new scope actions for key management Adds list-keys, rotate-keys and revoke-keys actions for scopes. * feat(scopes): Add ListKeys API endpoint This new endpoint lists all keys in a scope * feat(api): Add ListKeys to Scopes client This has to be a custom function because it is a custom action and doesn't map well to any of the existing templates, or generalises well in a way that would make it reasonable to create a new template. * feat(cli): Add new scopes list-keys command * add RotateKeys function to the kms repository * Add Rotate Keys Endpoint to Scopes Api (#2360) * add rotate keys endpoint * add tests for RotateKeys endpoint * remove that one comment I forgot about * addressing more PR comments * Add Rotate Keys CLI Command (#2395) * add cli command and client * add bats tests * fix some info text and pr comments * write a new rotate keys description * Add Missing DEK Foreign Keys (#2408) * add missing fks * mark key as nullable in gorm * update tests to use a new wrapper with a real key_id * fix formatting in test and add keys for auth_token test * replace key values * revert postgres_40_01_test.go * style: reformat sql for readability * refactor: change type of key_id columns to kms_private_id * Store key_id and scope_id with oplog entries, re-type columns referencing data key version (#2431) * fix(db): Correct types of data key version referencers * fix(oplog): Fix missing error handling * feat(oplog): Store key_id and scope_id with oplog entries This migration truncates all existing oplog entries, as migrating existing data would have been complex and likely unnecessary. * Update view references * Always delete oplog entries when dropping keys * Fix rebase issues * Fix sql tests * Add schema for new kms destruction jobs (#2479) * feat: Add schema for new kms destruction jobs The schema lets us manage destruction jobs per-table while providing a guarantee that only one run is running at a time. * Apply suggestions from code review Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com> * Fix table reference and tests Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com> * update domain, api, and cli to all include key versions in outputs (#2472) * update domain, api, and cli to all include key versions in outputs * fix go sum * add migration to fix immutable error * fix final test issue as well as make gen issue * rebase and fix * address johan pr comments * make gen, didn't realize comments were included * update migrations based on new migration 55 file 03 * Move migrations to 56 * feat: Add key version rewrapping function registry (#2478) The new RegisterTableRewrapFn can be used by a domain to register a callback to use when rewrapping data in a specific table name. * feat(kms): Add scopeId to RewrapFn (#2539) This makes it much easier for rewrap functions to look up the correct wrapper and limit the number of rows searched. * Update Root Cert Proto Definition (#2542) * update root cert proto definition to properly identify public_key as the table primary key * add the test fix as well * Rewrap Functions for Encrypted Tables (#2532) * add oidc rewrap * add argon2 config rewrap * add auth token rewrapping * add worker auth cert rewrapping * check the err * add username password credential rewrapping * add ssh private key credential rewrapping * add vault client certificate rewrap function * add host catalog secret rewrap function * cleanup a little * add host vault token rewrap function * add session credential rewrap function * add session rewrap function * add worker auth rewrap function * update test to include full assert gamut * add session rewrap function TEST * bring all tests up to current standards, make more readable, and include full assert gamut * rework all rewrap functions to utilize scope id * update comments, queries, sql refs, etc, in response to PR comments * forgot a comment * remove hmac updates and fix a couple comments again * Rewrapping Registered Tables Test (#2555) * add the registered table test as well as the two missing rewrap functions * address pr comments, add db r/w conversions, cmp.diff, sql comment, remove (now) faulty immutable test * Add ability to list key version destruction jobs, recurring jobs and destroying key versions (#2498) * feat(kms): Add ability to list data key version destruction jobs * feat(scopes): Add ListKeyVersionDestructionJobs * feat(cli): Add list-key-version-destruction-jobs * feat(kms): Add recurring job that performs table rewrapping The new recurring job runs for each job table with a registered rewrapping callback. It attempts to become the running run and start its rewrapping, gracefully handling sudden interruptions by resuming it's work if it finds evidence of sudden interruption. * feat(kms): Add recurring job for destroying key versions The new job monitors the database for data key version destruction jobs that have finished rewrapping all its data, performing the final data key version revocation. * feat(kms): Add DestroyKeyVersion DestroyKeyVersion immediately destroys a key version if it is a root key version or a data key version which encrypts no data. If the data key version encrypts data which needs to be rewrapped, it queues an asynchronous job to complete the rewrapping operation. * feat(scopes): Add DestroyKeyVersion API endpoint * feat(cli): Add scopes destroy-key-version CLI command * Ensure all secret updates include key ID (#2556) * feat(all): Ensure all secret updates include key ID When updating a secret, it is paramount that the key ID is also updated, as it is the only means we have of ensuring that we only destroy keys once there is no more data associated with the key. * Remove fix to session repository for now This breaks the current private key derivation method * add new encrypt/decrypt funcs and made necessary proto changes (#2557) * add new encrypt/decrypt funcs and made necessary proto changes * address all PR comments * fix test panic * add param checking to rewraps * fix(schema): Correctly organize migrations again * Review comments part 1 * Review comments part 2 * Review comments part 3 * Store cert private key, encrypt tofu token (#2583) * fix(session): store cert private key, encrypt tofu token Previously, we would derive a session certificate private key from the session key used in each project, and not store the key. We would also encrypt the tofu token with the database key but not store a reference to this key in the database. This change fixes this by replacing our key derivation with a key generation step, and instead store the generated key, encrypted, in the database. We also ensure that any new tofu tokens are encrypted with the same key. To handle existing sessions, we lazily rewrite the sessions whenever a user lists them. There is a minor risk that a user could end up destroying the database key that encrypts the tofu token before that session has been rewrapped, but this is going to be rare and temporary. * feat(session): Allow the random source for secrets to be configured * Review comments * Minor fixes * More minor fixes * More small fixes Co-authored-by: Danielle Miu <dani.miu@hashicorp.com> Co-authored-by: Danielle <29378233+DanielleMiu@users.noreply.github.com> Co-authored-by: Michael Gaffney <mike@gaffney.cc> Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com> * fix(session): Unset session key ID when scope is deleted Unsetting the key ID allows the session to live beyond the lifetime of the scope, while allowing the scope to cascade delete its keys. * fix(migrations): Rewrite migration 56 tests These tests can no longer use the normal test helpers as they try to use the new oplog table structure, so we have to manually write all the interactions. * fix(migrations): Rename migration 58 file name This was the name used in the release branch, update it to the new migration number. * fix(e2e): Add some comments and debug to kms tests * fix(session): Handle empty project and user IDs in read When a project or user is deleted, the session is automatically canceled and the respective field on the session is unset. LookupSession did not handle this case gracefully, and would error on decryption in this state. Skip decrypting sessions that do not have either a user or project to avoid this error. The decrypted values are not used for a canceled session. * Fix worker test * Increase test timeout Co-authored-by: Danielle Miu <dani.miu@hashicorp.com> Co-authored-by: Danielle <29378233+DanielleMiu@users.noreply.github.com> Co-authored-by: Michael Gaffney <mike@gaffney.cc> Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com>	3 years ago
Jeff Mitchell	0c96c6ff6c	Split grace period into multiple config values (#2578 ) (#2603 ) Co-authored-by: Todd <github@quaddi.com>	3 years ago
Todd	2a4ce02de6	Disconnect PKI workers from upstreams when they are no longer authorized (#2515 )	3 years ago
Johan Brandhorst-Satzkorn	f57454b6b9	Rewrite interface{} to any (#2535 ) * chore: Rewrite interface{} to any * Add ignore revs	3 years ago
Irena Rindos	da885a89c3	refactor(workerauth): consolidate workerAuth retrieval (#2512 ) * refactor worker auth	4 years ago
Irena Rindos	4493dfd527	use prior and current worker auth keys (#2495 ) * use prior and current worker auth keys	4 years ago
Irena Rindos	821d8317dc	address op state review feedback (#2453 ) * address op state review feedback	4 years ago
Irena Rindos	fc0ead73e6	feat(workers): worker operational state (#2427 ) * feat(workers): worker operational state	4 years ago
Irena Rindos	18dff62b7b	Merge BYOW GA branch to main (#2398 ) * feat(workers): Add ability to read and reinitialize cert authority (#2312) * feat(workers): Return Release Version on worker list and read (#2377) * feat(cli): Read and reinitialize worker certificate authority (#2387)	4 years ago
Timothy Messier	eb61ac6304	refact(session): Add context to session factory This adds a context to the session.RepositoryFactory, allowing errors.NewDeprecated to be replaced with errors.New. It also removes the common.SessionRepoFactory, instead packages just directly import from the session package.	4 years ago
Jeff Mitchell	01fb949d0b	Add controller-led worker auth flow (#2413 ) This adds an action to the API that allows pulling out an activation token that can be given to a worker in its config (directly or via env or file). The worker can pass this along with its normal request in place of its normal nonce and the server will automatically accept it. Co-authored-by: Irena Rindos <irenarindos@users.noreply.github.com>	4 years ago
Irena Rindos	db21ead31c	Vault proxy supporting code (#2415 ) * Vault proxy supporting methods	4 years ago
Jim	b0bc6bd632	refactor: eliminate gorm references in favor of go-dbw (#2386 )	4 years ago
Johan Brandhorst-Satzkorn	222a935223	Update grpc-gateway dependency (#2311 ) * chore(deps): Update grpc-gateway dependency This fixes an issue which would allow an attacker to trigger a (recovered) panic in the gateway handler. * fix(server): Use cmp.Diff instead of assert.Equal Testify is not great at handling protobuf types, and was for some reason returning a diff here when in reality there is none. Switch to go-cmp for the comparison to remove the error.	4 years ago
Jim	6fdcccb953	feature (server): add additional option for downstream workers DAG (#2278 )	4 years ago
Jim	b3ec54e81b	feature (server): add new error codes for downstream workers DAG (#2275 )	4 years ago
Haotian	28f53a64b4	feat(workers): implement worker service add/set/remove api tags	4 years ago
Jeff Mitchell	fb3b2209e5	Add WithType to worker listing (#2252 )	4 years ago
Haotian	83bf38f030	feat(worker): add/set/delete repo worker tags (#2236 ) implements repo changes for add/set/delete worker tag operations	4 years ago
Todd	9af6c09c36	Move generated pb.go file in to the internal/gen directory (#2225 )	4 years ago
Louis Ruch	4cacd0a605	bug(scheduler): Fix controller id length check (#2226 ) * bug(scheduler): Fix controller id length check The job_run table has a foreign key to the server_controller private_id. While this column is called a prviate_id it does not implement the wt_private_id postgres domain type, and therefore does not have a minimum length of 10 characters.	4 years ago
Todd	19b549c44a	Rename package servers to server (#2222 )	4 years ago

49 Commits (cae97fcd2e2019c05e09aec24c7068e6fbbfc5d1)