boundary

Commit Graph

Author	SHA1	Message	Date
Johan Brandhorst-Satzkorn	edd323b73a	Key Rotation/Destruction (#2477 ) (#2607 ) * Key Rotation/Destruction (#2477) * fix: Correct kms.CreateKeys comment This method does not return anything. * feat(kms): Add new ListKeys method The new ListKeys method wraps the underlying KMS wrapper, returning all the keys in the scope specified. * feat(scopes): Add new scope actions for key management Adds list-keys, rotate-keys and revoke-keys actions for scopes. * feat(scopes): Add ListKeys API endpoint This new endpoint lists all keys in a scope * feat(api): Add ListKeys to Scopes client This has to be a custom function because it is a custom action and doesn't map well to any of the existing templates, or generalises well in a way that would make it reasonable to create a new template. * feat(cli): Add new scopes list-keys command * add RotateKeys function to the kms repository * Add Rotate Keys Endpoint to Scopes Api (#2360) * add rotate keys endpoint * add tests for RotateKeys endpoint * remove that one comment I forgot about * addressing more PR comments * Add Rotate Keys CLI Command (#2395) * add cli command and client * add bats tests * fix some info text and pr comments * write a new rotate keys description * Add Missing DEK Foreign Keys (#2408) * add missing fks * mark key as nullable in gorm * update tests to use a new wrapper with a real key_id * fix formatting in test and add keys for auth_token test * replace key values * revert postgres_40_01_test.go * style: reformat sql for readability * refactor: change type of key_id columns to kms_private_id * Store key_id and scope_id with oplog entries, re-type columns referencing data key version (#2431) * fix(db): Correct types of data key version referencers * fix(oplog): Fix missing error handling * feat(oplog): Store key_id and scope_id with oplog entries This migration truncates all existing oplog entries, as migrating existing data would have been complex and likely unnecessary. * Update view references * Always delete oplog entries when dropping keys * Fix rebase issues * Fix sql tests * Add schema for new kms destruction jobs (#2479) * feat: Add schema for new kms destruction jobs The schema lets us manage destruction jobs per-table while providing a guarantee that only one run is running at a time. * Apply suggestions from code review Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com> * Fix table reference and tests Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com> * update domain, api, and cli to all include key versions in outputs (#2472) * update domain, api, and cli to all include key versions in outputs * fix go sum * add migration to fix immutable error * fix final test issue as well as make gen issue * rebase and fix * address johan pr comments * make gen, didn't realize comments were included * update migrations based on new migration 55 file 03 * Move migrations to 56 * feat: Add key version rewrapping function registry (#2478) The new RegisterTableRewrapFn can be used by a domain to register a callback to use when rewrapping data in a specific table name. * feat(kms): Add scopeId to RewrapFn (#2539) This makes it much easier for rewrap functions to look up the correct wrapper and limit the number of rows searched. * Update Root Cert Proto Definition (#2542) * update root cert proto definition to properly identify public_key as the table primary key * add the test fix as well * Rewrap Functions for Encrypted Tables (#2532) * add oidc rewrap * add argon2 config rewrap * add auth token rewrapping * add worker auth cert rewrapping * check the err * add username password credential rewrapping * add ssh private key credential rewrapping * add vault client certificate rewrap function * add host catalog secret rewrap function * cleanup a little * add host vault token rewrap function * add session credential rewrap function * add session rewrap function * add worker auth rewrap function * update test to include full assert gamut * add session rewrap function TEST * bring all tests up to current standards, make more readable, and include full assert gamut * rework all rewrap functions to utilize scope id * update comments, queries, sql refs, etc, in response to PR comments * forgot a comment * remove hmac updates and fix a couple comments again * Rewrapping Registered Tables Test (#2555) * add the registered table test as well as the two missing rewrap functions * address pr comments, add db r/w conversions, cmp.diff, sql comment, remove (now) faulty immutable test * Add ability to list key version destruction jobs, recurring jobs and destroying key versions (#2498) * feat(kms): Add ability to list data key version destruction jobs * feat(scopes): Add ListKeyVersionDestructionJobs * feat(cli): Add list-key-version-destruction-jobs * feat(kms): Add recurring job that performs table rewrapping The new recurring job runs for each job table with a registered rewrapping callback. It attempts to become the running run and start its rewrapping, gracefully handling sudden interruptions by resuming it's work if it finds evidence of sudden interruption. * feat(kms): Add recurring job for destroying key versions The new job monitors the database for data key version destruction jobs that have finished rewrapping all its data, performing the final data key version revocation. * feat(kms): Add DestroyKeyVersion DestroyKeyVersion immediately destroys a key version if it is a root key version or a data key version which encrypts no data. If the data key version encrypts data which needs to be rewrapped, it queues an asynchronous job to complete the rewrapping operation. * feat(scopes): Add DestroyKeyVersion API endpoint * feat(cli): Add scopes destroy-key-version CLI command * Ensure all secret updates include key ID (#2556) * feat(all): Ensure all secret updates include key ID When updating a secret, it is paramount that the key ID is also updated, as it is the only means we have of ensuring that we only destroy keys once there is no more data associated with the key. * Remove fix to session repository for now This breaks the current private key derivation method * add new encrypt/decrypt funcs and made necessary proto changes (#2557) * add new encrypt/decrypt funcs and made necessary proto changes * address all PR comments * fix test panic * add param checking to rewraps * fix(schema): Correctly organize migrations again * Review comments part 1 * Review comments part 2 * Review comments part 3 * Store cert private key, encrypt tofu token (#2583) * fix(session): store cert private key, encrypt tofu token Previously, we would derive a session certificate private key from the session key used in each project, and not store the key. We would also encrypt the tofu token with the database key but not store a reference to this key in the database. This change fixes this by replacing our key derivation with a key generation step, and instead store the generated key, encrypted, in the database. We also ensure that any new tofu tokens are encrypted with the same key. To handle existing sessions, we lazily rewrite the sessions whenever a user lists them. There is a minor risk that a user could end up destroying the database key that encrypts the tofu token before that session has been rewrapped, but this is going to be rare and temporary. * feat(session): Allow the random source for secrets to be configured * Review comments * Minor fixes * More minor fixes * More small fixes Co-authored-by: Danielle Miu <dani.miu@hashicorp.com> Co-authored-by: Danielle <29378233+DanielleMiu@users.noreply.github.com> Co-authored-by: Michael Gaffney <mike@gaffney.cc> Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com> * fix(session): Unset session key ID when scope is deleted Unsetting the key ID allows the session to live beyond the lifetime of the scope, while allowing the scope to cascade delete its keys. * fix(migrations): Rewrite migration 56 tests These tests can no longer use the normal test helpers as they try to use the new oplog table structure, so we have to manually write all the interactions. * fix(migrations): Rename migration 58 file name This was the name used in the release branch, update it to the new migration number. * fix(e2e): Add some comments and debug to kms tests * fix(session): Handle empty project and user IDs in read When a project or user is deleted, the session is automatically canceled and the respective field on the session is unset. LookupSession did not handle this case gracefully, and would error on decryption in this state. Skip decrypting sessions that do not have either a user or project to avoid this error. The decrypted values are not used for a canceled session. * Fix worker test * Increase test timeout Co-authored-by: Danielle Miu <dani.miu@hashicorp.com> Co-authored-by: Danielle <29378233+DanielleMiu@users.noreply.github.com> Co-authored-by: Michael Gaffney <mike@gaffney.cc> Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com>	3 years ago
Jeff Mitchell	0c96c6ff6c	Split grace period into multiple config values (#2578 ) (#2603 ) Co-authored-by: Todd <github@quaddi.com>	3 years ago
Hugo Vieira	d206635d74	fix(connection): Make bytes up and down a signed 64-bit integer This fixes a theoretical issue with database interaction. Because the fields in the database are signed 64-bit integers, it's theoretically possible to overflow them if we're using unsigned 64-bit integers in the application code.	4 years ago
Hugo Vieira	c81398cbf3	feat(worker): Report bytes up and down on Status updates With this commit, the Worker now sends BytesUp and BytesDown for each of the connections it handles to the Controller in the Status request. The Controller then persists these fields to the database. There is also a new trigger to prevent bytes_up and bytes_down to change after a connection has been closed, as the last update to a session_connection row should be the one that is performed as part of the connection closure process.	4 years ago
Johan Brandhorst-Satzkorn	f57454b6b9	Rewrite interface{} to any (#2535 ) * chore: Rewrite interface{} to any * Add ignore revs	4 years ago
Hugo Vieira	bf1486f75e	refact(target): Add context to RepositoryFactory constructor `errors.New` requires a context, so I've made that available from the Controller, and updated all `NewDeprecated` functions to `New` while I was at it	4 years ago
Timothy Messier	e65f1f32a7	chore(session): Remove repository method that was only used in tests This unexported method was only used by a single test, and was taking advantage of its implmentation to populate Connections rather than Sessions. This repository should only interact with Sessions, since there is a separate repository for Connections.	4 years ago
Timothy Messier	eb61ac6304	refact(session): Add context to session factory This adds a context to the session.RepositoryFactory, allowing errors.NewDeprecated to be replaced with errors.New. It also removes the common.SessionRepoFactory, instead packages just directly import from the session package.	4 years ago
Timothy Messier	2fbdcf6ce0	feat(session): Use permissions for limiting list results The session repository can now be passed an optional set of permissions. The permissions are used to restrict results from the repository. Currently this only impacts the List method. This enabled the sessions service to use a single query to retrieve sessions and replaces the previous behavior where the session service would first request all sessions for a given set of scopes, evaluate the permissions, and then request sessions based on a set of session ids.	4 years ago
Johan Brandhorst-Satzkorn	aef9073fa6	Upgrade to Go 1.19 (#2347 )	4 years ago
Damian Debkowski	27e9f775be	feat: add initial migration hook	4 years ago
Louis Ruch	d01f374490	bug(db): Fix credential purpose migration (#2351 ) * bug(db): Fix credential purpose migration	4 years ago
Louis Ruch	a17e973712	feat(credentials): Refactor credential purposes (#2260 ) * feat(credentials): Refactor credential purposes * Application credentials have been refactored to Brokered credentials * Egress credentials have been refactored to Injected Application credentials Co-authored-by: Jeff Mitchell <jeffrey.mitchell@gmail.com>	4 years ago
Jeff Malnick	1f66685864	feat: set default connection limit to unlimited (-1) instead of 1 (#2234 ) This avoids the confusion behavior of a session terminating as soon as a single connection using that session is closed.	4 years ago
Todd	19b549c44a	Rename package servers to server (#2222 )	4 years ago
Jeff Mitchell	704d68848c	Merge remote-tracking branch 'origin/main' into llb-byow	4 years ago
Todd	e379234259	Remove CanonicalAddress, GetApiTags(), and NewWorkerForStatus. (#2196 ) Minor testing tweaks are also included.	4 years ago
Todd	beecbbb8a2	Upsert creates a new KMS and Updates PKI workers (#2187 )	4 years ago
Louis Ruch	58d9d42a88	feat(session): Add support for session static credentials	4 years ago
Louis Ruch	00dfea1244	feat(target): Add support for static credential as target credential sources	4 years ago
Jeff Mitchell	2d274c3742	Fix merge brokenness	4 years ago
Jeff Mitchell	5d3facf561	Merge remote-tracking branch 'origin/main' into llb-byow	4 years ago
Timothy Messier	d741034829	perf(session): Change list behavior around terminated sessions It was determined that it is a more common use case for users to request non-terminated sessions (i.e. sessions in `pending`, `active`, or `canceling` when requesting sessions. Notably, end users would need non-terminated sessions to: - Establish a new connection to a session. - Terminate a session So now by default a list request to `/v1/sessions` will not include terminated sessions. To still enable use cases where terminated sessions are needed, this adds a new query parameter to include the terminated session: `/v1/sessions?include_terminated=true`. Note that this query parameter is considered experimental and subject to change.	4 years ago
Timothy Messier	32070678dc	perf(session): Remove connections from session list endpoint Including the nested connection information in the session list response had a negative impact on performance as it created a significant multiplier on the number of rows that needed to be fetched from the database. It was also determined that this information was used by neither the admin ui nor the desktop client. Note that nested connection information can still be retrieved via a read request for a specific session.	4 years ago
Timothy Messier	f9eab71a4a	feat(session): Add periodic job to delete terminated sessions	4 years ago
Timothy Messier	de2421cc25	feat(session): Add repository method to delete terminated sessions This will delete sessions that have been in the termianted state for longer than the given threshold.	4 years ago
Timothy Messier	a4d0e3c7f8	refact(session): Rename files with job to use a prefix This makes it so jobs will all be naturally grouped together in file listing.	4 years ago
Todd	731a45eba6	Merge the worker_status table with the worker table (#2111 )	4 years ago
Todd	a281d10206	Add worker config table and view (#2090 ) * Remove the server_id from session table, change it to worker_id on session_connection This is step 1 in updating the worker table. The worker is becoming a public resource so it must have a public id. We drop the server_id column on the session_connection table and add the worker_id column so we can use the wt_public_key type. Since db migrations are applied while the controller is offline, it should be safe to assume all session connections are not in a running state, thus allowing us to null out all the worker ids in the session_connection table. For now we don't have very good enforcement of the wt_public_id constraints, since the id is currently defined in the worker config, but this will soon change and the worker's id will be generated by the controller.	4 years ago
Todd	3fbf901739	Add source to worker tags table (#2087 ) * Remove the server_id from session table, change it to worker_id on session_connection This is step 1 in updating the worker table. The worker is becoming a public resource so it must have a public id. We drop the server_id column on the session_connection table and add the worker_id column so we can use the wt_public_key type. Since db migrations are applied while the controller is offline, it should be safe to assume all session connections are not in a running state, thus allowing us to null out all the worker ids in the session_connection table. For now we don't have very good enforcement of the wt_public_id contraints, since the id is currently defined in the worker config, but this will soon change and the worker's id will be generated by the controller. * Adding worker tag db writing test. * Move the server table back to the top. * Update comment for the worker object. * Update comment for the worker table. * Add worker tag sources ("config" or "api") * Revert renaming of UpsertWorker until later. * Remove no longer needed files. * Moving the TestWorker into the servers package. Fix the worker tags test. * Update internal/proto/controller/storage/servers/store/v1/worker.proto Co-authored-by: Irena Rindos <irenarindos@users.noreply.github.com> * Place the repos back together so they can be refactored in a different PR. Co-authored-by: Irena Rindos <irenarindos@users.noreply.github.com>	4 years ago
Todd	2ecd550e27	Add fields to server_worker to be a boundary resource (#2083 ) * Remove the server_id from session table, change it to worker_id on session_connection This is step 1 in updating the worker table. The worker is becoming a public resource so it must have a public id. We drop the server_id column on the session_connection table and add the worker_id column so we can use the wt_public_key type. Since db migrations are applied while the controller is offline, it should be safe to assume all session connections are not in a running state, thus allowing us to null out all the worker ids in the session_connection table. For now we don't have very good enforcement of the wt_public_id contraints, since the id is currently defined in the worker config, but this will soon change and the worker's id will be generated by the controller. * Updating server_id references to controller_id. * Adding worker tag db writing test. * Move the server table back to the top. * Update comment for the worker object. * Update comment for the worker table. * Update the newWorkerId call to take a context and fix its error. * Correct comment for the wt_network_address domain.	4 years ago
Todd	472d7d520a	Remove the server_id from session table, change it to worker_id on session_connection (#2070 ) * Remove the server_id from session table, change it to worker_id on session_connection This is step 1 in updating the worker table. The worker is becoming a public resource so it must have a public id. We drop the server_id column on the session_connection table and add the worker_id column so we can use the wt_public_key type. Since db migrations are applied while the controller is offline, it should be safe to assume all session connections are not in a running state, thus allowing us to null out all the worker ids in the session_connection table. For now we don't have very good enforcement of the wt_public_id constraints, since the id is currently defined in the worker config, but this will soon change and the worker's id will be generated by the controller.	4 years ago
Jeff Mitchell	2f06513a3a	Merge branch 'main' into llb-byow	4 years ago
Jim	785241237e	refactor (kms): adopt the go-kms-wrapping/extras/kms package (#2027 ) * feature (db): add Db.UnderlyingDB() This is needed to allow for integration with bits that need a dbw.DB feature (kms): add new go-kms-wrapping/extras/kms dependency * refactor (kms): schema changes required to adopt extras/kms pkg * refactor (kms/iam): rename to DeprecateCreateKeysTx(...) * refactor (kms): remove deprecated test * refactor (kms): refactor Kms type This refactor includes renaming the existing type to DeprecatedKms and then refactoring the Kms type to become a wrapper of go-kms-wrapping/extras/kms * refactor (kms): delete all the unneeded bits All this stuff is not needed now that we're using go-kms-wrapping/extras/kms * refactor (db): fixup tests after kms refactor * refactor (kms/tests): use testing.TB for test helpers Our test helper packages were not possible to use with benchmarks. This refactor switches to testing.TB to allow both tests and benchmarks to use the helpers. * tests (session benchmarks): refactor to use new Kms type * refactor (schema): rm test which relies on deprecated Kms code * refactor (kms): post rebase on main changes required * chore (kms refactor): re-add unit test from main * refactor (kms): uses migrations that don't include KMS schema changes	4 years ago
Johan Brandhorst-Satzkorn	8cef1017e9	fix(session): Fix typo in connection documentation (#2075 )	4 years ago
irenarindos	54c30d2cad	bug(session): null fkey trigger also checks for session terminated state	4 years ago
Jeff Mitchell	2a8e179692	Update against changes from merge	4 years ago
Jeff Mitchell	ab4d542fad	Add multihop auth support (#32 )	4 years ago
irenarindos	9e27605213	refact(servers): Split server table into worker and controller	4 years ago
Todd	687dd1bda6	Move the session cleanup job logic into the session package. (#2060 ) * Move the session cleanup job logic into the session package. This is part of a broader effort to more narrowly define the purpose of the internal/server directory to be for interacting with servers themselves. Future changes will continue to remove things from this directory until the purpose of the directory is clearly working primarily with the boundary servers resource.	4 years ago
irenarindos	c6764e85d9	fix(session): Update session state transition trigger	4 years ago
Timothy Messier	f00af362dc	fix(session): Include where clause with order and limit When applying the limit and order by, the where clause also needs to be applied to yield the correct results. This can be seen with an example. Assuming a data set where there are 100 sessions, with 10 sessions belonging to a user with id `u_user1` and 90 sessions belonging to `u_user2`. If the constructed query is like: select * from (select public_id from session limit 10) s, session_list ss where s.public_id = ss.public_id and s.user_id = 'u_user1' order by create_time asc; The initial sub-query with the limit could return sessions only belonging to `u_user2`, then it would apply the where clause and exclude all of the rows. Instead if the query is: with session_ids as ( select public_id from sessions as s where s.user_id = 'u_user1' order by create_time asc limit 10 ) select * from session_list where session_list.public_id in (select * from session_ids) order by create_time asc; The CTE applies the where clause ensuring that only sessions for `u_user1` are used for the final select from the view. Note that the CTE is use primarily for readability. This also adds an index on `session (create_time)` to aide the query planner in performing the order by.	4 years ago
Jeff Mitchell	4cf2a87e8d	Update targets to new listing method (#2045 ) This uses the prefetch method to fetch resource IDs and validate them for authz by the user before then fetching the full list of targets.	4 years ago
Johan Brandhorst-Satzkorn	9e424db0a3	test(session): Ignore unique error on session create Sometimes when creating a lot of sessions concurrently we can run into a unique constraint error in the warehouse triggers in the database. Simply retrying the call will allow the session creation to succeed.	4 years ago
Johan Brandhorst-Satzkorn	7f9b294a7e	refact(all): Use testing.TB for test helpers Our test helper packages were not possible to use with benchmarks. This refactor switches to testing.TB to allow both tests and benchmarks to use the helpers.	4 years ago
Jeff Mitchell	b41e983503	Reorder authz check for sessions (#2042 ) * Reorder session authz checks This implements a mechanism for service handlers to pass a repo satisfying an interface into the scope IDs listing function. The purpose of this interface is to allow the same function to perform authorization checking against resources rather than requiring service handlers to fetch all resources in the determined set of scopes and then perform this checking. While this could be done in each service handler, this approach centralizes the logic, which has the nice benefit of removing some boilerplate from service handler list functions that have adapted to it. Tests haven't changed, which is expected -- the function should return exactly what was being returned before, simply faster.	4 years ago
Jim	addbfee593	chore: upgrade gofumpt to v0.3.1 (#2028 )	4 years ago
Jeff Mitchell	58a448fc6a	Put session ID in ALPN (#1966 ) This allows us to use SNI for actual host (e.g. routing) information. We don't have the client cert yet so we can't look at that, so this provides a way for us to convey the information needed to look up that session's TLS stack. Using SNI for hosts means we also run into the fact that we don't have automatic agreement in terms of SANs. So when generating the certs we now also pass worker address information to the function to be encoded in the cert. Finally, there is a change in how the websocket dialing happens, because http.RoundTripper tries to be too clever for its own good and overwrites NextProtos on a whim.	4 years ago
Lars Lehtonen	b6ee5dc9cc	internal/session: fix dropped test errors (#1928 )	4 years ago
Jeff Mitchell	7eb29261b2	Update to go-kms-wrapping version 2, and plugin-based KMS (#1901 )	4 years ago
Timothy Messier	9aa2d4cd84	test(session): Speed up some tests around dead worker cleanup This removes or reduces the time that is waited in a number of tests by allowing the grace period time be a configuration option on the connection repository.	4 years ago
Timothy Messier	79670180a7	refact(session): Move domain logic of Worker Status into domain service This creates a domain service function `session.WorkerStatusReport`. The service performs two functions: 1. Checks the status for all of the reported sessions, returning a report of the sessions that are in a "non-active" state, i.e. "cacneling" or "terminated". 2. Looks for any "orphaned" session connections, i.e. connections that are active but were not reported by the worker. These connections are marked as closed. This also removes the "ShouldCloseConnectionsOnWorker" logic, as this was determined to not be a possible state, and was unnecessary.	4 years ago
Timothy Messier	4de6464978	refact(session): Make CloseDeadConnectionsForWorker more testable The query for CloseDeadConnectionsForWorker includes a time comparison to exclude results that are too close to the current time. This is done to account for a delay in reporting by the worker. However, since the delay was hardcoded in the query, it made it difficult to test. This moves the delay duration into a parameter that is set on the repository. The amount of the delay can be set at repository creation time using the new WithWorkerStateDelay option. This defaults to the previously hardcoded amount. This will allow tests to control the delay to exercise this functionality fully.	4 years ago
Timothy Messier	e0603534c0	lint(session): Whitespace and missing ; in sql queries	4 years ago
irenarindos	32132d6bc1	refactor(session connections): Refactor connection closure	4 years ago
irenarindos	3482f91a17	refactor(connections): Move connect connect to the Connection repo	4 years ago
Irena Rindos	5a2c0db586	fix(session connections): Create session connection repository	4 years ago
Irena Rindos	5d6aa3b729	refactor(session): Remove dead session termination code	4 years ago
Irena Rindos	963fbcbdf2	bug(session): Fix CreateSession foundStates logical ordering	4 years ago
Irena Rindos	edae7c899f	bug(session): Allow only valid state transitions	4 years ago
Jim	591ec476e7	refactor: Update internal/db and oplog to use go-dbw package for database operations. (#1785 ) * chore: Add new dbw dependency * refactor: Convert to use dbw.DB * refactor (db): Change GetTicket(...) signature * feature (db tests): Added TestDeleteWhere(...) Adding this allows us to eliminate a leak of the gorm library * refactor: Add ctx to ScanRows(...) signature This is required to refactor all the deprecated errors pkg calls. This refactor required adding a context to quite a few call chains and in particular the job scheduler bits * feature (db/tests): Add support for test opt: WithTestLogLevel(...) This test only option allows callers to specify a log level for the underlying db package (where supported/applicable) when calling TestSetup(...) * refactor (db): Simplify Db type Only store the underlying DB and create the RW on demand when needed. * refactor (oplog): Port to using dbw pkg for db operations * refactor (db): Remove gormDB(...) func * tests (db): Add unit tests for TestDeleteWhere(...) * chore: make fmt changes to formatting	4 years ago
Louis Ruch	5fe23ab14d	feat(session): Store session credentials Creates a new session_credential table and repository functions to add and list credentials attached to a session. All credentials are encrypted using the session scope id.	4 years ago
Jim	baa1d88f1f	feature: Add client ip to inbound request information (#1678 ) * feature: Add client ip to inbound request information * feature (workers): Add the user's client IP to a session's connection information (#1749)	4 years ago
Timothy Messier	533b434d1a	refact(controller/targets): Move tcp functionality into subpackage This follows a similar pattern as the target domain package, providing a subpackage for the tcp target functionality, allowing this to be pluggable.	4 years ago
Thor	ad31b6d6c3	repo: Return connection informat in Session response object (#1690 ) Add connections to the Session object * db: creat new session_list view * repo: session use new session_list view	4 years ago
Jim	dd2c3807cd	refactor: Move functions from kms pkg to new libs/crypto pkg (#1650 )	5 years ago
Jim	4df3b40def	feature (db): Add support for multi-column PKs (#1658 )	5 years ago
Timothy Messier	1b7cfb1704	feat(target): Add support for credential library validation Target subtypes must now implement another repository hook to validate any CredentialLibraries that are being added/set. This allows for subtypes to determine the valid set of CredentialLibraries. For tcp.Targets only CredentialLibaries with credential.ApplicationPurpose are allowed. If any other purpose is provided an error is returned.	5 years ago
Timothy Messier	2cc58ff901	refact(target): Change repository to take CredentialLibraries This change removes the hard-coded credential purpose for the Credential Libraries. It also moves the logic of creating CredentialLibraries from CredentialSources out of the repository and into the targets.Service. This refactor will allow for additional credential purposes to be supported in both the service and the repository.	5 years ago
Timothy Messier	eac433f7ff	refact(target): Move tcp into subpackage This moves all of the tcp target functionality into a tcp package. It also establishes a Register function in the target package to allow specific target types to be registered at initilazation time. When registering, a target type needs to provide its Subtype, as well as some RepositoryHooks that are used by the target Repository.	5 years ago
Jim	8d6dee09a9	refact: Add db.DB wrapper and refact all test fixtures to use it. (#1535 ) * refact: Changes required for gorm v2 * refact: Add db.DB wrapper and refact all test fixtures to use it Stop leaking the gorm abstraction within our unit tests. * refact: Migrate direct calls to pq to pgx driver The refactor will remove the remaining direct dependencies on the pq driver which has been deprecated in favor of the pgx driver. It includes the addition of common.SqlOpen which is a wrapper around sql.Open, which is needed to deal with the driver name changing from "postgres" to "pgx" but in our code base the dialect and driver are synonyms.	5 years ago
Jim	136ac00b49	refact: Changes required for gorm v2 (#1528 )	5 years ago
Timothy Messier	99d0d2f682	wh: Add wh_credential_dimension and bridge tables This adds new dimension table to record credential information. Since a session can have multiple credentials, the wh_session_accumulating_fact is is linked to the credential_dimensions via bridge tables, wh_credential_group and wh_credential_group_membership. When a session is inserted, the credental group and credential dimensions will be upserted. If the group changed in any way, a new group is inserted along with all of the corresponding credentials.	5 years ago
Jeff Mitchell	f8a51b987c	Migrate target host sets -> host sources (#1424 ) This follows the same model as https://github.com/hashicorp/boundary/pull/1413 See there and CHANGELOG for more details.	5 years ago
s-christoff	6b78108ecf	Update primary error functions to take a context, deprecate old functions (#1358 ) * add new error funcs which take a ctx and deprecate existing funcs * use context where available when writing error events * remove circular internal/errors pkg dep * suppress some chatty errors using errors.WithoutEvent() * convert auth oidc funcs to take context where needed to write events Co-authored-by: Jim <jlambert@hashicorp.com>	5 years ago
Jeff Mitchell	ab6f3eaeb4	Migrate credential-library nomenclature around targets to credential-source (#1413 ) See the CHANGELOG entry for more information. Co-authored-by: Jim <jlambert@hashicorp.com> Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com>	5 years ago
Jeff Mitchell	528d54b4ad	Add token/userinfo claims to account read output (#1419 ) This is very useful when trying to write managed groups filters as you can directly look at the structure of the claims for the accounts you're trying to filter. The output of the claims in text format on the CLI currently leaves something to be desired. I started revamping some of the formatting functions and it quickly got very fragile. Fundamentally at this point we're likely better off -- now that we have a good idea what our output format/standards are like -- rewriting the formatting code. I'll take that as a tasker as I have a lot of ideas...	5 years ago
Chris Marchesi	1ebb8efab6	Apply extra review feedback from #1340 (#1400 ) * internal/servers/controller: refactor WaitForNextWorkerStatusUpdate * internal/servers/worker: use connId instead of conn.GetConnectionId() * internal/servers/worker: remove Worker.logClose * internal/session: complement versus inclusive state search * internal/session: use ScanRows instead of Scan * internal/session: make closeConnectionsForDeadServersCte results gormable * internal/tests/cluster: rename TestWorkerSessionCleanup to TestSessionCleanup * Simplify SQL query for CloseConnectionsForDeadWorkers (#1410) Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com>	5 years ago
Jeff Mitchell	ce52acb968	Update strutil import	5 years ago
Jeff Mitchell	140c54e319	Repoint base62 import	5 years ago
Chris Marchesi	5a70875726	internal/servers/controller: Worker failure connection cleanup (#1340 ) This commit adds the support to do the following: * Mark connections for non-reporting workers as closed. This is the controller counterpart to the worker functionality (see #1330). This is written as a scheduled job that does the work DB-side in a single atomic query. * Works to reconcile states if such a broken controller-worker connection resumes and a worker reports a connection as connected that should be disconnected. In this case, the controller will send an update request, and the worker will honor it and terminate the connection. * Further refinement of the grace period setting has been added here. We have converged on the current server "liveness" setting as our default here, which is half of the previous 30s (15 seconds, in other words). Additionally, this is now configurable on the controller and worker side, with the caveat that it's currently impossible to do so in config as the setting has been untagged in HCL. This is exposed so that we can run some sophisticated testing scenarios where we skew the grace period to either the controller or worker to ensure the aforementioned reconciliation works. * Some repository functions have been added to support the new functionality, in addition to some test code to the worker to allow querying of session state while testing. * Finally, we've added some add timestamp subtraction functions as well, basically serving as the opposite of the addition functions.	5 years ago
Jeff Mitchell	2cbcf9a563	Update usage of shared-secure-libs (#1393 )	5 years ago
Chris Marchesi	48e55f156a	worker: ensure connections are closed in local state on controller fail (#1369 ) This backports some test work that is being added in #1340 to assert that the worker is actually marking connections closed in its local state on a failure to contact the controller. This was missed in #1357 during some late-stage refactoring. Summary of the changes: * If we can't actually contact the controller during connection closure to mark the connections as closed, we actually construct a "fake" response that just simply includes a closed state for all connections requested. This is then used to update the local connection state. * Added a 3 second timeout to the CloseConnection request to the controller to ensure the call to close connections completed in a timely manner and did not hang indefinitely. Testing enhancements: * Added a new helper in TestHelper called LookupSession that allows the lookup of a session from the worker's internal session state. * This is used in the new ExpectConnectionStateOnWorker check in the E2E test, which is part of a larger amount of enhancements to the E2E test that are coming in #1340. * Added a ConnectionStatusFromProtoVal helper function to the session package to allow the conversion of protobuf connection status values to their higher-level counterparts; essentially it is a reverse of ConnectionStatus.ProtoVal. This is used in some of our tests (with more coming in #1340) to just help with keeping things more idiomatic. Finally, there is some minor error refactoring, and we also change the timeout on internal status and connection close requests to the controller to 5 seconds.	5 years ago
Michael Gaffney	df35699c4e	Integrate with Vault to retrieve and manage per session credentials (#1308 ) * Fix down migrations * Udpates due to merge * Sentinel values only have to be greater than 0 * Create credential related sdk structs and methods. * Post merge interface updating. * Removing scratch code that was not cleaned up. * updates * Convert client certificate and client certificate keys into pem blocks and validate. * Add newline at end of file. * Faster check for "s" at the end of a resource name when generating from templates. * make fmt * updates * Add dynamic credentials to a session * Refactor: move contents of file * updates * Updates * Add InvalidDynamicCredential error * Assign credentials * Allow updating client certificates seperate from the client certificate key. * Replace application purpose string with enum value * PR feedback / fix go test . * Delete dead code * Use common view * Refactor: extract interface and rename method * Refactor: move code to eliminate privPurpLibrary * Add comment to requestMap * Refactor: do not export methods on an unexported struct * Update internal/db/schema/migrations/postgres/10/03_vault_credential.up.sql Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com> * Add comments to credential Purpose constants * Reorder switch statements to handle errors first * PR feedback * update * fix merge * Organizing the scheduler code and extract the individual store creation test helper function. * Run 'make fmt' * Create certificate when creating store. Allow vault address to be passed in as option * Refactor: rename database views and associated structs This change renames two database views and associated go structs to use a consistent naming pattern. Views ending in 'private' contain encrypted credentials needed for connecting to Vault. Views ending in 'public' mirror the 'private' views but do not include any encrypted values. This change does not include any changes to functionality or behavior. * Adding SAD target operations for credential libraries to the SDK. * Add status field and enumeration table for credential status * Refactor: rename Status to TokenStatus * Add status to vault credentials * Creating the template for the CLI and helper functions. * Add private database view for vault credentials * Add scope_id to view and cleanup TODO comment * Reformat embedded sql queries * Add vault client helpers * updates * white space * Add LookupLease test * CLI commands for Store work. Renamed fields in proto w/ vault prefix. * Test issue credentials with client TLS * Add additional checks to vault ping * Add an error for vault token capabilities and create a VaultToken error kind * Add support for working with Vault policies * Use Vault 1.7.2 for testing * Add testing helper for creating a vault client with a non-root token * Fix Vault renew lease test The Vault renew lease endpoint accepts an optional duration parameter for requesting the number of seconds to extend the lease by. However, Vault is not required to honor this request. * Add a var for the required capabilities of a boundary Vault token * Enhance the test helper for adding a policy to Vault * Allow the test role in postgresql to revoke credentials * Remove out of date comments * Enhance Vault test helpers for the database secrets engine * Reformat comments * Add a method for getting the capabilities of the current Vault token * Add revoke lease to internal Vault client * Reformat: combine err return line with check for nil line * Remove redundant checks in test code Remove checks in test code that are redundant with checks in the test harness. * Refactor CreateToken test helper to return the secret and the token * Fix build failures caused by signature change in vault.CreateToken * Fix Vault version and Vault API link in comments * ran `make gen` after merging from mgaffney-vault. * Remove Vault prefix from attribute fields in API, create a base CLI option for adding field prefixes. format now uses that prefix for error reporting. * Template now allows attaching attribute field prefixes. * Credential store uses PrefixAttributeFieldErrorsWithSubactionPrefix. * Fixing naming of vault flags and helper flags. * Updating more references to generated api methods. * Make the capabilities String method more readable * Add comment outlining PrintApiError usage of Options. * Refactor test to move declarations closer to first use * Fix comment * Implement revoke method on vault repository * Tests now verify certs, CA, and tokens, addresses can be updated in a store. * Added test for creating store with pk/cert in same field. * Allo updating certificates seperate from private key. Only contact backing vault service when updating token. * Fixing bad tests and removing unused code. * Adding tests for various ways of updating tokens. * Add private credential * Add token revocation job and cred renewal/revocation job * Regen after merge. * Fixing field mask related merge error. * Create Library CLI. Removes vault prefix from path. Adds a WithoutCleanup testOption for vault. * Set all credentials to revoked when a token is revoked * Fixing missed renaming update. * fix copy/paste error. * Fixing code generation errors after merge. * make gen after merge. * Update internal/cmd/commands/credentialstorescmd/vault_funcs.go Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com> * Update internal/cmd/commands/credentialstorescmd/vault_funcs.go Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com> * Update internal/cmd/commands/credentialstorescmd/vault_funcs.go Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com> * Refactor: move migrations from 10 to 12 * Removing logic from transaction. * wrapping client errors in the calling op for CredentialStore's client() method. * Redact token * redact client key * updates * Add trigger to revoke credentials when session is canceled or terminated * Fixing some help text for some flags. * Add ability to load values from files (file://) or environment variables (env://). * Updating comments * fix bad merge * feedback * Adding issued credentials to sessions. * Error on capabilities * updates * Expose NewVaulTestServer * Add godoc * Adding generated SessionCredential struct for API. * Progress on building handler test for Authorize Session. * feedback * Update policies * Add 'revoke' as a status for Vault Tokens * Add delete_time to vault credential store * Create dynamic credentials at session creation time. * Tests pass with credentials attached to AuthorizedSession result. * Only call issue if there are credentials to issue. * Fixing output only issues. Adding type to credential library. * Fix test to clear un/pw generated. confirm generated un/pw changes from 1 authorize session call to another. * Update lookupTokenError * Add delete_time to private credential store * Refactor: move private store to separate file * Fixing tests for AuthorizeSession and verifying error cases. * Only create the credential repo if there are credentials to issue. * Rename Library field to CredentialLibrary. * Fixing missing change of Library to CredentialLibrary. * Removing Default SDK functions for fields which are required and update CLI to match. * Replace resource specific id variables with "id" and the input parmater for paths to be just a single string and the resource name as it should show up in the path. * Soft delete vault credential stores * Prevent new SQL functions from silently overwriting existing functions * Rename "Hmac" to "HMAC" in the human readable cli response table. * Fix tests * Add vault database cleanup jobs * Add ability to set the token period in the vault test harness * updates * Add not_null_columns func test * remove unused struct * Up no output timeout * Validate current vault token on addr update * Fix client cert change test * add cert test * Update revocation job where * Fix comment * Use Vault's default port in CLI help strings * Remove down migrations * Increase migration file numbers by 1 * Run make migrations * Move migrations from 12 to 10 * Increase Circle CI timeout for acceptance tests * Fix spelling mistake in error messages * ASD cmds for Target and returning credentials to AuthSession request (#1338) * Add Add/Set/Remove commands for libraries on Targets. Return credentials on AuthorizeSession request. Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com> Co-authored-by: Louis Ruch <louisruch@gmail.com> Co-authored-by: Todd Knight <T.Alan.Knight@gmail.com> Co-authored-by: Jeff Mitchell <jeffrey.mitchell@gmail.com>	5 years ago
Jeff Mitchell	b47b71d26c	Refactor connection listing for tests (#1224 )	5 years ago
Jeff Mitchell	6f34da8923	Add cleanup of dead connections no longer reported by a worker (#1220 )	5 years ago
Michael Gaffney	0b5ac930ac	Fix godoc wording	5 years ago
Michael Gaffney	b8831b16f1	Fix some minor godoc problems	5 years ago
Michael Gaffney	88c3fdc043	Fix godoc spelling	5 years ago
Jim	75108cbc8b	Ongoing OIDC: return the primary account info along with the user. (#1145 )	5 years ago
Jim	dd0f34bc35	Add new OIDC auth method. (#1090 ) Co-authored-by: Todd Knight <T.Alan.Knight@gmail.com> Co-authored-by: Jeff Mitchell <jeffrey.mitchell@gmail.com> Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com>	5 years ago
Michael Gaffney	aafbada3ae	Remove calls to `t.Helper()` from test methods (not helpers) (#1063 )	5 years ago
Jim	c281e808d6	refactor to use WithOrderByCreateTime(...) (#1058 )	5 years ago
Louis Ruch	7387cec475	Remove sentinel errors (#968 ) * Remove sentinal errors	5 years ago
Jeff Mitchell	c63529e020	Fix WithKeyId after recent PR (#978 ) Fixing the session passing around its key ID (which had previously been blank) exposed another issue which was caught during bats tests: WithKeyId statements using the wrong key, which prior to now had been ignored because the argument was empty, resulting in an ineffective option. This fixes those situations. It also updates a bats test that didn't get updated in the CLI return status PR.	5 years ago
Jeff Mitchell	c3684d20db	Fix WithKeyId option (#970 ) Currently this would validate that a KeyId was in the returned wrapper but then return the base wrapper; and if it required a database lookup it wouldn't validate it anyways. All uses of this were for decryption so it didn't actually affect anything to this point (you'd error later instead of sooner) but for any case where we need to encrypt against a specific version or derive a key against a specific version this would be broken, in a way that could lead you to use the wrong key version. This also properly sends the Key ID around sessions. This worked currently for the same reason as above but would have broken for any session in flight once a key was rotated.	5 years ago
Louis Ruch	178d4efa64	ICU-745/Refactor internal/session to new domain errors (#877 ) * Refactor internal/sessions to use domain errors	5 years ago
Todd Knight	02cd972043	Create Sessions List Self Action (#888 )	5 years ago
Jeff Mitchell	6cd97a4a6e	Add support for recursive listing (#885 )	5 years ago
Jeff Mitchell	717a3b52ee	Add worker tagging (#862 ) This allows workers to have tags assigned to them, which can then be filtered to control which targets they are allowed to serve.	5 years ago

1 2 3 4

182 Commits (088564c3853229dec58f973a3cfabce70feefbdc)