boundary

Commit Graph

Author	SHA1	Message	Date
Johan Brandhorst-Satzkorn	a43cb90a32	internal/session: ensure orphaned connection cleanup (#5438 ) Previously, the orphaned connection cleanup job would not detect that a connection that was created before a worker had submitted a successful session info was orphaned. We now fall back to using the worker update time as an indicator if the session info time is not set.	1 year ago
Johan Brandhorst-Satzkorn	ec0a3f964b	Worker status refactor (#5417 ) * oss: feat(proto): statistics rpc added to server coordination service * oss: feat(session): add update statistics status service * oss: feat(handlers): add Statistics method to worker service server * oss: feat(worker): add statistics ticking * oss: internal/proto: define the new RoutingInfo RPC * oss: internal/server: repository changes for RoutingInfo RPC * oss: internal/daemon: add RoutingInfo handler to worker service The new RoutingInfo RPC is used to get information from the worker that is used to make routing decisions on the controller. It also handles the initial request from the worker. * oss: feat(db): server_worker_session_info table * oss: feat(proto): session info rpc for server coordination service * oss: feat(server): worker session info request data type * oss: feat(worker): session info ticker * oss: feat(handler): server coordination handler for session info * oss: feat(session): utilize session info request time for session cleanup * oss: proto: fix some gotags comments Some of these fields were missing comments, while others had incorrectly formatted comments. * oss: worker: send connection status to server This is required for the correct tracking of orphaned connections * oss: internal/daemon/worker: add function to wait for Statistics RPC * oss: internal/daemon/worker: add routing info RPC The new RoutingInfo RPC replaces part of the Status RPC, namely the part which is used by the controller and worker to perform decisions around the establishment of new sessions and new worker connections. * oss: internal/daemon/worker: remove unnecessary error check * oss: internal/daemon/worker: lock access to config and upstreams The new confLock protects access to the conf value and to the addressReceivers slice. * oss: worker: replace Status RPC with RoutingInfo * oss: worker: deprecate the Status RPC and related types The Status RPC and related messages have been replaced by the RoutingInfo/SessionInfo/Statistics RPCs. The controller will continue to support the Status RPC until two new minor versions have been released. * oss: internal/event: add default deny for new worker RPCs * oss: internal/daemon/worker: allow RPC interval to be set The new SessionInfo/RoutingInfo/Statistics RPCs run at long intervals, which make some of our cluster tests take a long time to complete. This allows their interval to be configured in tests. * oss: all: use TestWorkerRPCInterval in slow tests This should speed up these tests significantly. * oss: internal/tests/cluster: reset sys eventer in test --------- Co-authored-by: Damian Debkowski <damian.debkowski@hashicorp.com>	1 year ago
Hugo	846c849589	Worker Domain Refactor (#5338 ) * feat(server): Refactor LookupWorkerByName to be a test method * feat(server): Split worker tags table * refact(worker): Export api and config tags * refact(server): Return worker id from UpsertWorkerStatus * refact(server): Create ListWorkerStorageBucketUpdates domain function * refact(server): Use dedicated HCPb-worker list func * refact(server): Create ConnectionRoute domain function * refact(handlers): Move worker selection logic to internal/server package This commit moves all the logic to select a worker to handle a session to the internal/server package. Most functions were moved to this package with no changes, however, there are a couple of notable exceptions: - Given that we don't use GRPC errors in repos, all errors were adjusted to use the internal/errors package instead. - StorageBucketFilterCredIdFn is now used to prevent the internal/server package from importing internal/storage/plugin, which would cause a dep cycle. Callers of SelectSessionWorkers must implement this func which is responsible for returning a storage bucket's worker filter as well as its associated credential id. - authorizeSessionWithWorkerFilter now receives an internal/server.(Repository) object when called. This is transparent to SelectSessionWorkers callers as it is handled internally. feat(server): Add FilteredWithFeatures fn and new server options * refact(server): create new list and lookup worker queries * test(server): Add WorkerList unit tests * feat(server): Implement WorkerList randomness using crypto/rand pkg * chore(db): add index to server_worker table * refact(server): prefix test method with Test * refact(server): rename Downstreamers interface to Graph * chore(server): unexport and rename method * chore: add doc string to WorkerAddress (#5420) --------- Co-authored-by: Irena Rindos <irenarindos@users.noreply.github.com> Co-authored-by: irenarindos <irena.rindos@gmail.com>	1 year ago
Johan Brandhorst-Satzkorn	7b30697ddd	internal/session: fix race condition in connection cleanup (#5402 ) The totalClosed variable was accessed in both the Status and the Run functions, which may be called from independent goroutines.	1 year ago
dani	62c41191ff	[fix] allow IPv6 controllers, workers, and targets (#5221 ) Co-authored-by: Damian Debkowski <damian.debkowski@hashicorp.com>	1 year ago
Michael Gaffney	9529491fa6	feat(session): Delete terminated sessions in batches This changes the periodic job that deletes terminated sessions to delete the those sessions in batches. Prior to this, a single SQL DELETE command was used to delete all qualifying terminated sessions. In systems under heavy load, the statement could timeout leading to a situation where no qualifying terminated sessions were ever being deleted.	1 year ago
Michael Gaffney	9354a4ebcf	refactor(scheduler): Add statusThreshold parameter to job.Run The scheduler monitors running jobs to make sure they are not hung. If the status of a running job does not change after a duration (the interruptThreshold in the scheduler), the scheduler kills the job. This change passes the scheduler's interruptThreshold to the job when it calls Run() on the job.	1 year ago
Michael Gaffney	9119effa9e	fix(session): Fix query for orphaned connections (#5138 )	2 years ago
Timothy Messier	37bf59243a	refact(session): Replace state start_time/end_time with time range (#5134 ) This replaces the start_time, end_time, and previous_end_time columns on the session_state table with a time range column that records when the state was active. This allowed for dropping several constraints on the session_state table that were used to ensure that states for a session did not overlap. Instead this can be ensured with a single exclusion constraint. By reducing the number of constraints, it improves the performance for a number of queries involved with session state. Most notably when resolving delete cascades of resources related to sessions, such as targets, where as part of the delete transaction large numbers of sessions get canceled. Similarly, when sessions get deleted, and the corresponding session_state rows are deleted, these constraints all must be checked during the transaction. The domain layer continues to expose StartTime and EndTime fields on session.State. However it drops PreviousEndTime, which was never used by anything calling the domain layer. See: https://www.postgresql.org/docs/current/ddl-constraints.html#DDL-CONSTRAINTS-EXCLUSION https://www.postgresql.org/docs/current/rangetypes.html	2 years ago
Jeff Mitchell	62d4f8453b	Don't explode grants from the DB (#5104 ) This removes the Cartesian product in the DB for GrantsForUser in favor of returning the actual grant scope information and dealing with it in application code. (cherry picked from commit `913cd53682`)	2 years ago
Damian Debkowski	9576ed40dc	test: improve test to ensure ipv6 compatability (#5113 )	2 years ago
Johan Brandhorst-Satzkorn	a830892fec	Some test fixes (#5084 ) * internal/session: update variable name in test This was missed in #5071 * internal/tests/cluster: skip failing tests These tests will need to be fixed or removed. It is not clear why they are failing, but they are always failing. * Makefile: try upgrading golangci-lint * internal/host/plugin: fix flaky test Hosts contain IP addresses in string form, whose order is not defined.	2 years ago
Jeff Mitchell	66af7dba98	Allow reloading some server timings (#5071 ) Allows changing grace period and a few other things timing-related via HUP instead of just a restart. Tested manually via config changes and HUP.	2 years ago
Jim	28b465e4b8	refact: to use go-dbw CreateItems, DeleteItems (#4974 ) * refact: to use go-dbw CreateItems, DeleteItems Latest version of go-dbw updated the signature of CreateItems(...) and DeleteItems(...) on a happy note this latest version of go-dbw actually does batch inserts and updates for this functions. * all: regenerate protos with new protoc-gen-go version	2 years ago
Irena Rindos	2191aa1d03	refact(session connection): remove session connection state table (#4617 ) * refact(session connection): remove session connection state table	2 years ago
Timothy Messier	29cd7bc9d6	fix(session): Move encryption/decryption out of database transactions (#4803 ) Several session.Repository methods need to encrypt or decrypt fields when persisting or retrieving a Session. These encrypt/decrypt operations where being performed while within a database transaction. The first step in the encrypt/decrypt operation is to retrieve the kms.Wrapper. While the wrappers get cached in memory, the retrieval still results in a database query to check that the kms collection version has not changed. This query happens with a separate db.Reader, and therefore uses a separate database connection from the transaction being used by the session.Repository. If the controllers are configured with a relatively low `max_open_connections`, and be under sufficient load interactive with sessions, the controller can consume all of the available database connections, and get stuck with open transactions while waiting for a database connection to execute the kms related queries. This can be see by checking pg_stat_activity and seeing transactions stuck in `idle in transaction` with a wait_event of ClientRead. This commit changes all of the Repository methods the perform any encryption or decryption operations, including the retrieval of the kms.Wrapper, outside of a database transaction. An alternative approach would be to unsure that the kms.Wrapper retrieval happens within the same database transaction. However, it seems preferable to avoid doing these relatively expensive operations within a transaction when it is not necessary. As part of this fix, it also eliminates some logic that is no longer needed. During previous would with KMS wrapping, the method of how sessions were encrypted was changed, and some logic to lazily re-encrypt sessions was introduced. However, given that terminated sessions are deleted, and the KMS changes was introduced several versions ago (0.13.0), this lazy re-encrypt logic should no longer be necessary. Refs: `edd323b73a` See: https://www.postgresql.org/docs/current/monitoring-stats.html#MONITORING-PG-STAT-ACTIVITY-VIEW https://www.postgresql.org/docs/current/runtime-config-client.html#GUC-IDLE-IN-TRANSACTION-SESSION-TIMEOUT https://developer.hashicorp.com/boundary/docs/configuration/controller#max_open_connections	2 years ago
Timothy Messier	7c14aeaf41	fix(session): Use active transaction (#4592 ) When closing orphaned connections and checking if a session is not active, the repository was opening a transaction, but then calling the original reader instead of using the reader/writer from the transaction. This would result in a transaction being opend, then a separate transaction being used to run the query, and finally the original transaction would commit without any statements being executed. Co-authored-by: Sorawis Nilparuk (Bo) <sorawis.nilparuk@hashicorp.com>	2 years ago
Louis Ruch	c376d06355	feat: Add support for correlationId	2 years ago
Irena Rindos	1e5559173f	fix(session): return from workerStatusReport on first error (#4586 )	2 years ago
Jeff Mitchell	9408b2b299	Add some more fixes caught by race detector (#4488 )	2 years ago
Jeff Mitchell	014a5a7aae	Vet/race fixes (#4421 ) This addresses all found vet/race fixes in community edition. --------- Co-authored-by: Damian Debkowski <damian.debkowski@hashicorp.com>	2 years ago
Jeff Mitchell	35cef97bd4	Linter fixes PR 3 (#4415 ) * Linter fixes * fix(bsr): update test to register TEST protocol once --------- Co-authored-by: Damian Debkowski <damian.debkowski@hashicorp.com>	2 years ago
Jeff Mitchell	4068829509	Merge pull request #4401 from hashicorp/jefferai-linter-fixes Fix linter warnings, parts 1 and 2	2 years ago
Jeff Mitchell	25bce9e8ff	Fix linter warnings	2 years ago
Jim	3af259ff69	chore: update deps (#4394 ) * chore: update deps * tests: update sqlmock tests for breaking chg * tests (handlers): sort grants in checkEqualGrants(...) sort the expected and got grants in checkEqualGrants(...) to make the test more deterministic. * tests (asciicast): update backspace enc in TestEventMarshal(...)	2 years ago
Jim	6a37c93d77	fix: properly handle rows.Err() (#4261 ) * fix (auth): properly handle rows.Err() * fix (auth/ldap): properly handle rows.Err() * fix (auth/oidc): properly handle rows.Err() * fix (auth/password): properly handle rows.Err() * fix (authtoken): properly handle rows.Err() * fix (credential): properly handle rows.Err() * fix (credential/static): properly handle rows.Err() * fix (credential/vault): properly handle rows.Err() * fix (db): properly handle rows.Err() * fix (db/schema): properly handle rows.Err() * fix (db/migrations): properly handle rows.Err() * fix (host): properly handle rows.Err() * fix (host/plugin): properly handle rows.Err() * fix (host/static): properly handle rows.Err() * fix (iam): properly handle rows.Err() * fix (kms): properly handle rows.Err() * fix (oplog): properly handle rows.Err() * fix (pagination/purge): properly handle rows.Err() * fix (plugin): properly handle rows.Err() * fix (job/scheduler): properly handle rows.Err() * fix (server): properly handle rows.Err() * fix (session): properly handle rows.Err() * fix (target): properly handle rows.Err()	2 years ago
Jim	c6714c1302	refact: remove used limit parameter in query funcs (#4263 )	2 years ago
David Bond	422bd6c224	Use correct argument name in log line (#4252 ) This commit modifies a log line produced in the WorkerStatusReport function when closing orphaned connections that appears to incorrectly label a worker identifier as a controller identifier. Signed-off-by: David Bond <davidsbond93@gmail.com>	2 years ago
Johan Brandhorst-Satzkorn	9a23bd14f0	internal/session: tidy up query	2 years ago
Todd	5056c93845	Fix typo (#4226 )	2 years ago
Johan Brandhorst-Satzkorn	d4d1efce93	internal/session: sort public_id descending The previous sort would run the risk of missing items which collide on update or create times because of the compound comparison using strictly less than for both the timestamp and the public_id. Sorting both descending avoids thi issue.	2 years ago
Johan Brandhorst-Satzkorn	6757510205	internal/session: add helper for listing sessions This is used in an enterprise test	2 years ago
Johan Brandhorst-Satzkorn	3288f78f12	internal/session: validate token resource type	2 years ago
Johan Brandhorst-Satzkorn	576e25bd27	internal/session: fix some test typos	2 years ago
Michael Milton	9c3eb16046	Sessions list pagination (#4079 ) * Session list pagination refactor * address pr comments, add explanatory comments, clean up sql queries, move tests into proper files * fix build error * fix test error	2 years ago
Jim	0d19892e03	fix (events): convert multierror to stdlib error (#3743 ) * fix (events): convert multierror to stdlib error multierror errors will be converted to a stdlib error (because multierror doesn't support json marshaling). * refact (errors): convert from multierror to stdlib errors.Join(...) * refact: convert from pkg/errors to stdlib error wrapping	3 years ago
Irena Rindos	9733a250fa	refact(event): move observability/event to event (#3704 )	3 years ago
Jim	a16033b73a	fix (kms): move oplog DEK into its own internal kms (#3665 ) * fix (auth/ldap): fix which DEK is used for oplog during authenticate * chore: update go-kms-wrapping dep for new extras/kms changes The latest version supports setting a table name prefix for a kms which is needed to fix our oplog DEK problem * chore: update extras/kms dep This latest version supports an optional TableNamePrefix which is required to fix our existing oplog DEK issue. * fix (oplog): schema oplog DEK fix These changes will allow use to store the oplog DEK in it's own set of tables, so they won't get deleted when a scope is deleted * fix (kms): move oplog DEK into its own internal kms * tests: update tests to be compatible with oplog fix * tests: remove migration tests which are incompatible with oplog fix These tests no longer work because they are missing the required schema changes for the oplog fix	3 years ago
hashicorp-copywrite[bot]	29da0bcb92	[COMPLIANCE] License changes (#3567 ) * Adding explicit MPL license for sub-package. This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository. * Adding explicit MPL license for sub-package This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository. * Updating the license from MPL to Business Source License Going forward, this project will be licensed under the Business Source License v1.1. Please see our blog post for more details at https://hashi.co/bsl-blog, FAQ at www.hashicorp.com/licensing-faq, and details of the license at www.hashicorp.com/bsl. * Update copyright file headers to BUS-1.1 * Rerun make gen This will pick up the last of the license changes * Revert "Rerun make gen" This reverts commit `dbe0968bff`. * Remove copywrite from generation script The current version of copywrite is not sophisticated enough to only apply the copywrite changes to certain subdirectories, so to avoid accidentally relicensing our api and sdk modules, we will stop running the copywrite script until such time that it can do the right thing. * License protobuf API as MPL The protobuf API files are used to generate the SDK, since we want the SDK to be MPL, the protobuf files must also be MPL. This extends to all protobuf files whose sources end up in the SDK directory, including: - internal/proto/controller/api - internal/proto/controller/custom_options - internal/proto/plugin It also includes the generated service directory in internal/gen/controller/api/services, since they are generated from the protobuf sources used for the SDK. * Change website license file name This makes the naming scheme consistent with the rest of the repository. * Add note on licensing to README * Set copywrite license to BUSL * Revert "Add note on licensing to README" This reverts commit `4d865a91a7`. --------- Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com> Co-authored-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>	3 years ago
Jeff Mitchell	8322916fb4	Remove all deprecated Err/Wrap/E calls (#3286 )	3 years ago
Jeff Mitchell	18e59998e9	Add storage bucket worker filter handling	3 years ago
Todd	bc48f2d0e0	Add monitor session job proto and controller changes This adds a new Job type that is for generic session state monitoring. Monitoring sessions does not include any information about specific connections.	3 years ago
Jeff Mitchell	bbe9719e19	Add tracking of worker used for injection (#3204 )	3 years ago
Elim Tsiagbey	16f48686a7	Refactor Target Repository (#3092 ) - Refactor the Target domain repository functions so that they no longer return 4 different types. `Target` struct now contains `HostSource` & `CredentialSources` with Getters & Setters - Extending `Target` to contain `HostSource` & `CredentialSources` enables `funcs` to just return `Target` instead of multiple return types. This also allows easy future refactors and extensions of `Target`	3 years ago
Jeff Mitchell	b76b24a4ad	Move prefixes for many packages into the globals package (#3069 ) This is a prerequisite for some enhancements to grant validation	3 years ago
Johan Brandhorst-Satzkorn	3c29308673	chore: Add license headers to all files	3 years ago
Irena Rindos	00a73f64d9	bug(sessions): Fix authorizeSession race conditions in handleProxy (#2795 ) * initial commit	3 years ago
Todd	b359561e51	Filter WorkerList by feature and go from version string to version.Info (#2807 )	3 years ago
Damian Debkowski	1d3930a711	feat(handlers): Support address field on a Target	3 years ago
Damian Debkowski	bca7c371d8	feat(session): Support connecting to an address associated to a Target	3 years ago

1 2 3 4

186 Commits (deeac5c457e86c6c2bf6db50f00a385ecbf4a360)