2682 Commits (efe9fe780d4094de4ac5041e7b15c24ae6104b9c)

Author	SHA1	Message	Date
Irena Rindos	dfe4f91be3	refactor(server): refactor worker tags (#5445) ... * refactor(server): refactor worker tags	1 year ago
Lauren	4731bc8747	bump-ui-commit-904864b14 (#5449)	1 year ago
Johan Brandhorst-Satzkorn	a43cb90a32	internal/session: ensure orphaned connection cleanup (#5438) ... Previously, the orphaned connection cleanup job would not detect that a connection that was created before a worker had submitted a successful session info was orphaned. We now fall back to using the worker update time as an indicator if the session info time is not set.	1 year ago
Johan Brandhorst-Satzkorn	ec0a3f964b	Worker status refactor (#5417) ... * oss: feat(proto): statistics rpc added to server coordination service * oss: feat(session): add update statistics status service * oss: feat(handlers): add Statistics method to worker service server * oss: feat(worker): add statistics ticking * oss: internal/proto: define the new RoutingInfo RPC * oss: internal/server: repository changes for RoutingInfo RPC * oss: internal/daemon: add RoutingInfo handler to worker service The new RoutingInfo RPC is used to get information from the worker that is used to make routing decisions on the controller. It also handles the initial request from the worker. * oss: feat(db): server_worker_session_info table * oss: feat(proto): session info rpc for server coordination service * oss: feat(server): worker session info request data type * oss: feat(worker): session info ticker * oss: feat(handler): server coordination handler for session info * oss: feat(session): utilize session info request time for session cleanup * oss: proto: fix some gotags comments Some of these fields were missing comments, while others had incorrectly formatted comments. * oss: worker: send connection status to server This is required for the correct tracking of orphaned connections * oss: internal/daemon/worker: add function to wait for Statistics RPC * oss: internal/daemon/worker: add routing info RPC The new RoutingInfo RPC replaces part of the Status RPC, namely the part which is used by the controller and worker to perform decisions around the establishment of new sessions and new worker connections. * oss: internal/daemon/worker: remove unnecessary error check * oss: internal/daemon/worker: lock access to config and upstreams The new confLock protects access to the conf value and to the addressReceivers slice. * oss: worker: replace Status RPC with RoutingInfo * oss: worker: deprecate the Status RPC and related types The Status RPC and related messages have been replaced by the RoutingInfo/SessionInfo/Statistics RPCs. The controller will continue to support the Status RPC until two new minor versions have been released. * oss: internal/event: add default deny for new worker RPCs * oss: internal/daemon/worker: allow RPC interval to be set The new SessionInfo/RoutingInfo/Statistics RPCs run at long intervals, which make some of our cluster tests take a long time to complete. This allows their interval to be configured in tests. * oss: all: use TestWorkerRPCInterval in slow tests This should speed up these tests significantly. * oss: internal/tests/cluster: reset sys eventer in test --------- Co-authored-by: Damian Debkowski <damian.debkowski@hashicorp.com>	1 year ago
Hugo	346ac58fa3	feat(cmd): Default to azure plugin when performing db migration (#5422)	1 year ago
Timothy Messier	71af25f584	fix(db): Remove begin when assembling migrations (#5148) ... When building the set of migrations to run as part of executing the `database migrate` or `database init`, the migrations are collected and combined to run within a database transaction. As such the `begin` and `commit` statements in the migration files should get removed from the final set of statements that get executed. However, when the migrations files were updated to include copyright and license headers, this broke the logic that would strip out the `begin` statement. While this does not cause any issue with executing the statements in a single transaction, it does result in noise in the database server's logs, with log messages like: BEGIN WARNING: 25001: there is already a transaction in progress BEGIN LOCATION: BeginTransactionBlock, xact.c:3778 This commit fixes the logic to correctly strip the `begin` for files that have the header lines.	1 year ago
Hugo	846c849589	Worker Domain Refactor (#5338) ... * feat(server): Refactor LookupWorkerByName to be a test method * feat(server): Split worker tags table * refact(worker): Export api and config tags * refact(server): Return worker id from UpsertWorkerStatus * refact(server): Create ListWorkerStorageBucketUpdates domain function * refact(server): Use dedicated HCPb-worker list func * refact(server): Create ConnectionRoute domain function * refact(handlers): Move worker selection logic to internal/server package This commit moves all the logic to select a worker to handle a session to the internal/server package. Most functions were moved to this package with no changes, however, there are a couple of notable exceptions: - Given that we don't use GRPC errors in repos, all errors were adjusted to use the internal/errors package instead. - StorageBucketFilterCredIdFn is now used to prevent the internal/server package from importing internal/storage/plugin, which would cause a dep cycle. Callers of SelectSessionWorkers must implement this func which is responsible for returning a storage bucket's worker filter as well as its associated credential id. - authorizeSessionWithWorkerFilter now receives an internal/server.(*Repository) object when called. This is transparent to SelectSessionWorkers callers as it is handled internally. * feat(server): Add FilteredWithFeatures fn and new server options * refact(server): create new list and lookup worker queries * test(server): Add WorkerList unit tests * feat(server): Implement WorkerList randomness using crypto/rand pkg * chore(db): add index to server_worker table * refact(server): prefix test method with Test * refact(server): rename Downstreamers interface to Graph * chore(server): unexport and rename method * chore: add doc string to WorkerAddress (#5420) --------- Co-authored-by: Irena Rindos <irenarindos@users.noreply.github.com> Co-authored-by: irenarindos <irena.rindos@gmail.com>	1 year ago
Johan Brandhorst-Satzkorn	1cbe1ca463	internal/event: fix log processor race (#5401) ... The cloud events formatter Process function would call the signer without protecting it, which was racy with the Rotate function.	1 year ago
Johan Brandhorst-Satzkorn	7b30697ddd	internal/session: fix race condition in connection cleanup (#5402) ... The totalClosed variable was accessed in both the Status and the Run functions, which may be called from independent goroutines.	1 year ago
Damian Debkowski	868d7b8a2d	test(daemon): remove tests with multihop (#5397)	1 year ago
David Kanney	2342e4b644	fix(target): Assign session connection limit to SessionAuthorization.ConnectionLimit (#5396) ... * fix(target): Assign session connection limit to SessionAuthorization.ConnectionLimit * test(target): Update TestAuthorizeSession & TestAuthorizeSessionTypedCredentials to check session connection limit	1 year ago
Damian Debkowski	dd1482b5ac	test(cluster): check worker status multiple times (#5393)	1 year ago
Johan Brandhorst-Satzkorn	1a6320794e	cmd: propagate keyring reading errors (#5360) ... Reading a token from the keyring can sometimes fail for reasons other than the token not being present in the keyring. This PR introduces a new error used to indicate that the token requested could not be found in the keyring and propagates errors from the keyring up to the caller, so that the caller can act on them. Additionally, this fixes a bug in the client cache where it would incorrectly assume that a token had been removed from the keyring, when in fact there was an error reading from the keyring.	1 year ago
Damian Debkowski	98bb4c8c31	test(cluster): split tests between parallel and sequential packages (#5390)	1 year ago
Johan Brandhorst-Satzkorn	1b6b22ef4f	internal/daemon/worker: improve runtime and reduce noise (#5387) ... TestRotationTicking would previously wait for 5 rotations, which is pretty slow considering each rotation takes 20 seconds. Reduce the number to 3, and use require instead of assert to remove noise during flaky test runs. This doesn't fix the flakiness of this test, but it should reduce the log spam.	1 year ago
Johan Brandhorst-Satzkorn	20f11ab01b	Revert "internal/daemon: speed up multi-worker tests (#5370)" (#5386) ... Unfortunately, this started causing the enterprise tests to break in an unexpected way. Reverting to spend some more time testing the changes.	1 year ago
Johan Brandhorst-Satzkorn	862a30c389	internal/daemon: speed up multi-worker tests (#5370) ... Instead of sleeping, we can poll for the desired state. Also, run shutdowns in parallel.	1 year ago
David Kanney	f62e67584e	chore: Clarify GrantHash behavior when there are no GrantTuples (#5334) ... * chore: Simplify appending * test(grants): De-dupe hash comparison * chore(grants): Remove TODO & add clarifying comment * test(grants): Add test cases to enforce empty grants behavior	1 year ago
Johan Brandhorst-Satzkorn	c85fad27f6	internal/event: avoid reusing canceled context (#5304) ... Don't error in ReleaseGate if the context used for the original event has been cancelled. Any error in ReleaseGate causes the controller to terminate its startup. If a user was trying to send a request to the Boundary controller as it started up, the context tied to the request would be reused to attempt logging the observation associated with the request after the logging gate was released. This would always fail, as the context associated with the request was canceled. We now use a new context timeout for events logged after the release gate is released, if the original context was canceled.	1 year ago
Bharath Gajjala	58cc322c63	Refactor redundant append operations in test setup (#5211) ... * Refactor redundant append operations in test setup - Simplified code by removing unnecessary if-else conditions in the wantUserIds and wantGroupIds blocks. Fixes #5205 * chore: fmt fix	1 year ago
Johan Brandhorst-Satzkorn	10370edf59	internal/commands/server: prevent test panic on timeout (#5249) ... Previously, if one of these tests failed to start their servers before the timeout, there was a risk that the goroutine started to run the server would invoke t.Errorf after the test itself had concluded. Replace t.Errorf with fmt.Printf to avoid the panic, and include the test name for debugging.	1 year ago
Michael Gaffney	e086fd3007	test(batch): Fix data races in tests ... Fixes: 9b984b0ab6	1 year ago
Johan Brandhorst-Satzkorn	a2be306eb4	More managed groups limit fixes (#5245) ... * internal/handlers/managed_groups: ensure membership lookup is unlimited Previously, we would only show the first 10,000 members in the group, giving the impression that a complete list was performed. Now we list all members. * internal/handlers/accounts: ensure group lookup is unlimited Previously, we would only show the first 10,000 managed group memberships. Now we list all group memberships of an account.	1 year ago
Johan Brandhorst-Satzkorn	d830bdd416	internal/auth/oidc: ensure all managed groups are tested on auth (#5242) ... Previously, we would list the managed groups with an implied filter of db.DefaultLimit (10,000), which would incorrectly remove a user from managed group memberships if there were more than 10,000 managed groups in an auth method. Explicitly set the list limit to unlimited to ensure all managed groups are updated appropriately.	1 year ago
Elim Tsiagbey	514bcc19f6	feat(gcp): add GCP plugin (#5229) ... Add GCP as a supported plugin for dynamic host catalogs	1 year ago
Hugo	f227020b67	feat(scheduler): Use job table to select jobs to run ... This commit changes the logic around selecting jobs to run. Before this, Boundary used a view "job_jobs_to_run" which was a collation of all jobs that were not running and scheduled to run. When jobs were requested to run, Boundary would look at this view, grab a certain amount of jobs and insert them into the "job_run" table. Importantly, while the amount of jobs to be scheduled at once defaulted to 1 in the scheduler package, this was entirely overriden in the controller package to -1 (no limit, run all jobs that the query would return). With this commit, Boundary now looks at the "job" table directly for jobs to run, regardless of job run status. This was done to improve DB performance as the view (which queried all "job_run" rows) was causing significant DB contention. Additionally, the option to limit job runs is now removed since it was already hardcoded to -1 and not user-configurable anyways. Should the need arise to implement this limit again, this will have to be done in a different way since the new query does not account for running jobs and limiting the rows returned after jobs are already running would yield a no-op (the new query inserts nothing into "job_run" if any given job is already running).	1 year ago
Hugo	27c432a748	feat(scheduler): Drop completed job runs as they complete ... Before this change, upon the completion of a job run, the scheduler would update a job_run entry with the completed status as well as the number of objects affected by the run. Additionally, we then had a cleaner job whose scope was deleting these completed entries from job_run. This commit changes this design to drop the cleaner job the and rolls its functionality directly into the scheduler. Now, when any given job run is complete, the scheduler will automatically delete it from the job_run table itself. In addition, the completed status has been removed as a valid status as it is not set anymore. This change follows a decision that given the data in these job_run entries is already being deleted by the cleaner job, updating its state when a job completes just to have it deleted is not efficient. Additionally, no use-case for this data was surfaced.	1 year ago
dani	62c41191ff	[fix] allow IPv6 controllers, workers, and targets (#5221) ... Co-authored-by: Damian Debkowski <damian.debkowski@hashicorp.com>	1 year ago
Michael Gaffney	9529491fa6	feat(session): Delete terminated sessions in batches ... This changes the periodic job that deletes terminated sessions to delete the those sessions in batches. Prior to this, a single SQL DELETE command was used to delete all qualifying terminated sessions. In systems under heavy load, the statement could timeout leading to a situation where no qualifying terminated sessions were ever being deleted.	1 year ago
Michael Gaffney	9b984b0ab6	feat(batch): Add batching for SQL jobs ... This adds a new `batch` package under the `scheduler` package.	1 year ago
Michael Gaffney	8c291a2934	feat(scheduler): Add retries to job status ... This adds retries as an attribute to JobStatus allowing a job to indicate liveness to the scheduler when work has not progressed. A job could use this and the statusThreshold passed into the Run method to break work up into batches that would otherwise timeout if all the work was attempted at once, for example a long running SQL delete command.	1 year ago
Michael Gaffney	9354a4ebcf	refactor(scheduler): Add statusThreshold parameter to job.Run ... The scheduler monitors running jobs to make sure they are not hung. If the status of a running job does not change after a duration (the interruptThreshold in the scheduler), the scheduler kills the job. This change passes the scheduler's interruptThreshold to the job when it calls Run() on the job.	1 year ago
Michael Gaffney	50ab600519	docs(scheduler): Clarify Job and JobStatus ... This adds more details to the Job interface and the JobStatus struct to clarify how the scheduler uses them.	1 year ago
Irena Rindos	36158056d9	feat(workers): add VerifyKnownWorkers repo function (#5189)	1 year ago
Irena Rindos	dfde64e648	chore(server): fix const ops (#5190)	1 year ago
Jim	ffd0d2a7d3	feat (cache): impl. soft-delete of users in cache (#5173)	1 year ago
Irena Rindos	7798906e4e	chore: update version and add Next to changelog post-release (#5179) ... * chore: update version and add Next to changelog post-release	1 year ago
Cameron Perera	2e3fdb718c	bump ui commit to 0ff91f9 (#5172)	1 year ago
Lisbet Alvarez	987cdb3b64	bump ui commit to 857bf01 (#5170)	1 year ago
Johan Brandhorst-Satzkorn	ab922b5786	internal/clientcache/cmd/cache: restart cache on version mismatch (#5165) ... Restart the client cache if it is found to be a version below the version of the CLI during client cache hook execution.	1 year ago
Cameron Perera	f656509739	bump ui commit to 0d55812 (#5158)	1 year ago
Cameron Perera	a7105868d1	bump ui commit to 64e568bd (#5150)	1 year ago
Timothy Messier	21230e1c61	db: Re-order migrations for next release (#5146) ... Refs: 37bf59243a Refs: 3609658edd	1 year ago
Cameron Perera	d9055bd316	bump ui commit to d3c789c66c (#5147)	1 year ago
Michael Gaffney	9119effa9e	fix(session): Fix query for orphaned connections (#5138)	1 year ago
Timothy Messier	3609658edd	perf(session): Avoid select when canceling session (#5140) ... When a resource relating to a session is deleted, notably a target, host, or auth_token, the corresponding sessions are marked as canceled via a database trigger that calls the cancel_session function. This function was performing a select as part of the insert call to ensure that it would not attempt to insert a session_sate of canceled if it was an invalid state transition. However, since this function was first introduced, there has been additional constraints added that would prevent an invalid state transition. In addition, performing a select as part of an insert increases the chances for contention with other queries that interact with the session_state table, especially since a large number of sessions and session_state rows can be included in a single transaction when things like auth_tokens or targets get deleted. Now, the cancel_function will perform the insert, and catch any of the potential violations, allowing the query to continue without needing to perform the select, thus reduction the potential for contention.	1 year ago
Irena Rindos	10a33eb7a0	Bump boundary version & update changelog (#5143) ... * Bump version to 0.18	1 year ago
Hugo	ec2d5d4285	feat(host/plugin): Support empty host catalog secrets	1 year ago
Hugo	a5237cc171	feat(host/plugin): Use plugin client pkg func on job_set_sync	1 year ago
Hugo	5b5aa484da	feat(host/plugin): Extract plugin client functionality to pkg func	1 year ago