boundary

Commit Graph

Author	SHA1	Message	Date
Damian Debkowski	d875b16767	backport of commit `6e8d6a1ed2`	1 year ago
Elim Tsiagbey	c5cc20c649	fix(worker): add GCP plugin support on the worker (#5484 ) Enable worker to run GCP plugin	1 year ago
Johan Brandhorst-Satzkorn	ec0a3f964b	Worker status refactor (#5417 ) * oss: feat(proto): statistics rpc added to server coordination service * oss: feat(session): add update statistics status service * oss: feat(handlers): add Statistics method to worker service server * oss: feat(worker): add statistics ticking * oss: internal/proto: define the new RoutingInfo RPC * oss: internal/server: repository changes for RoutingInfo RPC * oss: internal/daemon: add RoutingInfo handler to worker service The new RoutingInfo RPC is used to get information from the worker that is used to make routing decisions on the controller. It also handles the initial request from the worker. * oss: feat(db): server_worker_session_info table * oss: feat(proto): session info rpc for server coordination service * oss: feat(server): worker session info request data type * oss: feat(worker): session info ticker * oss: feat(handler): server coordination handler for session info * oss: feat(session): utilize session info request time for session cleanup * oss: proto: fix some gotags comments Some of these fields were missing comments, while others had incorrectly formatted comments. * oss: worker: send connection status to server This is required for the correct tracking of orphaned connections * oss: internal/daemon/worker: add function to wait for Statistics RPC * oss: internal/daemon/worker: add routing info RPC The new RoutingInfo RPC replaces part of the Status RPC, namely the part which is used by the controller and worker to perform decisions around the establishment of new sessions and new worker connections. * oss: internal/daemon/worker: remove unnecessary error check * oss: internal/daemon/worker: lock access to config and upstreams The new confLock protects access to the conf value and to the addressReceivers slice. * oss: worker: replace Status RPC with RoutingInfo * oss: worker: deprecate the Status RPC and related types The Status RPC and related messages have been replaced by the RoutingInfo/SessionInfo/Statistics RPCs. The controller will continue to support the Status RPC until two new minor versions have been released. * oss: internal/event: add default deny for new worker RPCs * oss: internal/daemon/worker: allow RPC interval to be set The new SessionInfo/RoutingInfo/Statistics RPCs run at long intervals, which make some of our cluster tests take a long time to complete. This allows their interval to be configured in tests. * oss: all: use TestWorkerRPCInterval in slow tests This should speed up these tests significantly. * oss: internal/tests/cluster: reset sys eventer in test --------- Co-authored-by: Damian Debkowski <damian.debkowski@hashicorp.com>	1 year ago
Hugo	846c849589	Worker Domain Refactor (#5338 ) * feat(server): Refactor LookupWorkerByName to be a test method * feat(server): Split worker tags table * refact(worker): Export api and config tags * refact(server): Return worker id from UpsertWorkerStatus * refact(server): Create ListWorkerStorageBucketUpdates domain function * refact(server): Use dedicated HCPb-worker list func * refact(server): Create ConnectionRoute domain function * refact(handlers): Move worker selection logic to internal/server package This commit moves all the logic to select a worker to handle a session to the internal/server package. Most functions were moved to this package with no changes, however, there are a couple of notable exceptions: - Given that we don't use GRPC errors in repos, all errors were adjusted to use the internal/errors package instead. - StorageBucketFilterCredIdFn is now used to prevent the internal/server package from importing internal/storage/plugin, which would cause a dep cycle. Callers of SelectSessionWorkers must implement this func which is responsible for returning a storage bucket's worker filter as well as its associated credential id. - authorizeSessionWithWorkerFilter now receives an internal/server.(Repository) object when called. This is transparent to SelectSessionWorkers callers as it is handled internally. feat(server): Add FilteredWithFeatures fn and new server options * refact(server): create new list and lookup worker queries * test(server): Add WorkerList unit tests * feat(server): Implement WorkerList randomness using crypto/rand pkg * chore(db): add index to server_worker table * refact(server): prefix test method with Test * refact(server): rename Downstreamers interface to Graph * chore(server): unexport and rename method * chore: add doc string to WorkerAddress (#5420) --------- Co-authored-by: Irena Rindos <irenarindos@users.noreply.github.com> Co-authored-by: irenarindos <irena.rindos@gmail.com>	1 year ago
Damian Debkowski	868d7b8a2d	test(daemon): remove tests with multihop (#5397 )	1 year ago
Damian Debkowski	98bb4c8c31	test(cluster): split tests between parallel and sequential packages (#5390 )	1 year ago
Johan Brandhorst-Satzkorn	1b6b22ef4f	internal/daemon/worker: improve runtime and reduce noise (#5387 ) TestRotationTicking would previously wait for 5 rotations, which is pretty slow considering each rotation takes 20 seconds. Reduce the number to 3, and use require instead of assert to remove noise during flaky test runs. This doesn't fix the flakiness of this test, but it should reduce the log spam.	1 year ago
Johan Brandhorst-Satzkorn	20f11ab01b	Revert "internal/daemon: speed up multi-worker tests (#5370 )" (#5386 ) Unfortunately, this started causing the enterprise tests to break in an unexpected way. Reverting to spend some more time testing the changes.	1 year ago
Johan Brandhorst-Satzkorn	862a30c389	internal/daemon: speed up multi-worker tests (#5370 ) Instead of sleeping, we can poll for the desired state. Also, run shutdowns in parallel.	1 year ago
dani	62c41191ff	[fix] allow IPv6 controllers, workers, and targets (#5221 ) Co-authored-by: Damian Debkowski <damian.debkowski@hashicorp.com>	1 year ago
Hugo	ee9f9fb4a1	feat(worker): Support running AWS host plugin	2 years ago
Ryan Derr	de141e34a9	Worker Connection Failure Add Controller Target (#5109 ) * ICU-11692 Add Upstream Controller Addrs When Throwing Controller Connection Failure * Reference Consistant Load Variable For Client Connection	2 years ago
Johan Brandhorst-Satzkorn	41de35f2b1	internal/daemon/controller+worker: propagate downstream timeout (#5083 ) Propagate the downstream worker timeout into the downstream receiver and downstream ticker so we have a configurable consistent timeout value to use for worker to worker interactions. Also change the atomic.Pointer to an atomic.Int64 for consistency with other values.	2 years ago
Jeff Mitchell	66af7dba98	Allow reloading some server timings (#5071 ) Allows changing grace period and a few other things timing-related via HUP instead of just a restart. Tested manually via config changes and HUP.	2 years ago
Johan Brandhorst-Satzkorn	96688b36d5	internal/daemon: simplify time update tests (#5041 ) Using time.After makes the intent clearer	2 years ago
Hugo	abb2d50494	fix(worker): Use 16-bit int parsing for TCP port	2 years ago
Sorawis Nilparuk (Bo)	350ac90a57	implement configurable getDownstreamWorker timeout (#5007 ) * implement configurable getDownstreamWorker timeout * add parsing test * typos * fix tests * fix worker tests * 0 is a valid value	2 years ago
Jim	28b465e4b8	refact: to use go-dbw CreateItems, DeleteItems (#4974 ) * refact: to use go-dbw CreateItems, DeleteItems Latest version of go-dbw updated the signature of CreateItems(...) and DeleteItems(...) on a happy note this latest version of go-dbw actually does batch inserts and updates for this functions. * all: regenerate protos with new protoc-gen-go version	2 years ago
Elim Tsiagbey	670a43ca22	feat(worker): set storage bucket states for worker status Co-authored-by: Damian Debkowski <damian.debkowski@hashicorp.com>	2 years ago
Irena Rindos	b54608b9f2	worker auth: use new previous pkix field (#4894 ) * worker auth: use new previous pkix field	2 years ago
Michael Li	ca1196cdc9	test(worker): Address flake in TestMonitorUpstreamConnectionState (#4668 ) * test(worker): Address flake * test(worker): refactor `TestMonitorUpstreamConnectionState` tests * test(worker): remove assertion for grpc shutdown state --------- Co-authored-by: Elim Tsiagbey <elim.tsiagbey@hashicorp.com>	2 years ago
dani	1a77e4da86	Boundary Plugin MinIO (#4621 ) feat(storage): Add storage support for MinIO via new built-in plugin	2 years ago
Elim Tsiagbey	a23c57e7c6	feat: add storageEventListener nil check Only shutdown the storageEventListener if the storageEventListener is not nil	2 years ago
Elim Tsiagbey	3f94c900f0	feat: Add worker event listener - add worker event listener - add worker event handler - start event listener when a worker starts	2 years ago
Elim Tsiagbey	8506ed3e2c	feat: Report Local Storage State Status - add `LocalStorageState` to worker status reports - update worker service handler to process `LocalStorageState`. If is empty for workers older workers, we set the `LocalStorageState` to `unknown` - update field mask paths to support `LocalStorageState`	2 years ago
Damian Debkowski	aea473c77b	feat: new worker config for minimumAvailableCapacity	2 years ago
Jeff Mitchell	7e214026af	Fix issue with workers connecting in high latency conditions (#4535 ) At some point we migrated to grpc ConnectParams to control how often retries were happening. One value that we didn't set was the MinConnectTimeout. It turns out that if you don't set this, it won't use the default; instead if will actually use a value of zero, which gRPC internally ups to something that seems to be around 25ms to meet a minimum threshold. This is not documented in the function comments. This commit simply sets the value of MinConnectTimeout to the unexported gRPC default.	2 years ago
Jeff Mitchell	9408b2b299	Add some more fixes caught by race detector (#4488 )	2 years ago
Jeff Mitchell	014a5a7aae	Vet/race fixes (#4421 ) This addresses all found vet/race fixes in community edition. --------- Co-authored-by: Damian Debkowski <damian.debkowski@hashicorp.com>	2 years ago
Jeff Mitchell	35cef97bd4	Linter fixes PR 3 (#4415 ) * Linter fixes * fix(bsr): update test to register TEST protocol once --------- Co-authored-by: Damian Debkowski <damian.debkowski@hashicorp.com>	2 years ago
Jeff Mitchell	25bce9e8ff	Fix linter warnings	2 years ago
Jeff Mitchell	483f13226e	Adjust tofu token check	2 years ago
Jeff Mitchell	8e71caa2e3	Address some review feedback	2 years ago
Jeff Mitchell	84c0e94097	Add a bit more useful worker auth output (#4308 )	2 years ago
Jeff Mitchell	9332179a01	Remove the deprecated old KMS worker auth mechanism (#3935 ) This has been superceded by the nodeenrollment-based KMS worker auth mechanism. This commit only removes the ability to connect with this mechanism; some upgrade logic will remain intact for a bit and can be removed later.	2 years ago
Jeff Mitchell	cb1386932d	Use sdk wspb library and remove internal (#4025 )	2 years ago
Jeff Mitchell	92f7d8cfb2	Minor test fixes (#4004 ) * Minor test fixes The auth rotation test fix allows us to keep checking if we did not yet see the expected bytes in case the call to lookup succeeds but we are late in rotating due to parallelism. The api credentials test fix allows us to check if at _least_ one rotation has happened. That way if two rotations happen before we check again due to parallelism, this is deemed a successful result. * Add timeout to auth rotation test	2 years ago
Timothy Messier	dd23f6fdbf	Update websocket library (#3908 ) The nhooyr/websocket library now has a release with the new default behavior that removes the message size limit. This allows us to switch off of the fork and back to the upstream. In the same release that includes the fix we need for the boundary client, the wspb helper package was removed from websocket. This package contained some functions to read and write protobuf messages. However, it was using the deprecated github.com/golang/protobuf/proto library. To avoid this dependancy in the library they opted to just remove wspb. Thus this provides a replacement implementation that uses google.golang.org/protobuf/proto. Notably the newer proto library seems to lack an equivalent of proto.NewBuffer so writes to the websocket and not able to leverage a sync.Pool of byte.Buffers to reduce allocations. Refs: `88bb253554` See: `ac385120c6`	2 years ago
Elim Tsiagbey	5741807b4a	fix: monitorUpstreamConnectionState CPU consumption (#3884 ) * fix: monitorUpstreamConnectionState CPU consumption `monitorUpstreamConnectionState()` is a goroutine that listens for GRPC client connection changes using GRPC's `WaitForStateChange` method. `monitorUpstreamConnectionState()` was consuming the a lot of CPU. It was continuously running and not actually waiting for state changes. The `state` for `monitorUpstreamConnectionState()` was not getting correctly updated which caused the loop to continuously run instead of waiting for new state changes. Update GRPC `WaitForStateChange` method to use the same `state` that gets updated.	2 years ago
Jeff Mitchell	fb4a5a98dc	Add some target fields and move some things around (#3879 ) In preparation for putting proxy code in the api package this creates some necessary fields in the target output and it pre-moves credential parsing and proxy handshake code to their new places. This will make the diff for the actual proxy code move quite a bit smaller. Co-authored-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>	2 years ago
Jeff Mitchell	20ac42c851	Add some useful dev flags (#3865 )	2 years ago
Jeff Mitchell	602bbcc43e	Fix intermittent startup issues on slow systems (#3803 ) With a suitably slow system -- say, if you're waiting on a Windows Terminal to output text inside a virtual machine -- the expected order of things can be thrown off. One specific way this can happen is that the worker can attempt to dial and have its dial fail because of the slowness causing the dial context to time out. If, because of the slowness, we also haven't released the log gate yet, then the event with the failure information will be queued, along with the context that was used. Unfortunately, in some of these error cases, the context that was used was the dial context instead of the system base context. In many other places in the function it was the system base context so this is just a mismatch, probably from code written at different times. Normally this wouldn't be a problem as we'd fall back to the underlying logger, but when we release the log gate, things happen differently: it's a synchronous function and on error it causes us to abandon system startup entirely. This commit fixes the issue with using the incorrect context. It's an open question whether we should change the behavior around errors when replaying queued events, falling back to the underlying logger instead of erroring.	3 years ago
dani	9413837b6a	pass storage an empty context so it can close cleanly (#3653 ) Co-authored-by: Danielle Miu <29378233+DanielleMiu@users.noreply.github.com>	3 years ago
Jim	0d19892e03	fix (events): convert multierror to stdlib error (#3743 ) * fix (events): convert multierror to stdlib error multierror errors will be converted to a stdlib error (because multierror doesn't support json marshaling). * refact (errors): convert from multierror to stdlib errors.Join(...) * refact: convert from pkg/errors to stdlib error wrapping	3 years ago
Irena Rindos	7bd2d2c913	feat(worker): allow hcp cluster id to be sourced from env var (#3709 ) * feat(worker): allow hcp cluster id to be sourced from env var	3 years ago
Irena Rindos	9733a250fa	refact(event): move observability/event to event (#3704 )	3 years ago
Elim Tsiagbey	fabd4093de	feat: Worker Connectivity Health Check (#3650 ) * feat: Worker Connectivity Health Check Add worker upstream connection state. API calls to the worker's ops `/heath?worker_info=1` endpoint should now return a new filed called `upstream_connection_state` ``` { "worker_process_info":{ "state":"active", "active_session_count":0, "upstream_connection_state":"READY" } } ``` The states returned will be GRPC connection states: https://grpc.github.io/grpc/core/md_doc_connectivity-semantics-and-api.html There could be multiple upstream addresses but GRPC manual resolver determines the state of the connection.	3 years ago
hashicorp-copywrite[bot]	29da0bcb92	[COMPLIANCE] License changes (#3567 ) * Adding explicit MPL license for sub-package. This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository. * Adding explicit MPL license for sub-package This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository. * Updating the license from MPL to Business Source License Going forward, this project will be licensed under the Business Source License v1.1. Please see our blog post for more details at https://hashi.co/bsl-blog, FAQ at www.hashicorp.com/licensing-faq, and details of the license at www.hashicorp.com/bsl. * Update copyright file headers to BUS-1.1 * Rerun make gen This will pick up the last of the license changes * Revert "Rerun make gen" This reverts commit `dbe0968bff`. * Remove copywrite from generation script The current version of copywrite is not sophisticated enough to only apply the copywrite changes to certain subdirectories, so to avoid accidentally relicensing our api and sdk modules, we will stop running the copywrite script until such time that it can do the right thing. * License protobuf API as MPL The protobuf API files are used to generate the SDK, since we want the SDK to be MPL, the protobuf files must also be MPL. This extends to all protobuf files whose sources end up in the SDK directory, including: - internal/proto/controller/api - internal/proto/controller/custom_options - internal/proto/plugin It also includes the generated service directory in internal/gen/controller/api/services, since they are generated from the protobuf sources used for the SDK. * Change website license file name This makes the naming scheme consistent with the rest of the repository. * Add note on licensing to README * Set copywrite license to BUSL * Revert "Add note on licensing to README" This reverts commit `4d865a91a7`. --------- Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com> Co-authored-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>	3 years ago
Jeff Mitchell	8322916fb4	Remove all deprecated Err/Wrap/E calls (#3286 )	3 years ago
Jeff Mitchell	929ede1c18	Modify some tests to expose error cases that weren't being exposed (#3302 )	3 years ago

1 2 3 4

191 Commits (deeac5c457e86c6c2bf6db50f00a385ecbf4a360)