boundary

Commit Graph

Author	SHA1	Message	Date
Irena Rindos	dfe4f91be3	refactor(server): refactor worker tags (#5445 ) * refactor(server): refactor worker tags	1 year ago
Johan Brandhorst-Satzkorn	ec0a3f964b	Worker status refactor (#5417 ) * oss: feat(proto): statistics rpc added to server coordination service * oss: feat(session): add update statistics status service * oss: feat(handlers): add Statistics method to worker service server * oss: feat(worker): add statistics ticking * oss: internal/proto: define the new RoutingInfo RPC * oss: internal/server: repository changes for RoutingInfo RPC * oss: internal/daemon: add RoutingInfo handler to worker service The new RoutingInfo RPC is used to get information from the worker that is used to make routing decisions on the controller. It also handles the initial request from the worker. * oss: feat(db): server_worker_session_info table * oss: feat(proto): session info rpc for server coordination service * oss: feat(server): worker session info request data type * oss: feat(worker): session info ticker * oss: feat(handler): server coordination handler for session info * oss: feat(session): utilize session info request time for session cleanup * oss: proto: fix some gotags comments Some of these fields were missing comments, while others had incorrectly formatted comments. * oss: worker: send connection status to server This is required for the correct tracking of orphaned connections * oss: internal/daemon/worker: add function to wait for Statistics RPC * oss: internal/daemon/worker: add routing info RPC The new RoutingInfo RPC replaces part of the Status RPC, namely the part which is used by the controller and worker to perform decisions around the establishment of new sessions and new worker connections. * oss: internal/daemon/worker: remove unnecessary error check * oss: internal/daemon/worker: lock access to config and upstreams The new confLock protects access to the conf value and to the addressReceivers slice. * oss: worker: replace Status RPC with RoutingInfo * oss: worker: deprecate the Status RPC and related types The Status RPC and related messages have been replaced by the RoutingInfo/SessionInfo/Statistics RPCs. The controller will continue to support the Status RPC until two new minor versions have been released. * oss: internal/event: add default deny for new worker RPCs * oss: internal/daemon/worker: allow RPC interval to be set The new SessionInfo/RoutingInfo/Statistics RPCs run at long intervals, which make some of our cluster tests take a long time to complete. This allows their interval to be configured in tests. * oss: all: use TestWorkerRPCInterval in slow tests This should speed up these tests significantly. * oss: internal/tests/cluster: reset sys eventer in test --------- Co-authored-by: Damian Debkowski <damian.debkowski@hashicorp.com>	1 year ago
Hugo	846c849589	Worker Domain Refactor (#5338 ) * feat(server): Refactor LookupWorkerByName to be a test method * feat(server): Split worker tags table * refact(worker): Export api and config tags * refact(server): Return worker id from UpsertWorkerStatus * refact(server): Create ListWorkerStorageBucketUpdates domain function * refact(server): Use dedicated HCPb-worker list func * refact(server): Create ConnectionRoute domain function * refact(handlers): Move worker selection logic to internal/server package This commit moves all the logic to select a worker to handle a session to the internal/server package. Most functions were moved to this package with no changes, however, there are a couple of notable exceptions: - Given that we don't use GRPC errors in repos, all errors were adjusted to use the internal/errors package instead. - StorageBucketFilterCredIdFn is now used to prevent the internal/server package from importing internal/storage/plugin, which would cause a dep cycle. Callers of SelectSessionWorkers must implement this func which is responsible for returning a storage bucket's worker filter as well as its associated credential id. - authorizeSessionWithWorkerFilter now receives an internal/server.(Repository) object when called. This is transparent to SelectSessionWorkers callers as it is handled internally. feat(server): Add FilteredWithFeatures fn and new server options * refact(server): create new list and lookup worker queries * test(server): Add WorkerList unit tests * feat(server): Implement WorkerList randomness using crypto/rand pkg * chore(db): add index to server_worker table * refact(server): prefix test method with Test * refact(server): rename Downstreamers interface to Graph * chore(server): unexport and rename method * chore: add doc string to WorkerAddress (#5420) --------- Co-authored-by: Irena Rindos <irenarindos@users.noreply.github.com> Co-authored-by: irenarindos <irena.rindos@gmail.com>	1 year ago
Michael Gaffney	9354a4ebcf	refactor(scheduler): Add statusThreshold parameter to job.Run The scheduler monitors running jobs to make sure they are not hung. If the status of a running job does not change after a duration (the interruptThreshold in the scheduler), the scheduler kills the job. This change passes the scheduler's interruptThreshold to the job when it calls Run() on the job.	1 year ago
Irena Rindos	36158056d9	feat(workers): add VerifyKnownWorkers repo function (#5189 )	1 year ago
Irena Rindos	dfde64e648	chore(server): fix const ops (#5190 )	1 year ago
Damian Debkowski	9576ed40dc	test: improve test to ensure ipv6 compatability (#5113 )	1 year ago
Damian Debkowski	390fca3306	test(server): validate ipv6 address for upsert worker status (#5079 )	1 year ago
Johan Brandhorst-Satzkorn	a77769fdb6	internal/server: only use non-nil tags (#4996 )	2 years ago
Jim	28b465e4b8	refact: to use go-dbw CreateItems, DeleteItems (#4974 ) * refact: to use go-dbw CreateItems, DeleteItems Latest version of go-dbw updated the signature of CreateItems(...) and DeleteItems(...) on a happy note this latest version of go-dbw actually does batch inserts and updates for this functions. * all: regenerate protos with new protoc-gen-go version	2 years ago
Irena Rindos	2191aa1d03	refact(session connection): remove session connection state table (#4617 ) * refact(session connection): remove session connection state table	2 years ago
Damian Debkowski	1be0e5d413	fix: pass along storage bucket credential id for sb update job (#4947 )	2 years ago
Damian Debkowski	c0cf5c4e1b	feat(api): return remote storage states for worker	2 years ago
Damian Debkowski	00e4c5448b	feat: workers filtered by sbc state	2 years ago
Damian Debkowski	9bc5e492f6	feat(sbc): add upsert worker storage bucket job	2 years ago
Damian Debkowski	88c67e4eea	feat(proto): define storage bucket state in worker status	2 years ago
Irena Rindos	b54608b9f2	worker auth: use new previous pkix field (#4894 ) * worker auth: use new previous pkix field	2 years ago
Irena Rindos	08f391c5d6	workerauth: satisfy NodeIdLoader interface (#4870 ) * workerauth: satisfy NodeIdLoader interface	2 years ago
Elim Tsiagbey	3bef063c72	feat: Filter workers by local storage state function - add functionality to filter worker by local storage state. - add feature constraint for local storage state	2 years ago
Elim Tsiagbey	8506ed3e2c	feat: Report Local Storage State Status - add `LocalStorageState` to worker status reports - update worker service handler to process `LocalStorageState`. If is empty for workers older workers, we set the `LocalStorageState` to `unknown` - update field mask paths to support `LocalStorageState`	2 years ago
Elim Tsiagbey	79917f74b8	feat(worker): Worker Local Storage State DB Column - Add new `local_storage_state` column to `server_worker` table which keeps track of the state of the local disk space. - The valid values for `local_storage_state` are: -- `available`: The worker local storage state is at an acceptable state -- `lowStorage`: The worker is below the minimum threshold for local storage -- `criticallyLowStorage`: The worker local storage state is below the critical minimum threshold for local storage -- `outOfStorage`: The worker is out of local disk space -- `notConfigured`: The worker does not have a local storage path configured -- `unknown`: The default local storage state of a worker. Used when the local storage state of a worker is not yet known	2 years ago
Michael Li	9eb608feeb	chore: Update protobuf to 1.33 (#4485 ) Release: https://github.com/protocolbuffers/protobuf-go/releases/tag/v1.33.0 CVE: https://nvd.nist.gov/vuln/detail/CVE-2024-24786	2 years ago
Jeff Mitchell	35cef97bd4	Linter fixes PR 3 (#4415 ) * Linter fixes * fix(bsr): update test to register TEST protocol once --------- Co-authored-by: Damian Debkowski <damian.debkowski@hashicorp.com>	2 years ago
Jeff Mitchell	4068829509	Merge pull request #4401 from hashicorp/jefferai-linter-fixes Fix linter warnings, parts 1 and 2	2 years ago
Jeff Mitchell	25bce9e8ff	Fix linter warnings	2 years ago
Jim	3af259ff69	chore: update deps (#4394 ) * chore: update deps * tests: update sqlmock tests for breaking chg * tests (handlers): sort grants in checkEqualGrants(...) sort the expected and got grants in checkEqualGrants(...) to make the test more deterministic. * tests (asciicast): update backspace enc in TestEventMarshal(...)	2 years ago
Jeff Mitchell	c4effc078b	Remove some no-longer-needed x/crypto refs (#4371 )	2 years ago
Johan Brandhorst-Satzkorn	d9c6de4e8b	deps: update runc dependency (#4349 ) Fixes GHSA-xr7r-f8xq-vfvv	2 years ago
Jim	6a37c93d77	fix: properly handle rows.Err() (#4261 ) * fix (auth): properly handle rows.Err() * fix (auth/ldap): properly handle rows.Err() * fix (auth/oidc): properly handle rows.Err() * fix (auth/password): properly handle rows.Err() * fix (authtoken): properly handle rows.Err() * fix (credential): properly handle rows.Err() * fix (credential/static): properly handle rows.Err() * fix (credential/vault): properly handle rows.Err() * fix (db): properly handle rows.Err() * fix (db/schema): properly handle rows.Err() * fix (db/migrations): properly handle rows.Err() * fix (host): properly handle rows.Err() * fix (host/plugin): properly handle rows.Err() * fix (host/static): properly handle rows.Err() * fix (iam): properly handle rows.Err() * fix (kms): properly handle rows.Err() * fix (oplog): properly handle rows.Err() * fix (pagination/purge): properly handle rows.Err() * fix (plugin): properly handle rows.Err() * fix (job/scheduler): properly handle rows.Err() * fix (server): properly handle rows.Err() * fix (session): properly handle rows.Err() * fix (target): properly handle rows.Err()	2 years ago
Irena Rindos	9733a250fa	refact(event): move observability/event to event (#3704 )	2 years ago
Jim	a16033b73a	fix (kms): move oplog DEK into its own internal kms (#3665 ) * fix (auth/ldap): fix which DEK is used for oplog during authenticate * chore: update go-kms-wrapping dep for new extras/kms changes The latest version supports setting a table name prefix for a kms which is needed to fix our oplog DEK problem * chore: update extras/kms dep This latest version supports an optional TableNamePrefix which is required to fix our existing oplog DEK issue. * fix (oplog): schema oplog DEK fix These changes will allow use to store the oplog DEK in it's own set of tables, so they won't get deleted when a scope is deleted * fix (kms): move oplog DEK into its own internal kms * tests: update tests to be compatible with oplog fix * tests: remove migration tests which are incompatible with oplog fix These tests no longer work because they are missing the required schema changes for the oplog fix	3 years ago
Jim	d5411a26f6	chore: update protobuf dep (#3669 )	3 years ago
Lars Lehtonen	ee6803d106	internal/server: fix dropped testing error (#3015 )	3 years ago
hashicorp-copywrite[bot]	29da0bcb92	[COMPLIANCE] License changes (#3567 ) * Adding explicit MPL license for sub-package. This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository. * Adding explicit MPL license for sub-package This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository. * Updating the license from MPL to Business Source License Going forward, this project will be licensed under the Business Source License v1.1. Please see our blog post for more details at https://hashi.co/bsl-blog, FAQ at www.hashicorp.com/licensing-faq, and details of the license at www.hashicorp.com/bsl. * Update copyright file headers to BUS-1.1 * Rerun make gen This will pick up the last of the license changes * Revert "Rerun make gen" This reverts commit `dbe0968bff`. * Remove copywrite from generation script The current version of copywrite is not sophisticated enough to only apply the copywrite changes to certain subdirectories, so to avoid accidentally relicensing our api and sdk modules, we will stop running the copywrite script until such time that it can do the right thing. * License protobuf API as MPL The protobuf API files are used to generate the SDK, since we want the SDK to be MPL, the protobuf files must also be MPL. This extends to all protobuf files whose sources end up in the SDK directory, including: - internal/proto/controller/api - internal/proto/controller/custom_options - internal/proto/plugin It also includes the generated service directory in internal/gen/controller/api/services, since they are generated from the protobuf sources used for the SDK. * Change website license file name This makes the naming scheme consistent with the rest of the repository. * Add note on licensing to README * Set copywrite license to BUSL * Revert "Add note on licensing to README" This reverts commit `4d865a91a7`. --------- Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com> Co-authored-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>	3 years ago
Johan Brandhorst-Satzkorn	c644396916	server: allow release version to be set in test (#3517 ) The old logic made it impossible for the caller to set the release version with WithReleaseVersion. Only set the release version when it's not explicitly set already.	3 years ago
Irena Rindos	0fcf0e21d9	chore(server): Update worker auth repository to handle dupe nodes (#3410 ) * chore(server): Update worker auth repository to use nodee DuplicateRecordError	3 years ago
Irena Rindos	a8e363888a	bug(workerAuth): allow duplicate workerAuth inserts if records match (#3389 ) * bug(workerAuth): allow duplicate workerAuth inserts if records match	3 years ago
Jeff Mitchell	8322916fb4	Remove all deprecated Err/Wrap/E calls (#3286 )	3 years ago
Jeff Mitchell	18e59998e9	Add storage bucket worker filter handling	3 years ago
Jim	6f78889947	refact: add servers.ListWorkers(...) refactor servers.(repository).ListWorkers(...) into just servers.ListWorkers(...) used independently of the repo. The method func is refactor to use the new bits as well.	3 years ago
Michael Li	357e835fa5	doc: Add missing copyright headers (#3238 )	3 years ago
Jeff Mitchell	5716a30813	Update nodeenrollment to fix TLS issue; add debug (#3231 ) * Add more logging to worker auth rotation and test flags This makes it much easier to figure out when and why rotation is happening and control it for tests/debugging. * Update auth rotation logic Because we were looking at validity period of client certs, which may not have matched the actual rotation interval, it was possible (mostly in test scenarios) for the rotation to happen multiple times on the roots before we actually hit the worker rotation interval. This changes how the interval is calculated by looking at the difference between not-after values on the certs to understand the rotation interval being used by the controller. We then use that as the starting point for calculating a next-rotation time.	3 years ago
Jeff Mitchell	77fd51fb12	Add more logging to worker auth rotation and test flags This makes it much easier to figure out when and why rotation is happening and control it for tests/debugging. Remove cert skew Cherry-picked from release/0.12.x	3 years ago
Jeff Mitchell	bdf7bf7702	Do not allow PKI-KMS workers to have names updated via API (#3143 ) Do not allow PKI-KMS workers to have names updated via API This would cause their public IDs to change, which we do not want. Descriptions are also disallowed, since those come from the config file. This also adds logic to allow upgrading from old KMS method to new, with a test. Co-authored-by: Irena Rindos <irenarindos@users.noreply.github.com>	3 years ago
Jeff Mitchell	163ce184b8	KMS-PKI Workers (#3101 ) This adds a new mechanism for worker authentication that uses the third-party arbitration of the original KMS flow by leveraging recently-added capabilities in nodeenrollment to support a registration wrapper. This allows for workers to be registered to the system with just a name and a matching wrapper, while at the same time allowing these workers to be used for private Vault access and multi-hop as they contain per-worker encryption keys and support rotation. Nodes using this mode will generate and rotate credentials in-memory (using another recently added nodeenrollment capability) so will have a new set of keys on every startup. This has the side effect of ensuring that there is never a conflict or re-use of these credentials. Note that the normal test worker flow was using the now-deprecated KMS method; there is still a test that uses this (explicitly) but all other tests using test workers that did not specify a specific PKI flow now actually use this mechanism. This means there is fairly decent test coverage in a general sense, plus the specific tests updated/added for this feature. This also adds a new KMS type that can be used to have distinct KMSes for upstream authentication vs. accepting from downstream.	3 years ago
Jim	63a2b62b85	chore (deps): update GKW and protobuf; and run "make gen" (#3111 )	3 years ago
Jeff Mitchell	b76b24a4ad	Move prefixes for many packages into the globals package (#3069 ) This is a prerequisite for some enhancements to grant validation	3 years ago
Todd	1e89be1b1a	Downstream worker connections are tracked by worker id (#2949 ) When possible downstream workers will include their worker id information in the connections they establish with the upstream using the nodeenrollment library. The tracking listener now tracks connections that are connected in this same way instead of just connections of type *tls.Conn.	3 years ago
Johan Brandhorst-Satzkorn	3c29308673	chore: Add license headers to all files	3 years ago
Todd	d8aea569c5	Only return live workers in ListHcpbWorkersResponse (#2876 )	3 years ago

1 2

78 Commits (efe9fe780d4094de4ac5041e7b15c24ae6104b9c)