boundary

Commit Graph

Author	SHA1	Message	Date
Todd	2ecd550e27	Add fields to server_worker to be a boundary resource (#2083 ) * Remove the server_id from session table, change it to worker_id on session_connection This is step 1 in updating the worker table. The worker is becoming a public resource so it must have a public id. We drop the server_id column on the session_connection table and add the worker_id column so we can use the wt_public_key type. Since db migrations are applied while the controller is offline, it should be safe to assume all session connections are not in a running state, thus allowing us to null out all the worker ids in the session_connection table. For now we don't have very good enforcement of the wt_public_id contraints, since the id is currently defined in the worker config, but this will soon change and the worker's id will be generated by the controller. * Updating server_id references to controller_id. * Adding worker tag db writing test. * Move the server table back to the top. * Update comment for the worker object. * Update comment for the worker table. * Update the newWorkerId call to take a context and fix its error. * Correct comment for the wt_network_address domain.	4 years ago
irenarindos	8579a248d4	refactor(servers): minor sql adjustments to worker and controller tables	4 years ago
irenarindos	9e27605213	refact(servers): Split server table into worker and controller	4 years ago
Johan Brandhorst-Satzkorn	7f9b294a7e	refact(all): Use testing.TB for test helpers Our test helper packages were not possible to use with benchmarks. This refactor switches to testing.TB to allow both tests and benchmarks to use the helpers.	4 years ago
Timothy Messier	4599e64cab	feat(event): Remove BOUNDARY_DEVELOPER_ENABLE_EVENTS env var	4 years ago
Johan Brandhorst-Satzkorn	b1d6a1da9a	Switch to buf for protobuf generation (#1944 ) * Install buf with go install This reduces the impact of using buf on our dependency closure, which was significant. It also avoids potential issues with building buf given our own internal dependency constraints. This has caused issues (including crashes!) in the past. See https://github.com/cockroachdb/cockroach/issues/67216#issuecomment-874176186 for more discussion on using tool dependencies generally, and https://github.com/bufbuild/buf/issues/52 for an example of where this sort of thing caused an issue for a user. * Upgrade buf version We should migrate to v1 sooner rather than later, as it has some breaking changes from v0. * Change buf lint/breaking commands These commands were moved up one level for v1 * Migrate to buf.yaml to v1 * Remove unused imports These were uncovered by using "buf lint". * Use buf v1.3.1 * Depend on googleapis and grpc-gateway via BSR * Switch to buf for proto generation * Remove third party dependency * Remove unused tools and scripts * Move local protos up one level * Temporarily don't fail on breaking changes * Generate CI config * Fix generation of controller.json * Replace use of deprecated flag * Remove unused import, regenerate files * Remove temporary skipping over breaking check	4 years ago
Jeff Mitchell	7eb29261b2	Update to go-kms-wrapping version 2, and plugin-based KMS (#1901 )	4 years ago
Louis Ruch	f8b27645d9	fix(scheduler): Update job run query to not error on running conflict (#1914 )	4 years ago
Louis Ruch	8cab9a4315	feat(scheduler): Update job description on conflict (#1852 )	4 years ago
Louis Ruch	c4506ec743	feat(scheduler): Add support for run now option	4 years ago
Jim	591ec476e7	refactor: Update internal/db and oplog to use go-dbw package for database operations. (#1785 ) * chore: Add new dbw dependency * refactor: Convert to use dbw.DB * refactor (db): Change GetTicket(...) signature * feature (db tests): Added TestDeleteWhere(...) Adding this allows us to eliminate a leak of the gorm library * refactor: Add ctx to ScanRows(...) signature This is required to refactor all the deprecated errors pkg calls. This refactor required adding a context to quite a few call chains and in particular the job scheduler bits * feature (db/tests): Add support for test opt: WithTestLogLevel(...) This test only option allows callers to specify a log level for the underlying db package (where supported/applicable) when calling TestSetup(...) * refactor (db): Simplify Db type Only store the underlying DB and create the RW on demand when needed. * refactor (oplog): Port to using dbw pkg for db operations * refactor (db): Remove gormDB(...) func * tests (db): Add unit tests for TestDeleteWhere(...) * chore: make fmt changes to formatting	4 years ago
Jeff Mitchell	158ab09952	Merge branch 'main' into plugin-hostcatalogs	5 years ago
Jim	8d6dee09a9	refact: Add db.DB wrapper and refact all test fixtures to use it. (#1535 ) * refact: Changes required for gorm v2 * refact: Add db.DB wrapper and refact all test fixtures to use it Stop leaking the gorm abstraction within our unit tests. * refact: Migrate direct calls to pq to pgx driver The refactor will remove the remaining direct dependencies on the pq driver which has been deprecated in favor of the pgx driver. It includes the addition of common.SqlOpen which is a wrapper around sql.Open, which is needed to deal with the driver name changing from "postgres" to "pgx" but in our code base the dialect and driver are synonyms.	5 years ago
Jim	136ac00b49	refact: Changes required for gorm v2 (#1528 )	5 years ago
Todd Knight	843d126428	Merge branch 'main' into plugin-hostcatalogs	5 years ago
Louis Ruch	1c40100edd	tests(scheduler): fix unit test deadlock (#1550 ) If a new job is started as the test successfully compeltes and begins the scheduler shutdown process. The new job will deadlock waiting on channel reads.	5 years ago
Jeff Mitchell	f9c620a5d5	Merge branch 'main' into plugin-hostcatalogs	5 years ago
Jeff Mitchell	7fc712de44	Bump protoc	5 years ago
Todd Knight	db9e1106a9	feat(host): Add host plugin related schemas and tests (#1475 ) * feat(host): Add host plugin related schemas and tests This adds the schema files for host plugins, plugin based catalogs and plugin based sets. This also adds the protos and tests which utilize gorm to ensure the schemas enforce the constraints we expect. Along with this change was a migration which moved catalog names to the base type table so we can ensure all catalogs in a single scope, regardless of subtype, have a unique name. * Updated plugin schemas to do the following: - Add plugin_executable table - plugin table now also has scope_id and name w/ uniqueness constraints - host_plugin no longer has id_prefix column - host_plugin separates name and plugin_name - plugin_version table renamed version to symantic_version column. * Fix typo in delete_plugin_subtype trigger. Add update test for host plugins. * Update id prefixes. Fix public_id typos in protos and plugin_executable name in sql diagrams. * Created internal/plugin and moved base plugin structs to it and moved host plugin subdirectory to it. * Created internal/plugin and moved base plugin structs to it and moved host plugin subdirectory to it. * Add first pass at enums for OS and Arch supported by plugin executables. * Capitalizing the const enums for OS and Architectures. * Add comments to sql files. * Reviewer comments. Remove unsupported OSes and Archs from SQL and go constants. Add missing tests. * Be more permissive in the values the db accepts. The go logic can restrict which values it will allow getting populated into the db. * Removing constraints on which values can be placed inside operating_system and architecture columns. * Updating test helper functions to only create a single catalog or set. * Move files to new migration version. * Fixing job/plugin test. * Update table names, named a number of constraints, remove plugin version and executable tables. * Add constraints for plugin name and id prefix. * Rename constraint, remove unhelpful comment block. * Adding semantic version to plugin_host and set attributes to not null. * Remove semantic versioning from all locations. * Remove semantic_version from plugin_host table comment. * Fix op copy/paste error. Don't use deprecated errors methods. Update constraint names. Add docs. * Apply suggestions from code review Fixing constraint names. Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com>	5 years ago
Louis Ruch	61722659d1	test(scheduler) Remove un-needed test causing intermittent failures (#1501 )	5 years ago
Jim	477b541bf0	Events: ongoing clean up of eventing (#1448 )	5 years ago
Todd Knight	39cc245966	Make gen after make tools. Includes make fmt of existing non generated files. (#1438 )	5 years ago
Jim	68dd2a66ba	Remove unneeded hclog dependencies as we move forward with events (#1433 )	5 years ago
s-christoff	6b78108ecf	Update primary error functions to take a context, deprecate old functions (#1358 ) * add new error funcs which take a ctx and deprecate existing funcs * use context where available when writing error events * remove circular internal/errors pkg dep * suppress some chatty errors using errors.WithoutEvent() * convert auth oidc funcs to take context where needed to write events Co-authored-by: Jim <jlambert@hashicorp.com>	5 years ago
Louis Ruch	76a7ce93e8	Fix race with base context cancel and final run update in scheduler (#1422 ) * Fix race with base contedt cancel and final run update * Use scheduler waitgroup to wait for all running jobs to exit:	5 years ago
Jim	be8c3cb1ea	prevent panics when waitgroup == nil (#1423 )	5 years ago
Jim	70ad081e39	Events: convert from hclog to events (#1409 )	5 years ago
Louis Ruch	2dae498123	Ensure job progress is always reported (#1396 ) * Update completed and total when job finishes * Remove unused option in test * Add comment * Use writer for updates and inserts	5 years ago
Michael Gaffney	df35699c4e	Integrate with Vault to retrieve and manage per session credentials (#1308 ) * Fix down migrations * Udpates due to merge * Sentinel values only have to be greater than 0 * Create credential related sdk structs and methods. * Post merge interface updating. * Removing scratch code that was not cleaned up. * updates * Convert client certificate and client certificate keys into pem blocks and validate. * Add newline at end of file. * Faster check for "s" at the end of a resource name when generating from templates. * make fmt * updates * Add dynamic credentials to a session * Refactor: move contents of file * updates * Updates * Add InvalidDynamicCredential error * Assign credentials * Allow updating client certificates seperate from the client certificate key. * Replace application purpose string with enum value * PR feedback / fix go test . * Delete dead code * Use common view * Refactor: extract interface and rename method * Refactor: move code to eliminate privPurpLibrary * Add comment to requestMap * Refactor: do not export methods on an unexported struct * Update internal/db/schema/migrations/postgres/10/03_vault_credential.up.sql Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com> * Add comments to credential Purpose constants * Reorder switch statements to handle errors first * PR feedback * update * fix merge * Organizing the scheduler code and extract the individual store creation test helper function. * Run 'make fmt' * Create certificate when creating store. Allow vault address to be passed in as option * Refactor: rename database views and associated structs This change renames two database views and associated go structs to use a consistent naming pattern. Views ending in 'private' contain encrypted credentials needed for connecting to Vault. Views ending in 'public' mirror the 'private' views but do not include any encrypted values. This change does not include any changes to functionality or behavior. * Adding SAD target operations for credential libraries to the SDK. * Add status field and enumeration table for credential status * Refactor: rename Status to TokenStatus * Add status to vault credentials * Creating the template for the CLI and helper functions. * Add private database view for vault credentials * Add scope_id to view and cleanup TODO comment * Reformat embedded sql queries * Add vault client helpers * updates * white space * Add LookupLease test * CLI commands for Store work. Renamed fields in proto w/ vault prefix. * Test issue credentials with client TLS * Add additional checks to vault ping * Add an error for vault token capabilities and create a VaultToken error kind * Add support for working with Vault policies * Use Vault 1.7.2 for testing * Add testing helper for creating a vault client with a non-root token * Fix Vault renew lease test The Vault renew lease endpoint accepts an optional duration parameter for requesting the number of seconds to extend the lease by. However, Vault is not required to honor this request. * Add a var for the required capabilities of a boundary Vault token * Enhance the test helper for adding a policy to Vault * Allow the test role in postgresql to revoke credentials * Remove out of date comments * Enhance Vault test helpers for the database secrets engine * Reformat comments * Add a method for getting the capabilities of the current Vault token * Add revoke lease to internal Vault client * Reformat: combine err return line with check for nil line * Remove redundant checks in test code Remove checks in test code that are redundant with checks in the test harness. * Refactor CreateToken test helper to return the secret and the token * Fix build failures caused by signature change in vault.CreateToken * Fix Vault version and Vault API link in comments * ran `make gen` after merging from mgaffney-vault. * Remove Vault prefix from attribute fields in API, create a base CLI option for adding field prefixes. format now uses that prefix for error reporting. * Template now allows attaching attribute field prefixes. * Credential store uses PrefixAttributeFieldErrorsWithSubactionPrefix. * Fixing naming of vault flags and helper flags. * Updating more references to generated api methods. * Make the capabilities String method more readable * Add comment outlining PrintApiError usage of Options. * Refactor test to move declarations closer to first use * Fix comment * Implement revoke method on vault repository * Tests now verify certs, CA, and tokens, addresses can be updated in a store. * Added test for creating store with pk/cert in same field. * Allo updating certificates seperate from private key. Only contact backing vault service when updating token. * Fixing bad tests and removing unused code. * Adding tests for various ways of updating tokens. * Add private credential * Add token revocation job and cred renewal/revocation job * Regen after merge. * Fixing field mask related merge error. * Create Library CLI. Removes vault prefix from path. Adds a WithoutCleanup testOption for vault. * Set all credentials to revoked when a token is revoked * Fixing missed renaming update. * fix copy/paste error. * Fixing code generation errors after merge. * make gen after merge. * Update internal/cmd/commands/credentialstorescmd/vault_funcs.go Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com> * Update internal/cmd/commands/credentialstorescmd/vault_funcs.go Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com> * Update internal/cmd/commands/credentialstorescmd/vault_funcs.go Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com> * Refactor: move migrations from 10 to 12 * Removing logic from transaction. * wrapping client errors in the calling op for CredentialStore's client() method. * Redact token * redact client key * updates * Add trigger to revoke credentials when session is canceled or terminated * Fixing some help text for some flags. * Add ability to load values from files (file://) or environment variables (env://). * Updating comments * fix bad merge * feedback * Adding issued credentials to sessions. * Error on capabilities * updates * Expose NewVaulTestServer * Add godoc * Adding generated SessionCredential struct for API. * Progress on building handler test for Authorize Session. * feedback * Update policies * Add 'revoke' as a status for Vault Tokens * Add delete_time to vault credential store * Create dynamic credentials at session creation time. * Tests pass with credentials attached to AuthorizedSession result. * Only call issue if there are credentials to issue. * Fixing output only issues. Adding type to credential library. * Fix test to clear un/pw generated. confirm generated un/pw changes from 1 authorize session call to another. * Update lookupTokenError * Add delete_time to private credential store * Refactor: move private store to separate file * Fixing tests for AuthorizeSession and verifying error cases. * Only create the credential repo if there are credentials to issue. * Rename Library field to CredentialLibrary. * Fixing missing change of Library to CredentialLibrary. * Removing Default SDK functions for fields which are required and update CLI to match. * Replace resource specific id variables with "id" and the input parmater for paths to be just a single string and the resource name as it should show up in the path. * Soft delete vault credential stores * Prevent new SQL functions from silently overwriting existing functions * Rename "Hmac" to "HMAC" in the human readable cli response table. * Fix tests * Add vault database cleanup jobs * Add ability to set the token period in the vault test harness * updates * Add not_null_columns func test * remove unused struct * Up no output timeout * Validate current vault token on addr update * Fix client cert change test * add cert test * Update revocation job where * Fix comment * Use Vault's default port in CLI help strings * Remove down migrations * Increase migration file numbers by 1 * Run make migrations * Move migrations from 12 to 10 * Increase Circle CI timeout for acceptance tests * Fix spelling mistake in error messages * ASD cmds for Target and returning credentials to AuthSession request (#1338) * Add Add/Set/Remove commands for libraries on Targets. Return credentials on AuthorizeSession request. Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com> Co-authored-by: Louis Ruch <louisruch@gmail.com> Co-authored-by: Todd Knight <T.Alan.Knight@gmail.com> Co-authored-by: Jeff Mitchell <jeffrey.mitchell@gmail.com>	5 years ago
Louis Ruch	8e4323c31c	Scheduler run jobs failure (#1310 ) * Drop wt_private_id from job table * Add server id len constraint on job_run table	5 years ago
Louis Ruch	649ed32504	Scheduler updates (#1311 ) * Scheduler updates * Add convenience methods to db/timestamp.Timestamp * Remove recursive loop Co-authored-by: Michael Gaffney <mike@gaffney.cc>	5 years ago
Michael Gaffney	094bdcdd2b	protobuf: remove unused imports (#1267 ) This removes unused imports in the protobuf files which are reported as warnings by the protoc compiler.	5 years ago
Jeff Mitchell	7ee8ddba6a	True up some spelling	5 years ago
Louis Ruch	feb5d7385d	Louis job scheduler (#1197 ) * Adds job repo * Adds scheduler * Adds plugin table which is a place holder for future expansion of the scheduler. Currently all jobs that are created are linked to the pi_system plugin which are jobs that are held in a local registeredJobs map populated at controller startup.	5 years ago

34 Commits (4b526538af660cf4c29988cd514b18fe673dc8a8)