boundary

Commit Graph

Author	SHA1	Message	Date
Michael Gaffney	8159ddb08e	feat(db): Add function to update billing snapshot table This adds the `update_sessions_pending_daily_snapshot` SQL function which updates the `sessions_pending_daily_snapshot` table by inserting values for any days missing from the table. This also adds a check constraint to the `sessions_pending_daily_snapshot` table along with a `comment on table`. This also removes the hcp daily billing views since they are no longer needed. Ref: `ace2def49d`	3 years ago
Michael Gaffney	41bdfb4406	fix(db): Fix time zone consistency issues in monthly billing views The hcp monthly billing views build a range of buckets using the `generate_series` and the `now()` functions along with relative offsets like `+ 1 month`. The calculated start and end times for these views can vary depending on the time zone setting of the PostgreSQL server. This adds 3 table functions that call their respective views with the time zone set to UTC. Calling these functions instead of their underlying views directly prevents the returned values from varying when the server's time zone is not set to UTC. This also drops support for PostgreSQL 11. The included tests use the `date_trunc(field, source [, time_zone ])` function added in PostgreSQL 12. Fixes: `41b1b5b4b7`	3 years ago
Michael Gaffney	47ddd2e40a	fix(db): Fix generated series to yield consistent results The hcp billing `_all` views built a range of buckets using the `generate_series` function using the minimum value of the `session_pending_time` column from the `wh_session_accumulating_fact` table as the start time and `now()` as the end time. It then truncated values in the series using the `date_trunc` function. This allowed the generated series of buckets returned to vary and could result in one additional bucket being included in the series depending on the current time and the start time. As an illustrative example, the `hourly_all` query used: select date_trunc('hour',time) from generate_series( (select min(session_pending_time) -- start time from wh_session_accumulating_fact), now(), '1 hour'::interval ) as time; Substituting the start time with a hard coded value to show the issue: -- Show current time, where the minutes portion is 55 select now(); now ------------------------------- 2023-05-22 14:55:40.567605+00 (1 row) -- generate series with start time's minutes portion at 57 select date_trunc('hour',time) from generate_series( '2023-05-22 01:57:00'::timestamptz, now(), '1 hour'::interval ) as time; date_trunc ------------------------ 2023-05-22 01:00:00+00 2023-05-22 02:00:00+00 2023-05-22 03:00:00+00 2023-05-22 04:00:00+00 2023-05-22 05:00:00+00 2023-05-22 06:00:00+00 2023-05-22 07:00:00+00 2023-05-22 08:00:00+00 2023-05-22 09:00:00+00 2023-05-22 10:00:00+00 2023-05-22 11:00:00+00 2023-05-22 12:00:00+00 2023-05-22 13:00:00+00 (13 rows) -- Wait a few minutes select now(); now ------------------------------ 2023-05-22 14:58:48.75287+00 (1 row) -- Now run again and get an extra result, even though still -- within the same hour select date_trunc('hour',time) from generate_series( '2023-05-22 01:57:00'::timestamptz, now(), '1 hour'::interval ) as time; date_trunc ------------------------ 2023-05-22 01:00:00+00 2023-05-22 02:00:00+00 2023-05-22 03:00:00+00 2023-05-22 04:00:00+00 2023-05-22 05:00:00+00 2023-05-22 06:00:00+00 2023-05-22 07:00:00+00 2023-05-22 08:00:00+00 2023-05-22 09:00:00+00 2023-05-22 10:00:00+00 2023-05-22 11:00:00+00 2023-05-22 12:00:00+00 2023-05-22 13:00:00+00 2023-05-22 14:00:00+00 (14 rows) To fix this, we call the `generate_series` function truncating the start time argument to the bucket size: -- Hourly select bucket from generate_series( date_trunc('hour', select min(session_pending_time) from wh_session_accumulating_fact)), now(), '1 hour'::interval ) as bucket; -- Monthly select bucket from generate_series( date_trunc('month', select min(session_pending_time) from wh_session_accumulating_fact)), now(), '1 month'::interval ) as bucket; Fixes: `41b1b5b4b7`	3 years ago
Michael Gaffney	053e74f66c	feat(db): Add table for census upload job status	3 years ago
Michael Gaffney	8bf80e0544	test(db): Conditionally skip monthly billing view tests Certain tests fail when run on the first day of the month because of edge cases in the setup of test data. This adds conditional logic to skip those tests when run on the first day of a month.	3 years ago
Jim	d4e537c88d	chore: updgrade pgx direct deps to v5 (#3258 )	3 years ago
Jim	536551dc0c	fix: update go-dbw dep and refactor conversion of pgx errors (#3242 )	3 years ago
Timothy Messier	61a37ef4a4	fix(schema): Add missing endpoint column to session recording view	3 years ago
Timothy Messier	f53d65eacd	refact(sql): Rename migrations to avoid number conflicts	3 years ago
Todd	4063880bbe	Add credential history views	3 years ago
Michael Gaffney	a783c91b2d	feat(sql): Add dynamic credential hst tables to session recording This adds the `recording_dynamic_credential` join table and a trigger that runs as an after insert on the `recording_session` table. The join table joins the `recording_session`, `credential_vault_store_hst`, `credential_library_history_base`, and the `credential_purpose_enm` tables.	3 years ago
Michael Gaffney	3769f0851c	test(sql): Refactor the colors persona This changes multiple IDs in the colors persona to be 12 characters. It also renames some IDs in an attempt to make them more consistent. This also changes the order of the columns in some insert statements to follow the pattern of the other insert statements. This also adds documentation to the top of colors persona file in an attempt to explain the general patterns being followed in the file.	3 years ago
Michael Gaffney	03b1291b34	feat(sql): Add history tables for dynamic credentials This adds history tables for `credential_vault_store`, `credential_vault_library`, and `credential_vault_ssh_cert_library`. This also adds a base table for the credential library history tables.	3 years ago
Todd	e1540be9f3	Add host information to session recording view	3 years ago
Michael Gaffney	a8224f4f57	feat(sql): Add endpoint from session table to session recording This adds an `endpoint` column to the `recording_session` table and updates the session recording insert trigger to copy the value of `endpoint` from the `session` table to the `recording_session` table.	3 years ago
Michael Gaffney	06e4e46248	feat(sql): Add static credential hst tables to session recording This adds the `recording_static_credential` join table and a trigger that runs as an after insert on the `recording_session` table. The join table joins the `recording_session`, `credential_static_store_hst`, `credential_static_history_base`, and the `credential_purpose_enm` tables.	3 years ago
Michael Gaffney	3a64fb3d58	feat(sql): Add history tables for static credentials This adds history tables for `credential_static_store`, `credential_static_json_credential`, `credential_static_username_password_credential`, and `credential_static_ssh_private_key_credential`. This also adds a base table for the static credential history tables.	3 years ago
Todd	386eac072c	Add recording_channel_ssh_aggregate view	3 years ago
Johan Brandhorst-Satzkorn	11893a8885	db: add state and error_details to recording	3 years ago
Johan Brandhorst-Satzkorn	af8e4d63be	db: add channel summary details	3 years ago
Todd	ee6a5f05b5	Add historical target and scope information to session recording	3 years ago
Michael Gaffney	477e6df269	feat(sql): Add target and host history tables to session recording This adds four columns to the `recording_session` table which are foreign keys to the `iam_scope_hst`, `target_ssh_hst`, `host_history_base`, and `host_catalog_history_base` tables. The foreign key to the `iam_scope_hst` table references the project scope of the target at the time of the recording.	3 years ago
Michael Gaffney	73b710cd06	refactor(sql): Rename session recording migration This renames the migration with the session recording table to come after the history table migrations. This allows the session recording table to have foreign key references to the history tables without using alter table statements.	3 years ago
Michael Gaffney	37c6b8e540	feat(sql): Add history tables for hosts This adds history tables for `static_host` and `host_plugin_host`. This also adds a base table for the host history tables. This also adds a new test to the history table test suite. The new test verifies the contents of each history table against the contents of their operational table.	3 years ago
Michael Gaffney	b97dc07bb0	feat(sql): Add history tables for host catalogs This adds history tables for `static_host_catalog` and `host_plugin_catalog`. This also adds a base table for the host catalog history tables and an immutable table with one row for when a target has a direct address assigned to it. This also adds a reusable trigger function for tables that are immutable.	3 years ago
Michael Gaffney	7ebe393aee	feat(sql): Add history table for target_ssh This adds a history table for `target_ssh`. This also fixes a bug in the history table test suite. Prior to this change, the operational table name was calculated from the history table name using the SQL trim() function. The SQL trim function as used returned 'targe' for 'target_ssh_hst' instead of the expected value 'target_ssh'.	3 years ago
Todd	d365123045	add session recording aggregate view	3 years ago
Louis Ruch	7263d08bc5	feat(storage): Add target storage_bucket relation	3 years ago
Timothy Messier	21f6be9ab5	refact(sql): Move migrations to avoid number conflict	3 years ago
Michael Gaffney	882b7eca2f	feat(sql): Add IAM history table references to session recording This adds two columns to the `recording_session` table. One column is a foreign key to the `iam_user_hst` table and the other is a foreign key to the `iam_scope_hst` table. The foreign key to the `iam_scope_hst` table reference the IAM scope the user belongs to at the time of the recording.	3 years ago
Michael Gaffney	9fd01a79f0	refactor(sql): Move and rename session recording migration This moves and renames the migration with the session recording table to come after the history table migrations. This allows the session recording table to have foreign key references to the history tables without using alter table statements.	3 years ago
Michael Gaffney	4488f5f8cd	refactor(sql): Move and rename history table migrations This moves and renames the history table migration files into the correct migration directory.	3 years ago
Michael Gaffney	463f1e36af	feat(sql): Add history table for iam_user This adds a history table for `iam_user`.	3 years ago
Michael Gaffney	8ae658a570	feat(sql): Add history table and triggers for iam_scope This adds a history table for `iam_scope`. It also includes reusable triggers to maintain the included history table and future history tables. This also includes a test suite that runs on any table ending in `_hst`.	3 years ago
Michael Gaffney	327d950a1e	feat(sql): Add function to generate a URL safe ID This adds a SQL function that generates a secure random ID of at least 14 characters suitable for use as a public or private ID. IDs generated with the function will only contain characters from the unreserved character set defined in RFC 3986 Section 2.3 making them safe for use in a URL.	3 years ago
Michael Gaffney	94f77d36b3	feat(sql): Add a domain type for URL safe IDs This creates a new domain in SQL that constrains the characters of an ID to the unreserved character set defined RFC 3986 Section 2.3.	3 years ago
Michael Gaffney	647cf0d33d	feat(sql): Add URL safe base64 encoding function This adds a SQL function to encode binary data into the URL safe base64 encoding defined in RFC 4648.	3 years ago
Danielle Miu	a4a14fc7fc	merged commit for plugin rework, storage bucket sql and proto, storage bucket secret rewrapping, storage bucket repository	3 years ago
Johan Brandhorst-Satzkorn	7bb44b46f5	internal/db: add session recording tables Add the following new tables used for storing session recordings: * recording_session * recording_connection * recording_channel * recording_channel_ssh * storage_bucket	3 years ago
Michael Milton	ace2def49d	Add daily sessions pending view and accompanying snapshot table (#3217 ) * feat: add a new daily billing view * update views and table based on early feedback * address review comments * update CTE to be an accurate result of what we expect * remove 'with time zone' from daily range * remove timezone * Update internal/db/schema/migrations/oss/postgres/70/01_hcp_billing_daily.up.sql Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Timothy Messier <tim.messier@gmail.com> * go mod updates * fix tests --------- Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com> Co-authored-by: Timothy Messier <tim.messier@gmail.com>	3 years ago
Jeff Mitchell	bbe9719e19	Add tracking of worker used for injection (#3204 )	3 years ago
Haotian	762e0d7d0d	fix(sql): move added file to correct folder (#3197 ) * move rename_constraints.up.sql from 67 to 69 * update sql replacement reference comments	3 years ago
Haotian	b806297afe	style(db): rename constraint names to follow sql style guide (#2973 ) * programatically rename unique and foreign key constraints to enforce 'tablename_uq' and 'reftable_fkey' patterns * manually rename unique constraints for names over 63 characters * update changed constraint names in db update testcases	3 years ago
Jeff Mitchell	20391e3503	Add default client port to targets and use in connect command (#2767 )	3 years ago
Johan Brandhorst-Satzkorn	ecdb77b584	internal/db/sqltest: fix sessions aggregate (#3152 ) The sessions aggregate was inserting host_id and host_set_id, columns which are no longer present on the session table. Also fix the test that guards against regressions like this in the future.	3 years ago
Hugo Vieira	2ff9b8efac	feat(host/plugin): Support External Name field The new `ExternalName` field is an optional field that is used to store the cloud providers' human-identifiable name for a given compute instance. This field is not user-editable, but it is kept in sync with the cloud provider's through Boundary's `SetSyncJob`.	3 years ago
Johan Brandhorst-Satzkorn	bc31892f29	internal/db: add set_once_columns function (#3117 ) * internal/db: add set_once_columns function Also replace existing before_soft_delete_credential_vault_store function and trigger with the new function. * internal/db/schema: add note to replaced definitions	3 years ago
Jim	63a2b62b85	chore (deps): update GKW and protobuf; and run "make gen" (#3111 )	3 years ago
Jim	86192f75eb	feature (auth/ldap): add LDAP auth method along with associated accounts and managed groups (#2912 ) * feature (auth): required schema changes for auth ldap method (#2669) chore (auth/ldap): move schema changes to next avail migration number * feature (auth/ldap): define AuthMethod and all its value objects (#2703) * feature (auth/ldap): storage protos * feature (auth/ldap): define AuthMethod and all its value objects * feature (auth/ldap): add repo and reading an auth method (#2718) * feature (auth/ldap): add Repository.CreateAuthMethod(...) and Repository.DeleteAuthMethod(...) (#2724) * feature (auth/ldap): add Repository.UpdateAuthMethod(...) (#2739) * feature (auth/ldap): add Account * feature (auth/ldap): add AuthMethod.EnableGroups * fix (auth/ldap): refactor AuthMethod.oplog to enforce proper constraints * feature (auth/ldap): add Account repo functions * refactor (auth/ldap): remove entry attributes from account Realized the entry attributes could have absolutely anything in them (including binary data) and since we absolutely don't have to have them there's just no reason to take on the risk. * feature (auth/ldap): add AuthMethod.UseTokenGroups * feature (auth/ldap): add Authenticate(...) * refactor (auth/ldap): ensure options take a context as the 1st parameter * feature (auth/ldap): add Account attribute maps * chore (auth/ldap): make fmt deltas * feature (auth/ldap): add managed groups (#2760) * tests (auth/ldap): add missing unit test to Repository.DeleteAccount(...) Add bits to test the delete operation when you're not able to generate oplog metadata * feature (auth/ldap): add managed groups fixup! feature (auth/ldap): add managed groups (#2760) * feature (auth/ldap): service proto definition (#2761) * feature (handlers/authmethods): add handlers for ldap auth method operations (#2794) * feature (auth/ldap): add Account attribute maps * chore (auth/ldap): update cap/ldap to latest version * feature (auth/ldap): add ldap api generation definitions * feature (authmethods): add ldap repo NewService(...) * feature (authmethods/ldap): add proper mask_mapping to protobufs * feature (authmethods): add support to get an ldap auth method * refactor (auth/ldap): export TestGenerateCA(...) * feature (authmethods): add support to create an ldap auth method * feature (authmethods): add support to delete an ldap auth method * feature (authmethods): add support to list ldap auth methods * refactor (auth/ldap): make urls optional for NewAuthMethod(...) * refactor (auth/ldap): export ldap.TestInvalidPem * chore: make fmt changes * fix (auth/ldap): properly handle group search config Add constraints and tests to ensure when an ldap AuthMethod.EnableGroups is true, and UseTokenGroups is false; that there's a GroupDn configured for finding a user's associated groups * feature (authmethods): add support to update an ldap auth methods * chore (db/ldap): tmp mv migrations so there's no conflict with ongoing work * feature (verifier): add ldap auth method to verifier bits * fix (controller): prevent panic when controller stops when there's no listener * feature (authmethods): add support to authenticate via ldap auth methods * chore (migrations): fix whitespace in stmt * chore: fmt fixup * tests (auth/ldap): invalid err msg * feature (cli/authmethods): add support for ldap auth-methods CRUD and authenticate (#2810) * feature (authmethods): add CLI support for ldap auth methods CRUD * tests (api/auth): ldap auth method classification tests * feature (authmethods): add CLI support for ldap auth authenticate * feature (auth/ldap): set request timeouts for ldap server connections * feature (handlers/authmethods): handle u_anon listing properly. * feature (account/handers): ldap account and managed group CRUDL APIs (#2852) * feature (auth/ldap) add repository Listing of ManagedGroupMemberAccount * feature (controller/handlers): add ldapRepoFn to accounts service * feature (auth/ldap) register ldap managed group subtype * feature (account/handlers): ldap account CRUDL APIs * feature (controller/handlers): add ldapRepo to managed groups service * feature (account/handlers): ldap managed group CRUDL APIs * docs (domain): add LDAP accounts, auth-methods and managed groups (#2857) * feature (ldap/cli) add ldap accounts and managed groups CRUDL commands (#2856) * fix (handlers/authmethods): fix ldap authorized actions (#2892) * feature (cli/ldap/authenticate): use primary auth method if none is provided (#2890) * fix (auth/ldap): support setting the state attribute * feature (cli/ldap/authenticate): use primary auth method if none is provided * feature (wh/ldap) add tests for new ldap auth method and accounts (#2919) * refactor (migrations/ldap): mv to correct directory * chore: add copyright headers * fix (api/authmethods/ldap): renumber new LdapAuthMethodAttributes field * fix (auth/ldap): allow the auth method state to be updated (#2951) * chore: update sdk and api versions for llb - this is tmp until merging * tests (managed groups): add required errContains for new test	3 years ago
Irena Rindos	ab0a2ad689	Fix hcp billing test (#3045 ) * fix test failure	3 years ago
Irena Rindos	860af44556	Fix sql tests failing on the first of the month (#3042 ) * initial commit	3 years ago
Hugo	41b1b5b4b7	feat(db): Introduce migrations for ssh targets (#2960 ) Adds the schema for ssh targets. This also changes the target listing func to not error out when it encounters a target with an unknown type. Instead, it now ignores it and continues to the next one.	3 years ago
Timothy Messier	5c19dba3c9	ci(sqltest): Add github action workflow for sql tests (#2977 )	3 years ago
Johan Brandhorst-Satzkorn	3c29308673	chore: Add license headers to all files	3 years ago
Timothy Messier	6a41bf1237	feat(wh): Support vault ssh certificate credentials This expands the wh_credential_dimension to include fields for the new vault ssh certificate credential library.	3 years ago
Danielle	3a78a74571	feat(vault): Support Issuing SSH Cert Credentials Allows the new credential library type to be pulled from the db and used for issuing ssh certificate type credentials. The ssh cert library interfaces with vault's ssh secrets engine to either issu certificates directly from vault, or to generate them within boundary and have vault sign them as a certificate authority for authentication. See: https://developer.hashicorp.com/vault/api-docs/secret/ssh#sign-ssh-key https://developer.hashicorp.com/vault/api-docs/secret/ssh#generate-certificate-and-key	3 years ago
Haotian	823b8cdfc6	feat(vault): Add CRUDL methods for ssh certificate cred library (#2765 ) This defines a new credential library type for ssh certificates. It adds new repository methods to support CRUDL of the new ssh certificate credential library. Co-authored-by: Timothy Messier <tim.messier@gmail.com>	3 years ago
Johan Brandhorst-Satzkorn	55259029f1	Add job run cleaner (#2866 ) * chore: remove duplicate import * feat: Add job run cleaner The new job cleans job runs older than a configured interval. Configure the job to clean jobs older than 1 minute.	3 years ago
Irena Rindos	2e3fd041ef	Target worker filters migration fix (#2859 )	3 years ago
Timothy Messier	e2248c1360	fix(wh): Associate session fact with credential dimensions (#2787 ) The wh_upsert_credential_group function was incorrectly attempting to update the session connection fact table. There are a couple problems with this: 1. At the time that this runs, there would be no session connection facts for this session. 2. This function is meant to update the session fact table, there is a separate function that associates the credential group with the session connection facts. This also fixes another bug that was causing expired rows of the credential dimension to be included in the group membership. And finally it fixes a typo in the function name.	3 years ago
Damian Debkowski	a8ee18e177	fix(db): target address relabel migration id (#2769 )	3 years ago
Damian Debkowski	0dbe18e453	fix(db): remove migration test 59_01 This test uses the target domain repository code to test the ingress & egress worker filter. With our changes we have updated the target domain repository logic to fetch the target address. When this migration test is invoked, the target_address table does not exist and the domain code will always return an error making this test invalid.	3 years ago
Damian Debkowski	3dd4b1c63d	feat(db): Support a direct address association to a Target	3 years ago
Jim	1c9ca14a8e	fix (db): Improve error handling for commit errors. (#2728 ) We were not capturing the errors which happen on commit. We've changed to a multierror so we should get the commit error and any subsequent rollback errors in the same txn.	3 years ago
Danielle	46037cf85a	Add Staticcheck to Build Pipeline (#2421 ) Run staticcheck linters on pull requests	3 years ago
Johan Brandhorst-Satzkorn	eb113b5c9e	fix(db): Reorder migrations (#2688 ) Migration numbers need to be consistent between version upgrades. The migrations for the key lifecycle management were merged into v0.11.1 as number 56, but are currently 58 in main. This reorders them down to 56, fixing an issue when migrating from 0.11.1 to main.	3 years ago
Irena Rindos	834a2a88f7	feat(targets): Addition of egress and ingress worker filters (#2654 ) * feat(targets): Addition of egress and ingress worker filters	3 years ago
Haotian	96a630905f	formatting consistency for 'create function' statements (#2620 )	3 years ago
Johan Brandhorst-Satzkorn	8908dccf6d	Forward port some release fixes (#2631 ) * fix(worker): Verify contents of loaded session (#2629) * fix(worker): Verify contents of loaded session Previously, it was possible for an empty private key to be used when the session had no user or project associated with it. The new checks guarantee that we will fail gracefully in these cases. * Check against len instead of nil * Add CHANGELOG * fix(session): Always invoke all parts of session cancel trigger The session cancelation trigger would not set the key_id to null appropriately if another one of the fields on the session was already null, since that case statement would be matched first. The new structure matches all statements, in case any of them have special logic (such as in the project case).	3 years ago
Johan Brandhorst-Satzkorn	edd323b73a	Key Rotation/Destruction (#2477 ) (#2607 ) * Key Rotation/Destruction (#2477) * fix: Correct kms.CreateKeys comment This method does not return anything. * feat(kms): Add new ListKeys method The new ListKeys method wraps the underlying KMS wrapper, returning all the keys in the scope specified. * feat(scopes): Add new scope actions for key management Adds list-keys, rotate-keys and revoke-keys actions for scopes. * feat(scopes): Add ListKeys API endpoint This new endpoint lists all keys in a scope * feat(api): Add ListKeys to Scopes client This has to be a custom function because it is a custom action and doesn't map well to any of the existing templates, or generalises well in a way that would make it reasonable to create a new template. * feat(cli): Add new scopes list-keys command * add RotateKeys function to the kms repository * Add Rotate Keys Endpoint to Scopes Api (#2360) * add rotate keys endpoint * add tests for RotateKeys endpoint * remove that one comment I forgot about * addressing more PR comments * Add Rotate Keys CLI Command (#2395) * add cli command and client * add bats tests * fix some info text and pr comments * write a new rotate keys description * Add Missing DEK Foreign Keys (#2408) * add missing fks * mark key as nullable in gorm * update tests to use a new wrapper with a real key_id * fix formatting in test and add keys for auth_token test * replace key values * revert postgres_40_01_test.go * style: reformat sql for readability * refactor: change type of key_id columns to kms_private_id * Store key_id and scope_id with oplog entries, re-type columns referencing data key version (#2431) * fix(db): Correct types of data key version referencers * fix(oplog): Fix missing error handling * feat(oplog): Store key_id and scope_id with oplog entries This migration truncates all existing oplog entries, as migrating existing data would have been complex and likely unnecessary. * Update view references * Always delete oplog entries when dropping keys * Fix rebase issues * Fix sql tests * Add schema for new kms destruction jobs (#2479) * feat: Add schema for new kms destruction jobs The schema lets us manage destruction jobs per-table while providing a guarantee that only one run is running at a time. * Apply suggestions from code review Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com> * Fix table reference and tests Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com> * update domain, api, and cli to all include key versions in outputs (#2472) * update domain, api, and cli to all include key versions in outputs * fix go sum * add migration to fix immutable error * fix final test issue as well as make gen issue * rebase and fix * address johan pr comments * make gen, didn't realize comments were included * update migrations based on new migration 55 file 03 * Move migrations to 56 * feat: Add key version rewrapping function registry (#2478) The new RegisterTableRewrapFn can be used by a domain to register a callback to use when rewrapping data in a specific table name. * feat(kms): Add scopeId to RewrapFn (#2539) This makes it much easier for rewrap functions to look up the correct wrapper and limit the number of rows searched. * Update Root Cert Proto Definition (#2542) * update root cert proto definition to properly identify public_key as the table primary key * add the test fix as well * Rewrap Functions for Encrypted Tables (#2532) * add oidc rewrap * add argon2 config rewrap * add auth token rewrapping * add worker auth cert rewrapping * check the err * add username password credential rewrapping * add ssh private key credential rewrapping * add vault client certificate rewrap function * add host catalog secret rewrap function * cleanup a little * add host vault token rewrap function * add session credential rewrap function * add session rewrap function * add worker auth rewrap function * update test to include full assert gamut * add session rewrap function TEST * bring all tests up to current standards, make more readable, and include full assert gamut * rework all rewrap functions to utilize scope id * update comments, queries, sql refs, etc, in response to PR comments * forgot a comment * remove hmac updates and fix a couple comments again * Rewrapping Registered Tables Test (#2555) * add the registered table test as well as the two missing rewrap functions * address pr comments, add db r/w conversions, cmp.diff, sql comment, remove (now) faulty immutable test * Add ability to list key version destruction jobs, recurring jobs and destroying key versions (#2498) * feat(kms): Add ability to list data key version destruction jobs * feat(scopes): Add ListKeyVersionDestructionJobs * feat(cli): Add list-key-version-destruction-jobs * feat(kms): Add recurring job that performs table rewrapping The new recurring job runs for each job table with a registered rewrapping callback. It attempts to become the running run and start its rewrapping, gracefully handling sudden interruptions by resuming it's work if it finds evidence of sudden interruption. * feat(kms): Add recurring job for destroying key versions The new job monitors the database for data key version destruction jobs that have finished rewrapping all its data, performing the final data key version revocation. * feat(kms): Add DestroyKeyVersion DestroyKeyVersion immediately destroys a key version if it is a root key version or a data key version which encrypts no data. If the data key version encrypts data which needs to be rewrapped, it queues an asynchronous job to complete the rewrapping operation. * feat(scopes): Add DestroyKeyVersion API endpoint * feat(cli): Add scopes destroy-key-version CLI command * Ensure all secret updates include key ID (#2556) * feat(all): Ensure all secret updates include key ID When updating a secret, it is paramount that the key ID is also updated, as it is the only means we have of ensuring that we only destroy keys once there is no more data associated with the key. * Remove fix to session repository for now This breaks the current private key derivation method * add new encrypt/decrypt funcs and made necessary proto changes (#2557) * add new encrypt/decrypt funcs and made necessary proto changes * address all PR comments * fix test panic * add param checking to rewraps * fix(schema): Correctly organize migrations again * Review comments part 1 * Review comments part 2 * Review comments part 3 * Store cert private key, encrypt tofu token (#2583) * fix(session): store cert private key, encrypt tofu token Previously, we would derive a session certificate private key from the session key used in each project, and not store the key. We would also encrypt the tofu token with the database key but not store a reference to this key in the database. This change fixes this by replacing our key derivation with a key generation step, and instead store the generated key, encrypted, in the database. We also ensure that any new tofu tokens are encrypted with the same key. To handle existing sessions, we lazily rewrite the sessions whenever a user lists them. There is a minor risk that a user could end up destroying the database key that encrypts the tofu token before that session has been rewrapped, but this is going to be rare and temporary. * feat(session): Allow the random source for secrets to be configured * Review comments * Minor fixes * More minor fixes * More small fixes Co-authored-by: Danielle Miu <dani.miu@hashicorp.com> Co-authored-by: Danielle <29378233+DanielleMiu@users.noreply.github.com> Co-authored-by: Michael Gaffney <mike@gaffney.cc> Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com> * fix(session): Unset session key ID when scope is deleted Unsetting the key ID allows the session to live beyond the lifetime of the scope, while allowing the scope to cascade delete its keys. * fix(migrations): Rewrite migration 56 tests These tests can no longer use the normal test helpers as they try to use the new oplog table structure, so we have to manually write all the interactions. * fix(migrations): Rename migration 58 file name This was the name used in the release branch, update it to the new migration number. * fix(e2e): Add some comments and debug to kms tests * fix(session): Handle empty project and user IDs in read When a project or user is deleted, the session is automatically canceled and the respective field on the session is unset. LookupSession did not handle this case gracefully, and would error on decryption in this state. Skip decrypting sessions that do not have either a user or project to avoid this error. The decrypted values are not used for a canceled session. * Fix worker test * Increase test timeout Co-authored-by: Danielle Miu <dani.miu@hashicorp.com> Co-authored-by: Danielle <29378233+DanielleMiu@users.noreply.github.com> Co-authored-by: Michael Gaffney <mike@gaffney.cc> Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com>	3 years ago
Haotian	d97d34adc4	refactor(db): applies consistent formatting to 'comment on' statements	3 years ago
Johan Brandhorst-Satzkorn	a216c701ee	Silence events from DB errors (#2601 ) (#2602 ) If callers want errors to create events, they can do so by wrapping the error. Sometimes we don't want db errors to be events, and before this there was no way to supress them.	4 years ago
Irena Rindos	b26814a3cc	move user variables into globals (#2580 ) * move user variables into globals	4 years ago
Michael Gaffney	f114e20c6d	docs(db): fix comments on database domain types Fixes the comments on the wt_scope_id, wt_user_id, and wt_role_id domain types.	4 years ago
Michael Gaffney	ed4b00303a	docs(db): fix comments on kms database tables Fixes the comments on the kms_data_key and kms_data_key_version tables.	4 years ago
Michael Gaffney	c9cfb0e564	docs(db): fix comments on oidc database tables Fixes the comments on the auth_oidc_method and auth_oidc_account tables.	4 years ago
Michael Gaffney	3241c9a411	docs(db): fix comments on database functions Fixes the comments on functions wt_add_seconds, wt_add_seconds_to_now, wt_sub_seconds, and wt_sub_seconds_from_now which were the tragic victims of a copypasta error.	4 years ago
Johan Brandhorst-Satzkorn	f57454b6b9	Rewrite interface{} to any (#2535 ) * chore: Rewrite interface{} to any * Add ignore revs	4 years ago
Todd	fa39fe5b0f	Delete the base auth account when a concrete auth account is deleted (#2528 )	4 years ago
Irena Rindos	4493dfd527	use prior and current worker auth keys (#2495 ) * use prior and current worker auth keys	4 years ago
Timothy Messier	a2ef14283b	fix(org): Address bug that prevented deleting an org (#2465 ) The schema changes introduced in 0.10.2 introduced a bug where an org could not be deleted if there was a session for an org scoped user. It would result in an error like: scope.(Service).deleteFromRepo: unable to delete scope: \ iam.(Repository).DeleteScope: failed for o_VkNy46KwfS: \ iam.(Repository).delete: db.DoTx: iam.(Repository).delete: \ db.Delete: insert or update on table "session" violates foreign key constraint "target_fkey": integrity violation: error #1003 A worker around would be to first delete all projects in the org, then finally delete the org. This fix adds a regression test to the sqltests and updates several constraints on the session table to be deferrable. Since the org delete is a single transaction, this allows all of the necessary resources to be deleted before attempting to enforce these constrains. See: https://www.postgresql.org/docs/current/sql-set-constraints.html Blame: `3a28ab614`	4 years ago
Damian Debkowski	546c5dc5be	feat: static json credentials (#2423 )	4 years ago
Irena Rindos	821d8317dc	address op state review feedback (#2453 ) * address op state review feedback	4 years ago
Jeff Mitchell	53b5e532d5	Remove deprecated methods/fields on targets (#2393 )	4 years ago
Irena Rindos	fc0ead73e6	feat(workers): worker operational state (#2427 ) * feat(workers): worker operational state	4 years ago
Jeff Mitchell	9002930d65	Adapt shared lock skip logic to dbswap branch (#2437 )	4 years ago
Hugo Vieira	bed6e1a0ff	feat(server): Reload database on SIGHUP If the database url is updated on Boundary's config and then a SIGHUP triggered, we now connect to the new database, ensure it's OK to use and use that one going forwards instead. This allows a controller to be deployed in a more dynamic infrastructure where, for example, database credentials could change, without having to restart the controller. In this case only a database url config change and a SIGHUP signal to all controllers would be required. This uses an atomic.Pointer to store wrapped DB objects. This allows atomically swapping to the new object. A timeout is introduced so that if the object is swapped after the existing one has been looked up it isn't immediately invalidated. The timeout is a maximum; a ticker once a second that looks at stats to determine how many database connections are still in use and then closes the driver as soon as none are in use. This could be done with mutexes to avoid checking stats in a loop, but that carries its own risk: if a write lock is attempted to be acquired while a query is holding a read lock, the write lock will block, and any subsequent read locks will also block since write locks take priority. As a result, if a query is long-running or timing out, all other queries in the system will become blocked. Our default user request timeout is 90 seconds and some jobs have no timeout at all, so this is not a purely theoretical concern, even if it's unlikely. Refs: #2435 Co-authored-by: Jeff Mitchell <jeffrey.mitchell@gmail.com>	4 years ago
Hugo Vieira	621f8a9ff2	fix(schema): Database connection not cleanly closed This commit introduces closing the Schema Manager's underlying driver connection. We were only releasing the shared lock but not actually cleaning up the db connection and relied on the fact that the process would exit to do that for us.	4 years ago
Irena Rindos	18dff62b7b	Merge BYOW GA branch to main (#2398 ) * feat(workers): Add ability to read and reinitialize cert authority (#2312) * feat(workers): Return Release Version on worker list and read (#2377) * feat(cli): Read and reinitialize worker certificate authority (#2387)	4 years ago
Timothy Messier	2fbdcf6ce0	feat(session): Use permissions for limiting list results The session repository can now be passed an optional set of permissions. The permissions are used to restrict results from the repository. Currently this only impacts the List method. This enabled the sessions service to use a single query to retrieve sessions and replaces the previous behavior where the session service would first request all sessions for a given set of scopes, evaluate the permissions, and then request sessions based on a set of session ids.	4 years ago
Louis Ruch	d7c4c648ec	bug(vault): Correctly handle credential stores with expired tokens (#2399 ) * bug(vault): Correctly handle credential stores with expired tokens Boundary makes use of a database view to perform CRUD operations on Vault credential stores and libraries. This view did not include credential stores with tokens that have expired in Vault, causing errors when attempting to perform any action against them.	4 years ago
Jeff Mitchell	01fb949d0b	Add controller-led worker auth flow (#2413 ) This adds an action to the API that allows pulling out an activation token that can be given to a worker in its config (directly or via env or file). The worker can pass this along with its normal request in place of its normal nonce and the server will automatically accept it. Co-authored-by: Irena Rindos <irenarindos@users.noreply.github.com>	4 years ago
Johan Brandhorst-Satzkorn	de2c40fa77	Fork pgtap-docker to support arm64 (#2411 ) * feat(sql): Fork pgtap to support arm64 This forks pgtap-docker and builds the image locally, and updates our references to use our forked image. * fixup! feat(sql): Fork pgtap to support arm64	4 years ago
Damian Debkowski	3f70644978	fix(db): update host_plugin_set to set project_id value (#2410 )	4 years ago
Johan Brandhorst-Satzkorn	aef9073fa6	Upgrade to Go 1.19 (#2347 )	4 years ago
Danielle	07a4082aa1	Add SQL Migration 05 From go-kms-wrapping (#2371 ) * adding sql migration 05 from go-kms-wrapping * rename file from 05 to 01 to match boundary standards * move key rewrap to folder 47	4 years ago
Johan Brandhorst-Satzkorn	d709b97e18	fix(schema): Panic with error rather than function (#2391 )	4 years ago
Jim	b0bc6bd632	refactor: eliminate gorm references in favor of go-dbw (#2386 )	4 years ago
Louis Ruch	929a385fdb	fix: move 'drop target_all_subtypes' after whx views	4 years ago
Damian Debkowski	27e9f775be	feat: add initial migration hook	4 years ago
Timothy Messier	79866a287f	feat(schema): Add support for migration hooks This adds the ability to define hooks that will run prior to a schema migration. The hooks should only be used when a migration would fail due to data being in an invalid state, likely due to a bug in the previous schema definition. For example if a column was missing a constraint, and a new migration is attempting to add the constraint, there might already be values that violate the new constraint. In such cases, we would want the user to make a decision to address the data, and need to provide them with details about what data is problematic in case they want to manually address the problem. To help the user, if there is a sane way to address the data, we should provide an option to automatically "repair" the data, allowing the migration to continue. However, this still should be an explicit opt-in from the user, and we need to clearly describe what the repair option would do to the user so they understand the consequences of opting in. To accomplish this each migration can be provided a hook that contains a CheckFunc, RepairFunc, and RepairDescription. These functions must operate at the point-in-time of the migration immediately preceding the current migration, as as such they should not use and domain packages or repositories, instead they should use direct sql with the sql library. The description is used to explain what the RepairFunc will do. Hooks should be defined in `internal/db/schema/migrations/oss/oss.go` and registered with the schema edition. See the example in `internal/db/schema/manager_example_test.go` for more details.	4 years ago
Michael Gaffney	1fa79e9aa1	test(db): Add tests for inserts into target_host_set	4 years ago
Michael Gaffney	0512ddb060	test(db): Add tests for inserts into target_credential_library	4 years ago
Damian Debkowski	c4aa9a0358	test(sql) add tests to validate project scope for target_static_credential	4 years ago
Johan Brandhorst-Satzkorn	222a935223	Update grpc-gateway dependency (#2311 ) * chore(deps): Update grpc-gateway dependency This fixes an issue which would allow an attacker to trigger a (recovered) panic in the gateway handler. * fix(server): Use cmp.Diff instead of assert.Equal Testify is not great at handling protobuf types, and was for some reason returning a diff here when in reality there is none. Switch to go-cmp for the comparison to remove the error.	4 years ago
Louis Ruch	d01f374490	bug(db): Fix credential purpose migration (#2351 ) * bug(db): Fix credential purpose migration	4 years ago
Michael Gaffney	a6d9c228aa	Merge branch 'mgaffney-greppable-sql'	4 years ago
Jim	7aa7deafa3	chore: update go-dbw dependencies (#2298 ) * chore: update go-dbw dependencies This update included an update of the upstream gorm dependency. In the latest update, gorm appears to have become more sensitive when a slice is passed into operations. The ptr to slice must now be a ptr to slice of ptrs. This change required an update to how boundary handles slices of PrincipalRole, which now must return a ptr to PrincipalRole. There were also some tests that needed to be updated because of this change. * fixup! chore: update go-dbw dependencies	4 years ago
Louis Ruch	7fafadd70b	db: Add credential_sha256 to session_credentials (#2339 ) * db: Add credential_sha256 to session_credentials	4 years ago
Michael Gaffney	ac6f14a37f	style(sql): make migrations more greppable	4 years ago
Jeff Mitchell	02dd28f587	Add support for SSH private key passphrases (#2331 ) This adds support for encrypted SSH keys via passphrase. Co-authored-by: Louis Ruch <louisruch@gmail.com> Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com>	4 years ago
irenarindos	4908aba546	feat(vault): Add unimplemented worker filter support to OSS	4 years ago
Louis Ruch	a17e973712	feat(credentials): Refactor credential purposes (#2260 ) * feat(credentials): Refactor credential purposes * Application credentials have been refactored to Brokered credentials * Egress credentials have been refactored to Injected Application credentials Co-authored-by: Jeff Mitchell <jeffrey.mitchell@gmail.com>	4 years ago
Louis Ruch	ef5ac07f02	Add ssh_private_key support for Vault libraries and targets (#2263 ) * Add support for ssh_private_key credential type in Vault libraries * Add support for ssh_private_key credential type * Support jsonpointer for Vault library mapping overrides	4 years ago
Jeff Mitchell	011e2e7425	Add ssh private key type and add type to static store (#2262 ) Co-authored-by: Louis Ruch <louisruch@gmail.com>	4 years ago
Jim	3da78aa7e4	refactor (kms): Update dependency required a small refactor (#2253 )	4 years ago
Jeff Malnick	1f66685864	feat: set default connection limit to unlimited (-1) instead of 1 (#2234 ) This avoids the confusion behavior of a session terminating as soon as a single connection using that session is closed.	4 years ago
Timothy Messier	fd5d15af27	feat(config): Add database config options for idle connection management (#2176 ) This adds two new config options for the `database` stanza: - `max_idle_connections` - `max_idle_time` These correspond to settings for the underlying `sql.DB` instance. They can be used to have finer control over the controller's database connection pool. For example this can be used to mitigate the high connection overhead when connecting to postgres by setting a high `max_idle_connections` and either setting `max_idle_time` to zero or to a high duration. Alternatively, if using a connection pool like pgpool-II or pgbouncer, `max_idle_connections` can be set to 0. See: https://pkg.go.dev/database/sql#DB.SetMaxIdleConns https://pkg.go.dev/database/sql#DB.SetConnMaxIdleTime https://www.pgpool.net/mediawiki/index.php/Main_Page https://www.pgbouncer.org/	4 years ago
Damian Debkowski	3e9c99c217	refactor(user_password) rename all references of user_password to username_password (#2232 )	4 years ago
Jeff Mitchell	396edfca80	Add an extra cleanup task for sessions stuck in canceling state (#2229 ) * Add an extra cleanup task for sessions stuck in canceling state See the comment on the migrations file for details.	4 years ago
Louis Ruch	4cacd0a605	bug(scheduler): Fix controller id length check (#2226 ) * bug(scheduler): Fix controller id length check The job_run table has a foreign key to the server_controller private_id. While this column is called a prviate_id it does not implement the wt_private_id postgres domain type, and therefore does not have a minimum length of 10 characters.	4 years ago
Timothy Messier	f3c90df300	chore(db): Silence db logs by default for tests This removes a lot of log clutter from normal test execution with `go test -v`. There are a number of test that expect no results from the database or other error cases from the database. The previous default log level of dbw.Error would cause these cases to generate logs in the success case, creating noise in the test output. The new default for tests is dbw.Silent. This can be overriden in two ways. A test can be updated to pass in the db.WithTestLogLevel option, which allows for targeting specific tests to include additional output. This may make sense for tests that normally should not cause errors, but would report an error if it fails. Alternatively a new environment variable can be set, DEFAULT_TEST_LOG_LEVEL. This allows for adjusting the log level without changing any test file. The environment variable should be set to the string value of the log level, and is treated as case insensitive, ie: export DEFAULT_TEST_LOG_LEVEL="error"	4 years ago
Louis Ruch	9671daf6e0	Revert "refactor(user_passsword) change all references of user_password into username_password (#2189 )" This reverts commit `ab58b24142`.	4 years ago
Damian Debkowski	ab58b24142	refactor(user_passsword) change all references of user_password into username_password (#2189 ) * (refactor) change all sql references of user_password to username_password * refactor(pkg) renamed pkg userpassword to usernamepassword * refactor(proto) updated protocol buffers to use UsernamePassword * refactor(code) updated references from UserPassword to UsernamePassword in golang files * refactor(test) updated unit tests to reference UsernamePassword changes * refactor(ddtest) updated references in dbtests to use UsernamePassword * chore(sql-test) fixed spelling issue * chore(sql) added and updated comments in migartion & tables. fixed indentations * chore(proto) updated comments to reflect username_password changes * refactor(tests) updated unit tests to replace user-password with username-password * refactor(cmd) updated cmd help text to use username * chore(tests) update test name & comments * fix(cmd) fixed cmd usage definition for credentials update * refactor(tests) removed out of scope changes * fix(sql) incremented sql migration id & updated migration comments	4 years ago
Jeff Mitchell	704d68848c	Merge remote-tracking branch 'origin/main' into llb-byow	4 years ago
Jeff Mitchell	b0480eb7b3	Move migrations to 32	4 years ago
Todd	beecbbb8a2	Upsert creates a new KMS and Updates PKI workers (#2187 )	4 years ago
Irena Rindos	06d0f99536	bug(workersAPI): do not allow nonprintable or uppercase name or desc (#2186 ) * Send more info on KMS connect (#2182) * Update nodeenrollment to v0.1.0 * bug(workersAPI): do not allow nonprintable or uppercase name or desc * Put checks on create too * Apply printable check to description in DB * chore (worker): address PR feedback Co-authored-by: Jeff Mitchell <jeffrey.mitchell@gmail.com> Co-authored-by: Jim <jlambert@hashicorp.com>	4 years ago
Todd	577d8418cd	Make name column immutable for kms workers. (#2190 ) errcode 23601 is the error code used by the immutable_column() function.	4 years ago
Jim	9997f1a7bd	feature (workers): required schema changes (#2185 ) * feature (workers): required schema changes Changes required for consolidation of worker name, description and address fields * fixup! feature (workers): required schema changes * fixup! feature (workers): required schema changes * fixup! feature (workers): required schema changes * Fixing up address constraint. * last_status_time = now() on insert only if the worker is type = 'kms' * update last_status_time for pki and kms based on different columns being updated. * tweaking the constraints on last_status_time and address. Co-authored-by: Todd <github@quaddi.com>	4 years ago
Louis Ruch	631de0e5e2	chore(changelog): Update changelog	4 years ago
Louis Ruch	4b365db634	feat(target): Add support for static credentials during authorize-session	4 years ago
Louis Ruch	58d9d42a88	feat(session): Add support for session static credentials	4 years ago
Louis Ruch	00dfea1244	feat(target): Add support for static credential as target credential sources	4 years ago
Michael Gaffney	b6da1ccac7	feat(db): add static credential store schema	4 years ago
Todd	4b526538af	Make server_id in job_run type text since it is nullable. (#2177 )	4 years ago
Louis Ruch	b6ca61a7f6	refact(cred-libs): Update user_password to username_password (#2154 ) * refact(cred-libs): Update user_password to username_password	4 years ago
Jeff Mitchell	5d3facf561	Merge remote-tracking branch 'origin/main' into llb-byow	4 years ago
irenarindos	16d95262a6	feature(worker): add workerKeyId to status	4 years ago
Timothy Messier	32070678dc	perf(session): Remove connections from session list endpoint Including the nested connection information in the session list response had a negative impact on performance as it created a significant multiplier on the number of rows that needed to be fetched from the database. It was also determined that this information was used by neither the admin ui nor the desktop client. Note that nested connection information can still be retrieved via a read request for a specific session.	4 years ago
Timothy Messier	5b0abe2e27	feat(session): Add migration to delete terminated sessions There is a new periodic job that will delete terminated sessions that are older than one hour. However, since until now terminated sessions were not deleted, the first time the job runs there could be a large number of sessions that need deleted. This could have a negative impact on a running cluster. To mitigate the operational risk, this adds a migration to perform an initial delete while the cluster is offline.	4 years ago
Johan Brandhorst-Satzkorn	e7273d65c2	Bump Go version to v1.18.3 (#2158 ) * chore: Bump Go version Go 1.18.3 has some security patches over 1.18.1. See https://go.dev/doc/devel/release#go1.18.minor * Fix formatting This was missed in a recent PR	4 years ago
Gabriel Santos	3521c21cd5	fix: Refuse to start if max_open_connections is below 5 (#2097 ) * server refuses to start if max_open_connections is below 5 * suggested changes - Update internal/db/db.go Co-authored-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com> Co-authored-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>	4 years ago
Todd	1e3c941be1	Add active session count to the worker (#2145 )	4 years ago
irenarindos	642e41c9d7	feat(workerAuth): Implement nodeenrollment lib workerauth storage	4 years ago
Todd	731a45eba6	Merge the worker_status table with the worker table (#2111 )	4 years ago
irenarindos	bf3b0340e5	feat(workerauth): add storage for RootCertificates aggregate	4 years ago
Jim	70c5da1048	feature (workers): add repository CreateWorker(...) (#2105 )	4 years ago
Todd	a281d10206	Add worker config table and view (#2090 ) * Remove the server_id from session table, change it to worker_id on session_connection This is step 1 in updating the worker table. The worker is becoming a public resource so it must have a public id. We drop the server_id column on the session_connection table and add the worker_id column so we can use the wt_public_key type. Since db migrations are applied while the controller is offline, it should be safe to assume all session connections are not in a running state, thus allowing us to null out all the worker ids in the session_connection table. For now we don't have very good enforcement of the wt_public_id constraints, since the id is currently defined in the worker config, but this will soon change and the worker's id will be generated by the controller.	4 years ago
Todd	3fbf901739	Add source to worker tags table (#2087 ) * Remove the server_id from session table, change it to worker_id on session_connection This is step 1 in updating the worker table. The worker is becoming a public resource so it must have a public id. We drop the server_id column on the session_connection table and add the worker_id column so we can use the wt_public_key type. Since db migrations are applied while the controller is offline, it should be safe to assume all session connections are not in a running state, thus allowing us to null out all the worker ids in the session_connection table. For now we don't have very good enforcement of the wt_public_id contraints, since the id is currently defined in the worker config, but this will soon change and the worker's id will be generated by the controller. * Adding worker tag db writing test. * Move the server table back to the top. * Update comment for the worker object. * Update comment for the worker table. * Add worker tag sources ("config" or "api") * Revert renaming of UpsertWorker until later. * Remove no longer needed files. * Moving the TestWorker into the servers package. Fix the worker tags test. * Update internal/proto/controller/storage/servers/store/v1/worker.proto Co-authored-by: Irena Rindos <irenarindos@users.noreply.github.com> * Place the repos back together so they can be refactored in a different PR. Co-authored-by: Irena Rindos <irenarindos@users.noreply.github.com>	4 years ago
irenarindos	20c314787d	fixup! feat(pki): Add PKI schemas and protos	4 years ago
irenarindos	88a6ca036f	feat(pki): Add PKI schemas and protos	4 years ago
Todd	2ecd550e27	Add fields to server_worker to be a boundary resource (#2083 ) * Remove the server_id from session table, change it to worker_id on session_connection This is step 1 in updating the worker table. The worker is becoming a public resource so it must have a public id. We drop the server_id column on the session_connection table and add the worker_id column so we can use the wt_public_key type. Since db migrations are applied while the controller is offline, it should be safe to assume all session connections are not in a running state, thus allowing us to null out all the worker ids in the session_connection table. For now we don't have very good enforcement of the wt_public_id contraints, since the id is currently defined in the worker config, but this will soon change and the worker's id will be generated by the controller. * Updating server_id references to controller_id. * Adding worker tag db writing test. * Move the server table back to the top. * Update comment for the worker object. * Update comment for the worker table. * Update the newWorkerId call to take a context and fix its error. * Correct comment for the wt_network_address domain.	4 years ago
Todd	472d7d520a	Remove the server_id from session table, change it to worker_id on session_connection (#2070 ) * Remove the server_id from session table, change it to worker_id on session_connection This is step 1 in updating the worker table. The worker is becoming a public resource so it must have a public id. We drop the server_id column on the session_connection table and add the worker_id column so we can use the wt_public_key type. Since db migrations are applied while the controller is offline, it should be safe to assume all session connections are not in a running state, thus allowing us to null out all the worker ids in the session_connection table. For now we don't have very good enforcement of the wt_public_id constraints, since the id is currently defined in the worker config, but this will soon change and the worker's id will be generated by the controller.	4 years ago
Jeff Mitchell	2f06513a3a	Merge branch 'main' into llb-byow	4 years ago
Jim	785241237e	refactor (kms): adopt the go-kms-wrapping/extras/kms package (#2027 ) * feature (db): add Db.UnderlyingDB() This is needed to allow for integration with bits that need a dbw.DB feature (kms): add new go-kms-wrapping/extras/kms dependency * refactor (kms): schema changes required to adopt extras/kms pkg * refactor (kms/iam): rename to DeprecateCreateKeysTx(...) * refactor (kms): remove deprecated test * refactor (kms): refactor Kms type This refactor includes renaming the existing type to DeprecatedKms and then refactoring the Kms type to become a wrapper of go-kms-wrapping/extras/kms * refactor (kms): delete all the unneeded bits All this stuff is not needed now that we're using go-kms-wrapping/extras/kms * refactor (db): fixup tests after kms refactor * refactor (kms/tests): use testing.TB for test helpers Our test helper packages were not possible to use with benchmarks. This refactor switches to testing.TB to allow both tests and benchmarks to use the helpers. * tests (session benchmarks): refactor to use new Kms type * refactor (schema): rm test which relies on deprecated Kms code * refactor (kms): post rebase on main changes required * chore (kms refactor): re-add unit test from main * refactor (kms): uses migrations that don't include KMS schema changes	4 years ago
irenarindos	54c30d2cad	bug(session): null fkey trigger also checks for session terminated state	4 years ago
irenarindos	0ef523ba77	Remove column cascade	4 years ago
Todd	d8a443e238	Merge branch 'main' into llb-byow # Conflicts: # internal/daemon/controller/handlers/targets/target_service.go # internal/daemon/controller/handlers/workers/multihop_service.go # internal/daemon/controller/handlers/workers/worker_proxy_service.go # internal/daemon/controller/handlers/workers/worker_service.go # internal/daemon/worker/status.go # internal/daemon/worker/worker.go	4 years ago
Todd	b58dada40d	Move the server daemons into their own package (#2061 ) * Move the server daemons into their own package With this change the internal/server package now is dedicated to the server domain (workers and controllers) and not to the server daemons. The hope is that this will make it easier to keep the daemons decoupled from the representation of the servers in the database.	4 years ago
Jeff Mitchell	2a8e179692	Update against changes from merge	4 years ago
irenarindos	8579a248d4	refactor(servers): minor sql adjustments to worker and controller tables	4 years ago
irenarindos	9e27605213	refact(servers): Split server table into worker and controller	4 years ago
irenarindos	c6764e85d9	fix(session): Update session state transition trigger	4 years ago
Timothy Messier	f00af362dc	fix(session): Include where clause with order and limit When applying the limit and order by, the where clause also needs to be applied to yield the correct results. This can be seen with an example. Assuming a data set where there are 100 sessions, with 10 sessions belonging to a user with id `u_user1` and 90 sessions belonging to `u_user2`. If the constructed query is like: select * from (select public_id from session limit 10) s, session_list ss where s.public_id = ss.public_id and s.user_id = 'u_user1' order by create_time asc; The initial sub-query with the limit could return sessions only belonging to `u_user2`, then it would apply the where clause and exclude all of the rows. Instead if the query is: with session_ids as ( select public_id from sessions as s where s.user_id = 'u_user1' order by create_time asc limit 10 ) select * from session_list where session_list.public_id in (select * from session_ids) order by create_time asc; The CTE applies the where clause ensuring that only sessions for `u_user1` are used for the final select from the view. Note that the CTE is use primarily for readability. This also adds an index on `session (create_time)` to aide the query planner in performing the order by.	4 years ago
Johan Brandhorst-Satzkorn	7f9b294a7e	refact(all): Use testing.TB for test helpers Our test helper packages were not possible to use with benchmarks. This refactor switches to testing.TB to allow both tests and benchmarks to use the helpers.	4 years ago
Timothy Messier	4599e64cab	feat(event): Remove BOUNDARY_DEVELOPER_ENABLE_EVENTS env var	4 years ago
Jim	addbfee593	chore: upgrade gofumpt to v0.3.1 (#2028 )	4 years ago
Louis Ruch	e6b7b7f60a	feat(db): Suppress not found events during db lookups (#1979 )	4 years ago
Johan Brandhorst-Satzkorn	b1d6a1da9a	Switch to buf for protobuf generation (#1944 ) * Install buf with go install This reduces the impact of using buf on our dependency closure, which was significant. It also avoids potential issues with building buf given our own internal dependency constraints. This has caused issues (including crashes!) in the past. See https://github.com/cockroachdb/cockroach/issues/67216#issuecomment-874176186 for more discussion on using tool dependencies generally, and https://github.com/bufbuild/buf/issues/52 for an example of where this sort of thing caused an issue for a user. * Upgrade buf version We should migrate to v1 sooner rather than later, as it has some breaking changes from v0. * Change buf lint/breaking commands These commands were moved up one level for v1 * Migrate to buf.yaml to v1 * Remove unused imports These were uncovered by using "buf lint". * Use buf v1.3.1 * Depend on googleapis and grpc-gateway via BSR * Switch to buf for proto generation * Remove third party dependency * Remove unused tools and scripts * Move local protos up one level * Temporarily don't fail on breaking changes * Generate CI config * Fix generation of controller.json * Replace use of deprecated flag * Remove unused import, regenerate files * Remove temporary skipping over breaking check	4 years ago
Jeff Mitchell	7eb29261b2	Update to go-kms-wrapping version 2, and plugin-based KMS (#1901 )	4 years ago
irenarindos	32132d6bc1	refactor(session connections): Refactor connection closure	4 years ago
Irena Rindos	5a2c0db586	fix(session connections): Create session connection repository	4 years ago
Todd	e4d1e501cf	fix(warehouse): Handle duplicate addresses when migrating into wh_network_address_dimension (#1860 )	4 years ago
Todd	11ce54f8fb	Add DW network address dimension tables (#1855 )	4 years ago
Haotian	fa00a06bef	feat(host): Add host set information to static host (#1828 ) * feat(host): Add host set information to static host Updates LookupHost and ListHost API to include sets a static host is part of, as like with plugin hosts. * test: updated TestList_Plugin and TestCrud to be more consistent under race conditions	4 years ago
Irena Rindos	edae7c899f	bug(session): Allow only valid state transitions	4 years ago
Jim	591ec476e7	refactor: Update internal/db and oplog to use go-dbw package for database operations. (#1785 ) * chore: Add new dbw dependency * refactor: Convert to use dbw.DB * refactor (db): Change GetTicket(...) signature * feature (db tests): Added TestDeleteWhere(...) Adding this allows us to eliminate a leak of the gorm library * refactor: Add ctx to ScanRows(...) signature This is required to refactor all the deprecated errors pkg calls. This refactor required adding a context to quite a few call chains and in particular the job scheduler bits * feature (db/tests): Add support for test opt: WithTestLogLevel(...) This test only option allows callers to specify a log level for the underlying db package (where supported/applicable) when calling TestSetup(...) * refactor (db): Simplify Db type Only store the underlying DB and create the RW on demand when needed. * refactor (oplog): Port to using dbw pkg for db operations * refactor (db): Remove gormDB(...) func * tests (db): Add unit tests for TestDeleteWhere(...) * chore: make fmt changes to formatting	4 years ago
Louis Ruch	5fe23ab14d	feat(session): Store session credentials Creates a new session_credential table and repository functions to add and list credentials attached to a session. All credentials are encrypted using the session scope id.	4 years ago
Michael Gaffney	ba8188e6d7	feat(credential): Add mapping override to credential library on lookup This change returns any mapping override set on a credential library when the library is retrieved from the repository using the lookup method.	4 years ago
Michael Gaffney	0e637f47b5	feat(credential): Use credential type when issuing credentials This change uses the credential type attribute when issuing credentials including mapping overrides.	4 years ago
Michael Gaffney	c317901db4	feat(credential): add credential type to credential library on create This change adds a credential type attribute to credential library and allows it to be set when creating a credential library. Specifying the type of credential returned from a library allows for new features and additional error checking to be added over time. A User Password credential type and an Unspecified credential type are included with the expectation that additional credential types will be added in the future. This change also adds the credential type attribute to existing credential libraries and sets it to Unspecified. Known Limitations: - Credential type is only supported on the repository create method. Support for the other CRUD methods will be added in the future.	4 years ago
Jim	baa1d88f1f	feature: Add client ip to inbound request information (#1678 ) * feature: Add client ip to inbound request information * feature (workers): Add the user's client IP to a session's connection information (#1749)	4 years ago
Michael Gaffney	72d4d6d205	fix(sql): Remove warning from create_time trigger (#1751 ) Raising a warning in the default trigger used for setting the value in create_time columns creates unnecessary noise in the PostgreSQL log files. Closes #1744	4 years ago
Jim	983f7b019c	refactor (db): Remove unused gorm formatter (#1738 )	4 years ago
Jeff Mitchell	ad8ce50f69	Fix Windows (#1733 ) The embed package always uses Unix-style forward slashes. filepath.Join utilization looking for files would cause the slashes to be the wrong direction on Windows and result in the files not being found.	4 years ago
Michael Gaffney	55c6c5872f	feat(sqltest): Make container port configurable	4 years ago
Jeff Mitchell	866bfceefc	Add secrets_hmac to host catalog output (#1712 ) Co-authored-by: Jim <jlambert@hashicorp.com>	4 years ago
Thor	7fa2cd2526	rename migration from 01-09 after bad merge (#1709 ) * rename migration from 01-09 after bad merge	4 years ago
Thor	ad31b6d6c3	repo: Return connection informat in Session response object (#1690 ) Add connections to the Session object * db: creat new session_list view * repo: session use new session_list view	4 years ago
Jeff Mitchell	8110e0eab3	Increment version of host during updates (#1707 ) Currently host does not use a version, which means it's not protected against race conditions of either itself or its value objects. This adds and plumbs through version to ensure that if a host or its value objects change that they are not going to be affected by a race condition.	4 years ago
Jim	1b6c9a25c8	Add more opts when using Create + WithOnConflict (#1706 )	5 years ago
Jim	c6d79bc911	feature (db): Add WithOnConflict support to CreateItems(...) (#1699 )	5 years ago
Todd Knight	5850001a07	Rename file to match others in directory and remove TODO.	5 years ago
Todd Knight	f41bceda83	Merge from main and `make gen`	5 years ago
Todd Knight	884e1f2d18	Merge branch 'main' into plugin-hostcatalogs # Conflicts: # go.mod # sdk/go.mod	5 years ago
Jim	1ac151100e	docs (internal/db): Add docs Update/Delete operations using WithWhere (#1687 )	5 years ago
Todd	29b4537d3b	Reference PreferredEndpoints by top level struct name (#1688 ) * Reference PreferredEndpoints by top level struct name instead of embedded proto GetPreferredEndpoint method. Reorganize the sql files a little. * Make PreferredEndpoints replayable. Test refactoring to remove variables that persist across test calls. DeleteItems instead of trying to use WithWhere.	5 years ago
Jeff Mitchell	30e9f944a2	Add sync interval to host sets (#1680 ) This adds a parameter to host sets allowing the sync interval to be user-specified on a per-host-set basis. A value of -1 disables syncing; a value of null uses the current default.	5 years ago
Jim	c047c4d89d	refactor (db): Remove the internal/db test tables from the migrations (#1675 )	5 years ago

... 2 3 4 5 6 ...

625 Commits (fcafffdb24b84bba85d06fb208778efe2b9201c7)