boundary

Commit Graph

Author	SHA1	Message	Date
Johan Brandhorst-Satzkorn	dc4ea67d6e	internal/db: changes for pagination	2 years ago
Danielle Miu	39b3bb2d1d	fix(db): allow session_recording_aggregate to return recordings correctly	2 years ago
Danielle Miu	df02651bb5	feat(db): new cols for retain and delete session recordings, update structs, and tests	2 years ago
Hugo	978db86186	feat(internal/db): Storage policies implementation (#767 ) This commit implements part of the storage policy's database model, focusing specifically on the operational tables for this feature.	2 years ago
Johan Brandhorst-Satzkorn	b6fe9824ab	internal/db: optimize session list indexes	2 years ago
Johan Brandhorst-Satzkorn	69262c386c	internal/db: renumber migrations (#4218 )	2 years ago
Johan Brandhorst-Satzkorn	85ca909720	internal/db: add iam_scope indexes These will help the iam_scope list queries	2 years ago
Johan Brandhorst-Satzkorn	d4b60241db	internal/db: storage bucket pagination migrations (#4213 )	2 years ago
Michael Milton	52b1c1b4c7	Managed group list pagination (#4208 ) * wip * managed groups pagination * address PR comments * missed one change	2 years ago
Johan Brandhorst-Satzkorn	44971b3559	internal/db: migrations for accounts pagination	2 years ago
Johan Brandhorst-Satzkorn	0715003945	internal/db: renumber session recording migration	2 years ago
Johan Brandhorst-Satzkorn	cce52a16e6	internal/db: db migrations for session recordings (#4200 )	2 years ago
Johan Brandhorst-Satzkorn	e9b90d9ce1	internal/db: add schema changes for host catalog pagination	2 years ago
Johan Brandhorst-Satzkorn	f0dbddb9e3	internal/db: ensure new columns are not null (#4197 ) Ensure the new create_time and update_time columns are marked as not null after populating the data. Also ensure that the data is populated safely with an updated query pattern inspired by the auth method pattern.	2 years ago
Todd	40d9c3fd1d	Paginate iam groups (#4194 )	2 years ago
Johan Brandhorst-Satzkorn	8dc431fef5	internal/db: renumber auth methods migration (#4199 )	2 years ago
Michael Milton	8d50a0913f	Auth Method list pagination (#4165 ) * wip * implement list pagination for auth methods * remove unused code * update tests to fix sorting * address PR comments * simplify query, add token tests * change test name * wip sql testing * add boolean column to indicate whether auth methods should be returned to unauthenticated users, remove secrets from api response * is_active_public not null * remove secrets and unwrapping * move not null to immediately after data migration * order public_id descending * fix(db): Fix auth method migrations (#4193) * wip * fix db tests * fix failing api tests * reverse order of create time and update time in test --------- Co-authored-by: Michael Gaffney <mike@gaffney.cc>	2 years ago
Todd	0fad770d8a	Paginate iam user (#4185 ) * paginate iam user	2 years ago
Johan Brandhorst-Satzkorn	b3207d8e1c	internal/host: sort public_id descending The previous sort would run the risk of missing items which collide on update or create times because of the compound comparison using strictly less than for both the timestamp and the public_id. Sorting both descending avoids this issue.	2 years ago
Johan Brandhorst-Satzkorn	d94a25bbd2	internal/host: sort public_id descending The previous sort would run the risk of missing items which collide on update or create times because of the compound comparison using strictly less than for both the timestamp and the public_id. Sorting both descending avoids thi issue.	2 years ago
Johan Brandhorst-Satzkorn	d4f5677858	internal/iam: sort public_id descending The previous sort would run the risk of missing items which collide on update or create times because of the compound comparison using strictly less than for both the timestamp and the public_id. Sorting both descending avoids thi issue.	2 years ago
Johan Brandhorst-Satzkorn	f9950a7e1c	internal/authtoken: sort public_id descending The previous sort would run the risk of missing items which collide on update or create times because of the compound comparison using strictly less than for both the timestamp and the public_id. Sorting both descending avoids thi issue.	2 years ago
Johan Brandhorst-Satzkorn	d4d1efce93	internal/session: sort public_id descending The previous sort would run the risk of missing items which collide on update or create times because of the compound comparison using strictly less than for both the timestamp and the public_id. Sorting both descending avoids thi issue.	2 years ago
Johan Brandhorst-Satzkorn	6238a533b0	internal/credential: sort public_id descending The previous sort would run the risk of missing items which collide on update or create times because of the compound comparison using strictly less than for both the timestamp and the public_id. Sorting both descending avoids thi issue.	2 years ago
Johan Brandhorst-Satzkorn	fc9b26efa7	internal/target: sort public_id descending The previous sort would run the risk of missing items which collide on update or create times because of the compound comparison using strictly less than for both the timestamp and the public_id. Sorting both descending avoids thi issue.	2 years ago
Johan Brandhorst-Satzkorn	27393a8227	internal/db: add host set indexes	2 years ago
Johan Brandhorst-Satzkorn	21c593ab71	internal/db: add host indexes	2 years ago
Todd	5aa560387c	Add pagination for iam Roles (#4161 ) * Add pagination for iam Roles	2 years ago
Johan Brandhorst-Satzkorn	77b76ae685	internal/db: reorder pagination schema files	2 years ago
Johan Brandhorst-Satzkorn	22ce2b4468	internal/db: add cred store pagination schema changes Add create time, update time columns to base table, add indexes used by list queries.	2 years ago
Johan Brandhorst-Satzkorn	780dfe20ad	internal/db: move pagination migrations to same folder (#4142 ) Since rebasing on main, we are no longer working in 79, but 80 instead.	2 years ago
Todd	5daef58f54	SQLite allows having 1 set to MaxOpenConnections (#3941 ) * let sqlite have a single concurrent connection	2 years ago
Jim	fba9d1cdf1	Add client daemon repo, server, and cmd The commands added are: boundary daemon start boundary daemon stop boundary daemon add-token boundary daemon status boundary search If these actions are performed on a platform that doesn't support sqlite natively in go an error is returned. Besides some hooks into the cmd package all the significant logic resides in internal/clientcache feat (cmds/authenticate): output the token name when storing token feat: update go-dbw which now uses pure go sqlite driver refact (base.Command): export FlagOutputCurlString Exporting FlagOutputCurlString allows other commands that embed the base.Command to access the flag as needed feat: add a cache Repository to manage cache storage feat: add a "cache" CLI command This new command will support both managing and searching the local CLI cache daemon. feat: start cache daemon as needed when issue CLI commands Move files to new directories and add persona table (#3526) * Add persona table and move files to new directories Client daemon now uses unix sockets and update pathing (#3535) * Add unix socket listen and dialers with file permissions applied * Update the pathing to match RFC Cache refreshing now uses all stored personas. (#3545) Updating headers for LLB files Decouple daemon and cache server (#3566) * Separate daemon from server * Add persona handler to daemon server (#3579) Moved search to use mql and added tests for the search command (#3589) * search uses hashicorp/mql Windows support for client side daemon (#3618) * exec.Command for background process * Log to stderr when in foreground and file always * Use pid files across platform for start and stop Requests with version info must match the daemon's version (#3640) * requests with version info must match daemons Update dependencies and breakout cgo/pure go sqlite drivers (#3657) upgrade go-dbw deps and add cgo build tags Enable graceful shutdown in windows (#3695) * Enable graceful shutdown in windows Co-authored-by: Jim <jlambert@hashicorp.com> Targets delete when a persona changes address or user id (#3673) * Handle cache refresh in repo Add sessions to the client side cache (#3682) * Add sessions to repository * Search command supports sessions * adding session related tests to db and refresh Change cache schema from personas to tokens+users (#3737) * Change from personas to tokens+users * Separate keyring tokens from auth tokens Add support for keyringless auth tokens (#3765) * Add support for keyringless auth tokens Remove expired auth tokens from cache (#3768) * Remove expired auth tokens from cache Complete the TODO of checking for inflight tx (#3779) Delete tokens that the controller says are invalid (#3782) fix search test with boundary token check (#3799) Collect resource info in repository_<resource>s.go (#3769) * Collect resource info in repository_<resource>s.go * Move search logic into the domain (#3794) Daemon and search restricted to supported platforms (#3801) * Daemon and search restricted to supported platforms Reorganize the clientcache packages (#3814) * Refactor for self contained domain logic * Put clientcache stuff under internal	2 years ago
Michael Milton	6ba3676c37	Auth token pagination (#4113 ) * Add pagination support for auth tokens * change analyze count estimate comment * adjust queries for efficiency and security * even more efficient queries	2 years ago
Johan Brandhorst-Satzkorn	fcb47f3772	internal/db: add cred library pagination schema changes	2 years ago
Johan Brandhorst-Satzkorn	497c158b9d	internal/db: add migration for static credential table The static credential table is used rather than the credential table as all credentials returned by ListCredentials are static at this time.	2 years ago
Michael Milton	9c3eb16046	Sessions list pagination (#4079 ) * Session list pagination refactor * address pr comments, add explanatory comments, clean up sql queries, move tests into proper files * fix build error * fix test error	2 years ago
Johan Brandhorst-Satzkorn	6181f34497	internal/db: move new migrations to new folder	2 years ago
Johan Brandhorst-Satzkorn	f0f7e635a0	internal/db: add target schema for pagination design The new design uses the base table for querying, so the base table needs to have both create_time and update_time. In this commit we add the update_time to the target base table. We also add a number of new indexes to help with looking up targets.	2 years ago
Todd	5bfc8a88b1	Add indexes on update_time (#3974 ) * Add indexes on update_time Co-authored-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com> Co-authored-by: Michael Gaffney <mike@gaffney.cc>	2 years ago
Johan Brandhorst-Satzkorn	43cff11865	internal/db: renumber migration	2 years ago
Johan Brandhorst-Satzkorn	dd2b373866	internal/db: add Now method (#3887 ) The Now() method returns the current transaction timestamp.	2 years ago
Johan Brandhorst-Satzkorn	38536f4eef	internal/db: add deleted tables for all resources (#3885 ) Creates a new foo_deleted table for each subtype of each resource that needs to support pagination. The deleted tables are used to track deletions of resources and return them to users of the new pagination API. We track only the ID and delete_time of the resource, so there's no way to filter out resources that the user could not see before it was deleted. This is intentional to keep the design simple and is not considered a security issue.	2 years ago
Damian Debkowski	2f62f65252	fix(db): make bucket_prefix column immutable for storage buckets (#4108 )	2 years ago
Elim Tsiagbey	8b8d2822df	feat(oidc): OIDC Prompt (#4053 ) Boundary OIDC method does not currently support passing in prompts during authentication. This change adds the capability of passing OIDC prompts. Prompts are optional OIDC parameters that determine the behaviour of the authentication server: https://openid.net/specs/openid-connect-core-1_0.html#AuthRequest ## Changes - New `auth_oidc_prompt` table which contains all the prompts for OIDC auth method - New `auth_oidc_prompt_enm` table which contains possible enum values for a prompt - Currently supported: - `none`: The Authorization Server MUST NOT display any authentication or consent user interface pages - `login`: The Authorization Server SHOULD prompt the End-User for reauthentication - `consent`: The Authorization Server SHOULD prompt the End-User for consent before returning information to the Client - `select_account`: The Authorization Server SHOULD prompt the End-User to select a user account - `oidc_auth_method_with_value_obj` view has been updated to return `prompt` value - Add `prompt` option for OIDC auth method CLI create and update - Pass configured prompt during OIDC authentication - Add `prompt` API validation for create and update	2 years ago
Johan Brandhorst-Satzkorn	940eb61282	sqltest: fix typo (#3886 )	3 years ago
dani	c89777de41	Expose Valid Principals for Vault SSH Signed Certs (#3791 ) * allow additional valid principals when creating vault ssh signed cert cred lib * add additional_valid_principals field to credential libraries documentation page	3 years ago
Jim	1c330543a7	fix (oplog keys): fix issue with migration of oplog keys to new tables (#3868 ) We inadvertently used kms.GetWrapper(ctx, am.ScopeId, kms.KeyPurposeDatabase) which resulted in the use of a database key for an oplog entry. See: https://github.com/hashicorp/boundary/pull/3665/files#diff-bcf83b5283886c98fc45b6435fdc94d87d545b2768c6beac8acd14eb9d6b13f1 This fixes the issue with the missing "Database" purpose key when we converted the oplog keys into their own tables. We chosen to truncate the existing oplog entries to eliminate any references to a Database kms key in existing oplog entries.	3 years ago
Jim	4ed675be16	feat (db): add Writer.IsTx(...) (#3770 )	3 years ago
Jim	0d19892e03	fix (events): convert multierror to stdlib error (#3743 ) * fix (events): convert multierror to stdlib error multierror errors will be converted to a stdlib error (because multierror doesn't support json marshaling). * refact (errors): convert from multierror to stdlib errors.Join(...) * refact: convert from pkg/errors to stdlib error wrapping	3 years ago
Irena Rindos	9733a250fa	refact(event): move observability/event to event (#3704 )	3 years ago
Michael Gaffney	0af36d3d8a	refactor(sql): Consolidate changes to oplog_entry into one migration	3 years ago
Michael Gaffney	3befccffd7	refactor(sql): Reorder and rename oplog kms migrations	3 years ago
Michael Gaffney	d8393e5793	style(sql): fix indenting and remove trailing whitespace	3 years ago
Jim	a16033b73a	fix (kms): move oplog DEK into its own internal kms (#3665 ) * fix (auth/ldap): fix which DEK is used for oplog during authenticate * chore: update go-kms-wrapping dep for new extras/kms changes The latest version supports setting a table name prefix for a kms which is needed to fix our oplog DEK problem * chore: update extras/kms dep This latest version supports an optional TableNamePrefix which is required to fix our existing oplog DEK issue. * fix (oplog): schema oplog DEK fix These changes will allow use to store the oplog DEK in it's own set of tables, so they won't get deleted when a scope is deleted * fix (kms): move oplog DEK into its own internal kms * tests: update tests to be compatible with oplog fix * tests: remove migration tests which are incompatible with oplog fix These tests no longer work because they are missing the required schema changes for the oplog fix	3 years ago
Jim	dc89ca28a1	feat: add API support for additional LDAP auth method fields. (#3679 ) * feat (auth/ldap): add Repo support for max page size and deref aliases (#3670) * chore: update cap/ldap dependency This latest version of cap/ldap adds support for MaxPageSize and DerefAliases * feat (auth/ldap): add MaxPageSize and DerefAliases to storage proto * feat (api/authmethods): add new ldap fields. Add support for maximum_page_size and dereference_aliases * feat (auth/ldap): schema changes to support new LDAP attributes * feat (auth/ldap): add Repo support for max page size and deref aliases * feat (handlers/ldap): add support for new fields (#3674) Added API support for new ldap fields: MaximumPageSize and DereferenceAliases * chore: add CHANGELOG entry for new api auth ldap fields	3 years ago
Jim	d5411a26f6	chore: update protobuf dep (#3669 )	3 years ago
hashicorp-copywrite[bot]	29da0bcb92	[COMPLIANCE] License changes (#3567 ) * Adding explicit MPL license for sub-package. This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository. * Adding explicit MPL license for sub-package This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository. * Updating the license from MPL to Business Source License Going forward, this project will be licensed under the Business Source License v1.1. Please see our blog post for more details at https://hashi.co/bsl-blog, FAQ at www.hashicorp.com/licensing-faq, and details of the license at www.hashicorp.com/bsl. * Update copyright file headers to BUS-1.1 * Rerun make gen This will pick up the last of the license changes * Revert "Rerun make gen" This reverts commit `dbe0968bff`. * Remove copywrite from generation script The current version of copywrite is not sophisticated enough to only apply the copywrite changes to certain subdirectories, so to avoid accidentally relicensing our api and sdk modules, we will stop running the copywrite script until such time that it can do the right thing. * License protobuf API as MPL The protobuf API files are used to generate the SDK, since we want the SDK to be MPL, the protobuf files must also be MPL. This extends to all protobuf files whose sources end up in the SDK directory, including: - internal/proto/controller/api - internal/proto/controller/custom_options - internal/proto/plugin It also includes the generated service directory in internal/gen/controller/api/services, since they are generated from the protobuf sources used for the SDK. * Change website license file name This makes the naming scheme consistent with the rest of the repository. * Add note on licensing to README * Set copywrite license to BUSL * Revert "Add note on licensing to README" This reverts commit `4d865a91a7`. --------- Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com> Co-authored-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>	3 years ago
Damian Debkowski	e285a8ef6f	refactor: make secrets optional for storage bucket (#3353 )	3 years ago
Michael Milton	8527f0305d	Fix issue where snapshot conflicts weren't being handled (#3406 ) * fix issue where conflicts weren't being handled * Instead of using on collision, alter the daily_range * move tests to appropriate file * Apply suggestions from code review Co-authored-by: Michael Gaffney <mike@gaffney.cc> --------- Co-authored-by: Michael Gaffney <mike@gaffney.cc>	3 years ago
Michael Milton	23c40a0bc3	Update census to run less frequently (#3346 ) * update census to run less frequently * Update internal/census/census_job.go Co-authored-by: Michael Gaffney <mike@gaffney.cc> * change nextrunin duration --------- Co-authored-by: Michael Gaffney <mike@gaffney.cc>	3 years ago
Jeff Mitchell	8322916fb4	Remove all deprecated Err/Wrap/E calls (#3286 )	3 years ago
Timothy Messier	603642f271	fix(session): List performance with large number of session connections (#3288 ) As part of the performance improvements for listing sessions for v0.9.0, the view used to retrieve sessions for listing was updated to no longer include a join on session_connection. However, a later migration that was adding additional columns to this view inadvertently re-added the join and session_connection columns. This resulted in requests to list sessions performing this join, and returning additional rows for each connection in a session. The application would ignore this additional data, since it is no longer returned via the API. This would be particularly impactful in cases where there are a large number of connections per session, as this would greatly increase the amount of data returned by the database. This fix removes the session_connection join from this view, which should restore the original performance improvements even in cases where there is a large number of session connections per session. See: https://github.com/hashicorp/boundary/pull/2160 Ref: `32070678dc` Fixes: `1e3c941be1` (cherry picked from commit `7aef341682`)	3 years ago
Timothy Messier	876f9bb07f	fix(session): List performance with large number of session connections As part of the performance improvements for listing sessions for v0.9.0, the view used to retrieve sessions for listing was updated to no longer include a join on session_connection. However, a later migration that was adding additional columns to this view inadvertently re-added the join and session_connection columns. This resulted in requests to list sessions performing this join, and returning additional rows for each connection in a session. The application would ignore this additional data, since it is no longer returned via the API. This would be particularly impactful in cases where there are a large number of connections per session, as this would greatly increase the amount of data returned by the database. This fix removes the session_connection join from this view, which should restore the original performance improvements even in cases where there is a large number of session connections per session. See: https://github.com/hashicorp/boundary/pull/2160 Ref: `32070678dc` Fixes: `1e3c941be1`	3 years ago
Michael Gaffney	99a8777866	fix(db): Add index to session connection table This index improves the performance for two queries on the `session_connection` table. The execution time for the first query went from 40.105 ms to 1.405 ms. The execution time for the second query went from 13.808 ms to 0.217 ms. Query 1 explain plan before index: Update on session_connection (cost=6465.17..6557.68 rows=11 width=50) (actual time=38.868..39.147 rows=3 loops=1) -> Nested Loop (cost=6465.17..6557.68 rows=11 width=50) (actual time=38.471..38.506 rows=3 loops=1) -> HashAggregate (cost=6464.75..6464.86 rows=11 width=40) (actual time=38.447..38.450 rows=3 loops=1) Group Key: (session_connection_1.public_id)::text Batches: 1 Memory Usage: 24kB -> Nested Loop Semi Join (cost=0.42..6464.72 rows=11 width=40) (actual time=29.184..38.434 rows=3 loops=1) -> Seq Scan on session_connection session_connection_1 (cost=0.00..6329.38 rows=16 width=20) (actual time=15.744..38.014 rows=23 loops=1) Filter: ((worker_id)::text = 'w__________k'::text) Rows Removed by Filter: 163607 -> Index Scan using session_connection_state_connection_id_end_time_key on session_connection_state (cost=0.42..8.45 rows=1 width=20) (actual time=0.016..0.016 rows=0 loops=23) Index Cond: (((connection_id)::text = (session_connection_1.public_id)::text) AND (end_time IS NULL)) Filter: ((state = ANY ('{authorized,connected}'::text[])) AND (start_time < (CURRENT_TIMESTAMP - '00:00:05'::interval))) Rows Removed by Filter: 1 -> Index Scan using session_connection_pkey on session_connection (cost=0.42..8.44 rows=1 width=20) (actual time=0.016..0.016 rows=1 loops=3) Index Cond: ((public_id)::text = (session_connection_1.public_id)::text) Planning Time: 0.793 ms Trigger immutable_columns: time=0.474 calls=3 Trigger update_connection_state_on_closed_reason: time=0.159 calls=3 Trigger update_time_column: time=0.034 calls=3 Trigger update_version_column: time=0.041 calls=3 Trigger wh_update_session_connection: time=0.688 calls=3 Execution Time: 40.105 ms Query 1 explain plan after index: Update on session_connection (cost=152.71..219.87 rows=8 width=50) (actual time=0.497..0.683 rows=3 loops=1) -> Nested Loop (cost=152.71..219.87 rows=8 width=50) (actual time=0.266..0.292 rows=3 loops=1) -> HashAggregate (cost=152.29..152.37 rows=8 width=40) (actual time=0.256..0.258 rows=3 loops=1) Group Key: (session_connection_1.public_id)::text Batches: 1 Memory Usage: 24kB -> Nested Loop Semi Join (cost=4.94..152.27 rows=8 width=40) (actual time=0.199..0.251 rows=3 loops=1) -> Bitmap Heap Scan on session_connection session_connection_1 (cost=4.51..50.76 rows=12 width=20) (actual time=0.017..0.035 rows=10 loops=1) Recheck Cond: ((worker_id)::text = 'w__________k'::text) Heap Blocks: exact=9 -> Bitmap Index Scan on session_connection_worker_id_closed_reason_ix (cost=0.00..4.51 rows=12 width=0) (actual time=0.011..0.011 rows=10 loops=1) Index Cond: ((worker_id)::text = 'w__________k'::text) -> Index Scan using session_connection_state_connection_id_end_time_key on session_connection_state (cost=0.42..8.45 rows=1 width=20) (actual time=0.021..0.021 rows=0 loops=10) Index Cond: (((connection_id)::text = (session_connection_1.public_id)::text) AND (end_time IS NULL)) Filter: ((state = ANY ('{authorized,connected}'::text[])) AND (start_time < (CURRENT_TIMESTAMP - '00:00:05'::interval))) Rows Removed by Filter: 1 -> Index Scan using session_connection_pkey on session_connection (cost=0.42..8.44 rows=1 width=20) (actual time=0.010..0.010 rows=1 loops=3) Index Cond: ((public_id)::text = (session_connection_1.public_id)::text) Planning Time: 0.614 ms Trigger immutable_columns: time=0.279 calls=3 Trigger update_connection_state_on_closed_reason: time=0.185 calls=3 Trigger update_time_column: time=0.027 calls=3 Trigger update_version_column: time=0.032 calls=3 Trigger wh_update_session_connection: time=0.450 calls=3 Execution Time: 1.405 ms Query 2 explain plan before index: Limit (cost=1002.40..6147.94 rows=1 width=253) (actual time=12.244..13.726 rows=1 loops=1) -> Nested Loop Left Join (cost=1002.40..6147.93 rows=1 width=253) (actual time=12.243..13.724 rows=1 loops=1) Join Filter: ((w.public_id)::text = (session_connection.worker_id)::text) CTE worker_config_tags -> GroupAggregate (cost=1.71..2.26 rows=22 width=59) (actual time=0.044..0.082 rows=22 loops=1) Group Key: ct_1.worker_id, ct_1.source -> Sort (cost=1.71..1.77 rows=22 width=49) (actual time=0.031..0.034 rows=22 loops=1) Sort Key: ct_1.worker_id, ct_1.source Sort Method: quicksort Memory: 27kB -> Seq Scan on server_worker_tag ct_1 (cost=0.00..1.22 rows=22 width=49) (actual time=0.007..0.010 rows=22 loops=1) -> Nested Loop Left Join (cost=0.14..9.28 rows=1 width=245) (actual time=0.106..0.108 rows=1 loops=1) Join Filter: ((w.public_id)::text = (ct.worker_id)::text) -> Nested Loop Left Join (cost=0.14..8.72 rows=1 width=213) (actual time=0.103..0.104 rows=1 loops=1) Join Filter: ((w.public_id)::text = (wt.worker_id)::text) -> Index Scan using server_worker_pkey on server_worker w (cost=0.14..8.15 rows=1 width=181) (actual time=0.012..0.012 rows=1 loops=1) Index Cond: ((public_id)::text = 'w__________k'::text) -> CTE Scan on worker_config_tags wt (cost=0.00..0.55 rows=1 width=64) (actual time=0.090..0.090 rows=0 loops=1) Filter: (((worker_id)::text = 'w__________k'::text) AND (source = 'api'::text)) Rows Removed by Filter: 22 -> CTE Scan on worker_config_tags ct (cost=0.00..0.55 rows=1 width=64) (actual time=0.001..0.002 rows=1 loops=1) Filter: (((worker_id)::text = 'w__________k'::text) AND (source = 'configuration'::text)) Rows Removed by Filter: 8 -> GroupAggregate (cost=1000.00..6136.36 rows=1 width=21) (actual time=12.133..13.610 rows=0 loops=1) Group Key: session_connection.worker_id -> Gather (cost=1000.00..6136.35 rows=1 width=13) (actual time=12.132..13.608 rows=0 loops=1) Workers Planned: 2 Workers Launched: 2 -> Parallel Seq Scan on session_connection (cost=0.00..5136.25 rows=1 width=13) (actual time=9.012..9.012 rows=0 loops=3) Filter: ((closed_reason IS NULL) AND ((worker_id)::text = 'w__________k'::text)) Rows Removed by Filter: 54544 Planning Time: 0.398 ms Execution Time: 13.808 ms Query 2 explain plan after index: Limit (cost=2.82..20.03 rows=1 width=253) (actual time=0.132..0.134 rows=1 loops=1) -> Nested Loop Left Join (cost=2.82..20.02 rows=1 width=253) (actual time=0.131..0.133 rows=1 loops=1) Join Filter: ((w.public_id)::text = (session_connection.worker_id)::text) CTE worker_config_tags -> GroupAggregate (cost=1.71..2.26 rows=22 width=59) (actual time=0.046..0.086 rows=22 loops=1) Group Key: ct_1.worker_id, ct_1.source -> Sort (cost=1.71..1.77 rows=22 width=49) (actual time=0.033..0.035 rows=22 loops=1) Sort Key: ct_1.worker_id, ct_1.source Sort Method: quicksort Memory: 27kB -> Seq Scan on server_worker_tag ct_1 (cost=0.00..1.22 rows=22 width=49) (actual time=0.007..0.011 rows=22 loops=1) -> Nested Loop Left Join (cost=0.14..9.28 rows=1 width=245) (actual time=0.120..0.120 rows=1 loops=1) Join Filter: ((w.public_id)::text = (ct.worker_id)::text) -> Nested Loop Left Join (cost=0.14..8.72 rows=1 width=213) (actual time=0.117..0.118 rows=1 loops=1) Join Filter: ((w.public_id)::text = (wt.worker_id)::text) -> Index Scan using server_worker_pkey on server_worker w (cost=0.14..8.15 rows=1 width=181) (actual time=0.011..0.012 rows=1 loops=1) Index Cond: ((public_id)::text = 'w__________k'::text) -> CTE Scan on worker_config_tags wt (cost=0.00..0.55 rows=1 width=64) (actual time=0.104..0.105 rows=0 loops=1) Filter: (((worker_id)::text = 'w__________k'::text) AND (source = 'api'::text)) Rows Removed by Filter: 22 -> CTE Scan on worker_config_tags ct (cost=0.00..0.55 rows=1 width=64) (actual time=0.002..0.002 rows=1 loops=1) Filter: (((worker_id)::text = 'w__________k'::text) AND (source = 'configuration'::text)) Rows Removed by Filter: 8 -> GroupAggregate (cost=0.42..8.45 rows=1 width=21) (actual time=0.010..0.010 rows=0 loops=1) Group Key: session_connection.worker_id -> Index Only Scan using session_connection_worker_id_closed_reason_ix on session_connection (cost=0.42..8.44 rows=1 width=13) (actual time=0.010..0.010 rows=0 loops=1) Index Cond: ((worker_id = 'w__________k'::text) AND (closed_reason IS NULL)) Heap Fetches: 0 Planning Time: 0.544 ms Execution Time: 0.217 ms Closes #3281	3 years ago
Johan Brandhorst-Satzkorn	610df9892b	all: propagate more history metadata (#3279 ) A few target and host properties were missed when reading historical data from the history tables.	3 years ago
Michael Gaffney	8159ddb08e	feat(db): Add function to update billing snapshot table This adds the `update_sessions_pending_daily_snapshot` SQL function which updates the `sessions_pending_daily_snapshot` table by inserting values for any days missing from the table. This also adds a check constraint to the `sessions_pending_daily_snapshot` table along with a `comment on table`. This also removes the hcp daily billing views since they are no longer needed. Ref: `ace2def49d`	3 years ago
Michael Gaffney	41bdfb4406	fix(db): Fix time zone consistency issues in monthly billing views The hcp monthly billing views build a range of buckets using the `generate_series` and the `now()` functions along with relative offsets like `+ 1 month`. The calculated start and end times for these views can vary depending on the time zone setting of the PostgreSQL server. This adds 3 table functions that call their respective views with the time zone set to UTC. Calling these functions instead of their underlying views directly prevents the returned values from varying when the server's time zone is not set to UTC. This also drops support for PostgreSQL 11. The included tests use the `date_trunc(field, source [, time_zone ])` function added in PostgreSQL 12. Fixes: `41b1b5b4b7`	3 years ago
Michael Gaffney	47ddd2e40a	fix(db): Fix generated series to yield consistent results The hcp billing `_all` views built a range of buckets using the `generate_series` function using the minimum value of the `session_pending_time` column from the `wh_session_accumulating_fact` table as the start time and `now()` as the end time. It then truncated values in the series using the `date_trunc` function. This allowed the generated series of buckets returned to vary and could result in one additional bucket being included in the series depending on the current time and the start time. As an illustrative example, the `hourly_all` query used: select date_trunc('hour',time) from generate_series( (select min(session_pending_time) -- start time from wh_session_accumulating_fact), now(), '1 hour'::interval ) as time; Substituting the start time with a hard coded value to show the issue: -- Show current time, where the minutes portion is 55 select now(); now ------------------------------- 2023-05-22 14:55:40.567605+00 (1 row) -- generate series with start time's minutes portion at 57 select date_trunc('hour',time) from generate_series( '2023-05-22 01:57:00'::timestamptz, now(), '1 hour'::interval ) as time; date_trunc ------------------------ 2023-05-22 01:00:00+00 2023-05-22 02:00:00+00 2023-05-22 03:00:00+00 2023-05-22 04:00:00+00 2023-05-22 05:00:00+00 2023-05-22 06:00:00+00 2023-05-22 07:00:00+00 2023-05-22 08:00:00+00 2023-05-22 09:00:00+00 2023-05-22 10:00:00+00 2023-05-22 11:00:00+00 2023-05-22 12:00:00+00 2023-05-22 13:00:00+00 (13 rows) -- Wait a few minutes select now(); now ------------------------------ 2023-05-22 14:58:48.75287+00 (1 row) -- Now run again and get an extra result, even though still -- within the same hour select date_trunc('hour',time) from generate_series( '2023-05-22 01:57:00'::timestamptz, now(), '1 hour'::interval ) as time; date_trunc ------------------------ 2023-05-22 01:00:00+00 2023-05-22 02:00:00+00 2023-05-22 03:00:00+00 2023-05-22 04:00:00+00 2023-05-22 05:00:00+00 2023-05-22 06:00:00+00 2023-05-22 07:00:00+00 2023-05-22 08:00:00+00 2023-05-22 09:00:00+00 2023-05-22 10:00:00+00 2023-05-22 11:00:00+00 2023-05-22 12:00:00+00 2023-05-22 13:00:00+00 2023-05-22 14:00:00+00 (14 rows) To fix this, we call the `generate_series` function truncating the start time argument to the bucket size: -- Hourly select bucket from generate_series( date_trunc('hour', select min(session_pending_time) from wh_session_accumulating_fact)), now(), '1 hour'::interval ) as bucket; -- Monthly select bucket from generate_series( date_trunc('month', select min(session_pending_time) from wh_session_accumulating_fact)), now(), '1 month'::interval ) as bucket; Fixes: `41b1b5b4b7`	3 years ago
Michael Gaffney	053e74f66c	feat(db): Add table for census upload job status	3 years ago
Michael Gaffney	8bf80e0544	test(db): Conditionally skip monthly billing view tests Certain tests fail when run on the first day of the month because of edge cases in the setup of test data. This adds conditional logic to skip those tests when run on the first day of a month.	3 years ago
Jim	d4e537c88d	chore: updgrade pgx direct deps to v5 (#3258 )	3 years ago
Jim	536551dc0c	fix: update go-dbw dep and refactor conversion of pgx errors (#3242 )	3 years ago
Timothy Messier	61a37ef4a4	fix(schema): Add missing endpoint column to session recording view	3 years ago
Timothy Messier	f53d65eacd	refact(sql): Rename migrations to avoid number conflicts	3 years ago
Todd	4063880bbe	Add credential history views	3 years ago
Michael Gaffney	a783c91b2d	feat(sql): Add dynamic credential hst tables to session recording This adds the `recording_dynamic_credential` join table and a trigger that runs as an after insert on the `recording_session` table. The join table joins the `recording_session`, `credential_vault_store_hst`, `credential_library_history_base`, and the `credential_purpose_enm` tables.	3 years ago
Michael Gaffney	3769f0851c	test(sql): Refactor the colors persona This changes multiple IDs in the colors persona to be 12 characters. It also renames some IDs in an attempt to make them more consistent. This also changes the order of the columns in some insert statements to follow the pattern of the other insert statements. This also adds documentation to the top of colors persona file in an attempt to explain the general patterns being followed in the file.	3 years ago
Michael Gaffney	03b1291b34	feat(sql): Add history tables for dynamic credentials This adds history tables for `credential_vault_store`, `credential_vault_library`, and `credential_vault_ssh_cert_library`. This also adds a base table for the credential library history tables.	3 years ago
Todd	e1540be9f3	Add host information to session recording view	3 years ago
Michael Gaffney	a8224f4f57	feat(sql): Add endpoint from session table to session recording This adds an `endpoint` column to the `recording_session` table and updates the session recording insert trigger to copy the value of `endpoint` from the `session` table to the `recording_session` table.	3 years ago
Michael Gaffney	06e4e46248	feat(sql): Add static credential hst tables to session recording This adds the `recording_static_credential` join table and a trigger that runs as an after insert on the `recording_session` table. The join table joins the `recording_session`, `credential_static_store_hst`, `credential_static_history_base`, and the `credential_purpose_enm` tables.	3 years ago
Michael Gaffney	3a64fb3d58	feat(sql): Add history tables for static credentials This adds history tables for `credential_static_store`, `credential_static_json_credential`, `credential_static_username_password_credential`, and `credential_static_ssh_private_key_credential`. This also adds a base table for the static credential history tables.	3 years ago
Todd	386eac072c	Add recording_channel_ssh_aggregate view	3 years ago
Johan Brandhorst-Satzkorn	11893a8885	db: add state and error_details to recording	3 years ago
Johan Brandhorst-Satzkorn	af8e4d63be	db: add channel summary details	3 years ago
Todd	ee6a5f05b5	Add historical target and scope information to session recording	3 years ago
Michael Gaffney	477e6df269	feat(sql): Add target and host history tables to session recording This adds four columns to the `recording_session` table which are foreign keys to the `iam_scope_hst`, `target_ssh_hst`, `host_history_base`, and `host_catalog_history_base` tables. The foreign key to the `iam_scope_hst` table references the project scope of the target at the time of the recording.	3 years ago
Michael Gaffney	73b710cd06	refactor(sql): Rename session recording migration This renames the migration with the session recording table to come after the history table migrations. This allows the session recording table to have foreign key references to the history tables without using alter table statements.	3 years ago
Michael Gaffney	37c6b8e540	feat(sql): Add history tables for hosts This adds history tables for `static_host` and `host_plugin_host`. This also adds a base table for the host history tables. This also adds a new test to the history table test suite. The new test verifies the contents of each history table against the contents of their operational table.	3 years ago
Michael Gaffney	b97dc07bb0	feat(sql): Add history tables for host catalogs This adds history tables for `static_host_catalog` and `host_plugin_catalog`. This also adds a base table for the host catalog history tables and an immutable table with one row for when a target has a direct address assigned to it. This also adds a reusable trigger function for tables that are immutable.	3 years ago
Michael Gaffney	7ebe393aee	feat(sql): Add history table for target_ssh This adds a history table for `target_ssh`. This also fixes a bug in the history table test suite. Prior to this change, the operational table name was calculated from the history table name using the SQL trim() function. The SQL trim function as used returned 'targe' for 'target_ssh_hst' instead of the expected value 'target_ssh'.	3 years ago
Todd	d365123045	add session recording aggregate view	3 years ago
Louis Ruch	7263d08bc5	feat(storage): Add target storage_bucket relation	3 years ago
Timothy Messier	21f6be9ab5	refact(sql): Move migrations to avoid number conflict	3 years ago
Michael Gaffney	882b7eca2f	feat(sql): Add IAM history table references to session recording This adds two columns to the `recording_session` table. One column is a foreign key to the `iam_user_hst` table and the other is a foreign key to the `iam_scope_hst` table. The foreign key to the `iam_scope_hst` table reference the IAM scope the user belongs to at the time of the recording.	3 years ago
Michael Gaffney	9fd01a79f0	refactor(sql): Move and rename session recording migration This moves and renames the migration with the session recording table to come after the history table migrations. This allows the session recording table to have foreign key references to the history tables without using alter table statements.	3 years ago
Michael Gaffney	4488f5f8cd	refactor(sql): Move and rename history table migrations This moves and renames the history table migration files into the correct migration directory.	3 years ago
Michael Gaffney	463f1e36af	feat(sql): Add history table for iam_user This adds a history table for `iam_user`.	3 years ago
Michael Gaffney	8ae658a570	feat(sql): Add history table and triggers for iam_scope This adds a history table for `iam_scope`. It also includes reusable triggers to maintain the included history table and future history tables. This also includes a test suite that runs on any table ending in `_hst`.	3 years ago
Michael Gaffney	327d950a1e	feat(sql): Add function to generate a URL safe ID This adds a SQL function that generates a secure random ID of at least 14 characters suitable for use as a public or private ID. IDs generated with the function will only contain characters from the unreserved character set defined in RFC 3986 Section 2.3 making them safe for use in a URL.	3 years ago
Michael Gaffney	94f77d36b3	feat(sql): Add a domain type for URL safe IDs This creates a new domain in SQL that constrains the characters of an ID to the unreserved character set defined RFC 3986 Section 2.3.	3 years ago
Michael Gaffney	647cf0d33d	feat(sql): Add URL safe base64 encoding function This adds a SQL function to encode binary data into the URL safe base64 encoding defined in RFC 4648.	3 years ago
Danielle Miu	a4a14fc7fc	merged commit for plugin rework, storage bucket sql and proto, storage bucket secret rewrapping, storage bucket repository	3 years ago
Johan Brandhorst-Satzkorn	7bb44b46f5	internal/db: add session recording tables Add the following new tables used for storing session recordings: * recording_session * recording_connection * recording_channel * recording_channel_ssh * storage_bucket	3 years ago
Michael Milton	ace2def49d	Add daily sessions pending view and accompanying snapshot table (#3217 ) * feat: add a new daily billing view * update views and table based on early feedback * address review comments * update CTE to be an accurate result of what we expect * remove 'with time zone' from daily range * remove timezone * Update internal/db/schema/migrations/oss/postgres/70/01_hcp_billing_daily.up.sql Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Timothy Messier <tim.messier@gmail.com> * go mod updates * fix tests --------- Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com> Co-authored-by: Timothy Messier <tim.messier@gmail.com>	3 years ago
Jeff Mitchell	bbe9719e19	Add tracking of worker used for injection (#3204 )	3 years ago
Haotian	762e0d7d0d	fix(sql): move added file to correct folder (#3197 ) * move rename_constraints.up.sql from 67 to 69 * update sql replacement reference comments	3 years ago
Haotian	b806297afe	style(db): rename constraint names to follow sql style guide (#2973 ) * programatically rename unique and foreign key constraints to enforce 'tablename_uq' and 'reftable_fkey' patterns * manually rename unique constraints for names over 63 characters * update changed constraint names in db update testcases	3 years ago
Jeff Mitchell	20391e3503	Add default client port to targets and use in connect command (#2767 )	3 years ago
Johan Brandhorst-Satzkorn	ecdb77b584	internal/db/sqltest: fix sessions aggregate (#3152 ) The sessions aggregate was inserting host_id and host_set_id, columns which are no longer present on the session table. Also fix the test that guards against regressions like this in the future.	3 years ago
Hugo Vieira	2ff9b8efac	feat(host/plugin): Support External Name field The new `ExternalName` field is an optional field that is used to store the cloud providers' human-identifiable name for a given compute instance. This field is not user-editable, but it is kept in sync with the cloud provider's through Boundary's `SetSyncJob`.	3 years ago
Johan Brandhorst-Satzkorn	bc31892f29	internal/db: add set_once_columns function (#3117 ) * internal/db: add set_once_columns function Also replace existing before_soft_delete_credential_vault_store function and trigger with the new function. * internal/db/schema: add note to replaced definitions	3 years ago
Jim	63a2b62b85	chore (deps): update GKW and protobuf; and run "make gen" (#3111 )	3 years ago
Jim	86192f75eb	feature (auth/ldap): add LDAP auth method along with associated accounts and managed groups (#2912 ) * feature (auth): required schema changes for auth ldap method (#2669) chore (auth/ldap): move schema changes to next avail migration number * feature (auth/ldap): define AuthMethod and all its value objects (#2703) * feature (auth/ldap): storage protos * feature (auth/ldap): define AuthMethod and all its value objects * feature (auth/ldap): add repo and reading an auth method (#2718) * feature (auth/ldap): add Repository.CreateAuthMethod(...) and Repository.DeleteAuthMethod(...) (#2724) * feature (auth/ldap): add Repository.UpdateAuthMethod(...) (#2739) * feature (auth/ldap): add Account * feature (auth/ldap): add AuthMethod.EnableGroups * fix (auth/ldap): refactor AuthMethod.oplog to enforce proper constraints * feature (auth/ldap): add Account repo functions * refactor (auth/ldap): remove entry attributes from account Realized the entry attributes could have absolutely anything in them (including binary data) and since we absolutely don't have to have them there's just no reason to take on the risk. * feature (auth/ldap): add AuthMethod.UseTokenGroups * feature (auth/ldap): add Authenticate(...) * refactor (auth/ldap): ensure options take a context as the 1st parameter * feature (auth/ldap): add Account attribute maps * chore (auth/ldap): make fmt deltas * feature (auth/ldap): add managed groups (#2760) * tests (auth/ldap): add missing unit test to Repository.DeleteAccount(...) Add bits to test the delete operation when you're not able to generate oplog metadata * feature (auth/ldap): add managed groups fixup! feature (auth/ldap): add managed groups (#2760) * feature (auth/ldap): service proto definition (#2761) * feature (handlers/authmethods): add handlers for ldap auth method operations (#2794) * feature (auth/ldap): add Account attribute maps * chore (auth/ldap): update cap/ldap to latest version * feature (auth/ldap): add ldap api generation definitions * feature (authmethods): add ldap repo NewService(...) * feature (authmethods/ldap): add proper mask_mapping to protobufs * feature (authmethods): add support to get an ldap auth method * refactor (auth/ldap): export TestGenerateCA(...) * feature (authmethods): add support to create an ldap auth method * feature (authmethods): add support to delete an ldap auth method * feature (authmethods): add support to list ldap auth methods * refactor (auth/ldap): make urls optional for NewAuthMethod(...) * refactor (auth/ldap): export ldap.TestInvalidPem * chore: make fmt changes * fix (auth/ldap): properly handle group search config Add constraints and tests to ensure when an ldap AuthMethod.EnableGroups is true, and UseTokenGroups is false; that there's a GroupDn configured for finding a user's associated groups * feature (authmethods): add support to update an ldap auth methods * chore (db/ldap): tmp mv migrations so there's no conflict with ongoing work * feature (verifier): add ldap auth method to verifier bits * fix (controller): prevent panic when controller stops when there's no listener * feature (authmethods): add support to authenticate via ldap auth methods * chore (migrations): fix whitespace in stmt * chore: fmt fixup * tests (auth/ldap): invalid err msg * feature (cli/authmethods): add support for ldap auth-methods CRUD and authenticate (#2810) * feature (authmethods): add CLI support for ldap auth methods CRUD * tests (api/auth): ldap auth method classification tests * feature (authmethods): add CLI support for ldap auth authenticate * feature (auth/ldap): set request timeouts for ldap server connections * feature (handlers/authmethods): handle u_anon listing properly. * feature (account/handers): ldap account and managed group CRUDL APIs (#2852) * feature (auth/ldap) add repository Listing of ManagedGroupMemberAccount * feature (controller/handlers): add ldapRepoFn to accounts service * feature (auth/ldap) register ldap managed group subtype * feature (account/handlers): ldap account CRUDL APIs * feature (controller/handlers): add ldapRepo to managed groups service * feature (account/handlers): ldap managed group CRUDL APIs * docs (domain): add LDAP accounts, auth-methods and managed groups (#2857) * feature (ldap/cli) add ldap accounts and managed groups CRUDL commands (#2856) * fix (handlers/authmethods): fix ldap authorized actions (#2892) * feature (cli/ldap/authenticate): use primary auth method if none is provided (#2890) * fix (auth/ldap): support setting the state attribute * feature (cli/ldap/authenticate): use primary auth method if none is provided * feature (wh/ldap) add tests for new ldap auth method and accounts (#2919) * refactor (migrations/ldap): mv to correct directory * chore: add copyright headers * fix (api/authmethods/ldap): renumber new LdapAuthMethodAttributes field * fix (auth/ldap): allow the auth method state to be updated (#2951) * chore: update sdk and api versions for llb - this is tmp until merging * tests (managed groups): add required errContains for new test	3 years ago
Irena Rindos	ab0a2ad689	Fix hcp billing test (#3045 ) * fix test failure	3 years ago
Irena Rindos	860af44556	Fix sql tests failing on the first of the month (#3042 ) * initial commit	3 years ago
Hugo	41b1b5b4b7	feat(db): Introduce migrations for ssh targets (#2960 ) Adds the schema for ssh targets. This also changes the target listing func to not error out when it encounters a target with an unknown type. Instead, it now ignores it and continues to the next one.	3 years ago
Timothy Messier	5c19dba3c9	ci(sqltest): Add github action workflow for sql tests (#2977 )	3 years ago
Johan Brandhorst-Satzkorn	3c29308673	chore: Add license headers to all files	3 years ago
Timothy Messier	6a41bf1237	feat(wh): Support vault ssh certificate credentials This expands the wh_credential_dimension to include fields for the new vault ssh certificate credential library.	3 years ago
Danielle	3a78a74571	feat(vault): Support Issuing SSH Cert Credentials Allows the new credential library type to be pulled from the db and used for issuing ssh certificate type credentials. The ssh cert library interfaces with vault's ssh secrets engine to either issu certificates directly from vault, or to generate them within boundary and have vault sign them as a certificate authority for authentication. See: https://developer.hashicorp.com/vault/api-docs/secret/ssh#sign-ssh-key https://developer.hashicorp.com/vault/api-docs/secret/ssh#generate-certificate-and-key	3 years ago
Haotian	823b8cdfc6	feat(vault): Add CRUDL methods for ssh certificate cred library (#2765 ) This defines a new credential library type for ssh certificates. It adds new repository methods to support CRUDL of the new ssh certificate credential library. Co-authored-by: Timothy Messier <tim.messier@gmail.com>	3 years ago
Johan Brandhorst-Satzkorn	55259029f1	Add job run cleaner (#2866 ) * chore: remove duplicate import * feat: Add job run cleaner The new job cleans job runs older than a configured interval. Configure the job to clean jobs older than 1 minute.	3 years ago
Irena Rindos	2e3fd041ef	Target worker filters migration fix (#2859 )	3 years ago
Timothy Messier	e2248c1360	fix(wh): Associate session fact with credential dimensions (#2787 ) The wh_upsert_credential_group function was incorrectly attempting to update the session connection fact table. There are a couple problems with this: 1. At the time that this runs, there would be no session connection facts for this session. 2. This function is meant to update the session fact table, there is a separate function that associates the credential group with the session connection facts. This also fixes another bug that was causing expired rows of the credential dimension to be included in the group membership. And finally it fixes a typo in the function name.	3 years ago
Damian Debkowski	a8ee18e177	fix(db): target address relabel migration id (#2769 )	3 years ago
Damian Debkowski	0dbe18e453	fix(db): remove migration test 59_01 This test uses the target domain repository code to test the ingress & egress worker filter. With our changes we have updated the target domain repository logic to fetch the target address. When this migration test is invoked, the target_address table does not exist and the domain code will always return an error making this test invalid.	3 years ago
Damian Debkowski	3dd4b1c63d	feat(db): Support a direct address association to a Target	3 years ago
Jim	1c9ca14a8e	fix (db): Improve error handling for commit errors. (#2728 ) We were not capturing the errors which happen on commit. We've changed to a multierror so we should get the commit error and any subsequent rollback errors in the same txn.	3 years ago
Danielle	46037cf85a	Add Staticcheck to Build Pipeline (#2421 ) Run staticcheck linters on pull requests	3 years ago
Johan Brandhorst-Satzkorn	eb113b5c9e	fix(db): Reorder migrations (#2688 ) Migration numbers need to be consistent between version upgrades. The migrations for the key lifecycle management were merged into v0.11.1 as number 56, but are currently 58 in main. This reorders them down to 56, fixing an issue when migrating from 0.11.1 to main.	3 years ago
Irena Rindos	834a2a88f7	feat(targets): Addition of egress and ingress worker filters (#2654 ) * feat(targets): Addition of egress and ingress worker filters	3 years ago
Haotian	96a630905f	formatting consistency for 'create function' statements (#2620 )	3 years ago
Johan Brandhorst-Satzkorn	8908dccf6d	Forward port some release fixes (#2631 ) * fix(worker): Verify contents of loaded session (#2629) * fix(worker): Verify contents of loaded session Previously, it was possible for an empty private key to be used when the session had no user or project associated with it. The new checks guarantee that we will fail gracefully in these cases. * Check against len instead of nil * Add CHANGELOG * fix(session): Always invoke all parts of session cancel trigger The session cancelation trigger would not set the key_id to null appropriately if another one of the fields on the session was already null, since that case statement would be matched first. The new structure matches all statements, in case any of them have special logic (such as in the project case).	3 years ago
Johan Brandhorst-Satzkorn	edd323b73a	Key Rotation/Destruction (#2477 ) (#2607 ) * Key Rotation/Destruction (#2477) * fix: Correct kms.CreateKeys comment This method does not return anything. * feat(kms): Add new ListKeys method The new ListKeys method wraps the underlying KMS wrapper, returning all the keys in the scope specified. * feat(scopes): Add new scope actions for key management Adds list-keys, rotate-keys and revoke-keys actions for scopes. * feat(scopes): Add ListKeys API endpoint This new endpoint lists all keys in a scope * feat(api): Add ListKeys to Scopes client This has to be a custom function because it is a custom action and doesn't map well to any of the existing templates, or generalises well in a way that would make it reasonable to create a new template. * feat(cli): Add new scopes list-keys command * add RotateKeys function to the kms repository * Add Rotate Keys Endpoint to Scopes Api (#2360) * add rotate keys endpoint * add tests for RotateKeys endpoint * remove that one comment I forgot about * addressing more PR comments * Add Rotate Keys CLI Command (#2395) * add cli command and client * add bats tests * fix some info text and pr comments * write a new rotate keys description * Add Missing DEK Foreign Keys (#2408) * add missing fks * mark key as nullable in gorm * update tests to use a new wrapper with a real key_id * fix formatting in test and add keys for auth_token test * replace key values * revert postgres_40_01_test.go * style: reformat sql for readability * refactor: change type of key_id columns to kms_private_id * Store key_id and scope_id with oplog entries, re-type columns referencing data key version (#2431) * fix(db): Correct types of data key version referencers * fix(oplog): Fix missing error handling * feat(oplog): Store key_id and scope_id with oplog entries This migration truncates all existing oplog entries, as migrating existing data would have been complex and likely unnecessary. * Update view references * Always delete oplog entries when dropping keys * Fix rebase issues * Fix sql tests * Add schema for new kms destruction jobs (#2479) * feat: Add schema for new kms destruction jobs The schema lets us manage destruction jobs per-table while providing a guarantee that only one run is running at a time. * Apply suggestions from code review Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com> * Fix table reference and tests Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com> * update domain, api, and cli to all include key versions in outputs (#2472) * update domain, api, and cli to all include key versions in outputs * fix go sum * add migration to fix immutable error * fix final test issue as well as make gen issue * rebase and fix * address johan pr comments * make gen, didn't realize comments were included * update migrations based on new migration 55 file 03 * Move migrations to 56 * feat: Add key version rewrapping function registry (#2478) The new RegisterTableRewrapFn can be used by a domain to register a callback to use when rewrapping data in a specific table name. * feat(kms): Add scopeId to RewrapFn (#2539) This makes it much easier for rewrap functions to look up the correct wrapper and limit the number of rows searched. * Update Root Cert Proto Definition (#2542) * update root cert proto definition to properly identify public_key as the table primary key * add the test fix as well * Rewrap Functions for Encrypted Tables (#2532) * add oidc rewrap * add argon2 config rewrap * add auth token rewrapping * add worker auth cert rewrapping * check the err * add username password credential rewrapping * add ssh private key credential rewrapping * add vault client certificate rewrap function * add host catalog secret rewrap function * cleanup a little * add host vault token rewrap function * add session credential rewrap function * add session rewrap function * add worker auth rewrap function * update test to include full assert gamut * add session rewrap function TEST * bring all tests up to current standards, make more readable, and include full assert gamut * rework all rewrap functions to utilize scope id * update comments, queries, sql refs, etc, in response to PR comments * forgot a comment * remove hmac updates and fix a couple comments again * Rewrapping Registered Tables Test (#2555) * add the registered table test as well as the two missing rewrap functions * address pr comments, add db r/w conversions, cmp.diff, sql comment, remove (now) faulty immutable test * Add ability to list key version destruction jobs, recurring jobs and destroying key versions (#2498) * feat(kms): Add ability to list data key version destruction jobs * feat(scopes): Add ListKeyVersionDestructionJobs * feat(cli): Add list-key-version-destruction-jobs * feat(kms): Add recurring job that performs table rewrapping The new recurring job runs for each job table with a registered rewrapping callback. It attempts to become the running run and start its rewrapping, gracefully handling sudden interruptions by resuming it's work if it finds evidence of sudden interruption. * feat(kms): Add recurring job for destroying key versions The new job monitors the database for data key version destruction jobs that have finished rewrapping all its data, performing the final data key version revocation. * feat(kms): Add DestroyKeyVersion DestroyKeyVersion immediately destroys a key version if it is a root key version or a data key version which encrypts no data. If the data key version encrypts data which needs to be rewrapped, it queues an asynchronous job to complete the rewrapping operation. * feat(scopes): Add DestroyKeyVersion API endpoint * feat(cli): Add scopes destroy-key-version CLI command * Ensure all secret updates include key ID (#2556) * feat(all): Ensure all secret updates include key ID When updating a secret, it is paramount that the key ID is also updated, as it is the only means we have of ensuring that we only destroy keys once there is no more data associated with the key. * Remove fix to session repository for now This breaks the current private key derivation method * add new encrypt/decrypt funcs and made necessary proto changes (#2557) * add new encrypt/decrypt funcs and made necessary proto changes * address all PR comments * fix test panic * add param checking to rewraps * fix(schema): Correctly organize migrations again * Review comments part 1 * Review comments part 2 * Review comments part 3 * Store cert private key, encrypt tofu token (#2583) * fix(session): store cert private key, encrypt tofu token Previously, we would derive a session certificate private key from the session key used in each project, and not store the key. We would also encrypt the tofu token with the database key but not store a reference to this key in the database. This change fixes this by replacing our key derivation with a key generation step, and instead store the generated key, encrypted, in the database. We also ensure that any new tofu tokens are encrypted with the same key. To handle existing sessions, we lazily rewrite the sessions whenever a user lists them. There is a minor risk that a user could end up destroying the database key that encrypts the tofu token before that session has been rewrapped, but this is going to be rare and temporary. * feat(session): Allow the random source for secrets to be configured * Review comments * Minor fixes * More minor fixes * More small fixes Co-authored-by: Danielle Miu <dani.miu@hashicorp.com> Co-authored-by: Danielle <29378233+DanielleMiu@users.noreply.github.com> Co-authored-by: Michael Gaffney <mike@gaffney.cc> Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com> * fix(session): Unset session key ID when scope is deleted Unsetting the key ID allows the session to live beyond the lifetime of the scope, while allowing the scope to cascade delete its keys. * fix(migrations): Rewrite migration 56 tests These tests can no longer use the normal test helpers as they try to use the new oplog table structure, so we have to manually write all the interactions. * fix(migrations): Rename migration 58 file name This was the name used in the release branch, update it to the new migration number. * fix(e2e): Add some comments and debug to kms tests * fix(session): Handle empty project and user IDs in read When a project or user is deleted, the session is automatically canceled and the respective field on the session is unset. LookupSession did not handle this case gracefully, and would error on decryption in this state. Skip decrypting sessions that do not have either a user or project to avoid this error. The decrypted values are not used for a canceled session. * Fix worker test * Increase test timeout Co-authored-by: Danielle Miu <dani.miu@hashicorp.com> Co-authored-by: Danielle <29378233+DanielleMiu@users.noreply.github.com> Co-authored-by: Michael Gaffney <mike@gaffney.cc> Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com>	3 years ago
Haotian	d97d34adc4	refactor(db): applies consistent formatting to 'comment on' statements	3 years ago
Johan Brandhorst-Satzkorn	a216c701ee	Silence events from DB errors (#2601 ) (#2602 ) If callers want errors to create events, they can do so by wrapping the error. Sometimes we don't want db errors to be events, and before this there was no way to supress them.	4 years ago
Irena Rindos	b26814a3cc	move user variables into globals (#2580 ) * move user variables into globals	4 years ago
Michael Gaffney	f114e20c6d	docs(db): fix comments on database domain types Fixes the comments on the wt_scope_id, wt_user_id, and wt_role_id domain types.	4 years ago
Michael Gaffney	ed4b00303a	docs(db): fix comments on kms database tables Fixes the comments on the kms_data_key and kms_data_key_version tables.	4 years ago
Michael Gaffney	c9cfb0e564	docs(db): fix comments on oidc database tables Fixes the comments on the auth_oidc_method and auth_oidc_account tables.	4 years ago
Michael Gaffney	3241c9a411	docs(db): fix comments on database functions Fixes the comments on functions wt_add_seconds, wt_add_seconds_to_now, wt_sub_seconds, and wt_sub_seconds_from_now which were the tragic victims of a copypasta error.	4 years ago
Johan Brandhorst-Satzkorn	f57454b6b9	Rewrite interface{} to any (#2535 ) * chore: Rewrite interface{} to any * Add ignore revs	4 years ago
Todd	fa39fe5b0f	Delete the base auth account when a concrete auth account is deleted (#2528 )	4 years ago
Irena Rindos	4493dfd527	use prior and current worker auth keys (#2495 ) * use prior and current worker auth keys	4 years ago
Timothy Messier	a2ef14283b	fix(org): Address bug that prevented deleting an org (#2465 ) The schema changes introduced in 0.10.2 introduced a bug where an org could not be deleted if there was a session for an org scoped user. It would result in an error like: scope.(Service).deleteFromRepo: unable to delete scope: \ iam.(Repository).DeleteScope: failed for o_VkNy46KwfS: \ iam.(Repository).delete: db.DoTx: iam.(Repository).delete: \ db.Delete: insert or update on table "session" violates foreign key constraint "target_fkey": integrity violation: error #1003 A worker around would be to first delete all projects in the org, then finally delete the org. This fix adds a regression test to the sqltests and updates several constraints on the session table to be deferrable. Since the org delete is a single transaction, this allows all of the necessary resources to be deleted before attempting to enforce these constrains. See: https://www.postgresql.org/docs/current/sql-set-constraints.html Blame: `3a28ab614`	4 years ago
Damian Debkowski	546c5dc5be	feat: static json credentials (#2423 )	4 years ago
Irena Rindos	821d8317dc	address op state review feedback (#2453 ) * address op state review feedback	4 years ago
Jeff Mitchell	53b5e532d5	Remove deprecated methods/fields on targets (#2393 )	4 years ago

1 2 3 4 5 ...

641 Commits (moduli-e2e-admin-ui)