boundary

Commit Graph

Author	SHA1	Message	Date
Timothy Messier	9d82958c9f	test(sqltest): Shorten prepared statement identifiers Some of the prepared statement identifies used in these tests exceeded the identifier character limit. This results in postgres truncating the name, which is fine for a test, but does result in some noise in the test output. This change shortens the identifies to eliminate noise like the following: psql:/test/credential/vault/credential_vault_library_ssh_private_key_mapping_override.sql:49: NOTICE: identifier "insert_credential_vault_library_ssh_private_key_mapping_override" will be truncated to "insert_credential_vault_library_ssh_private_key_mapping_overrid" The tests should no longer create notices like these.	2 years ago
Jeff Mitchell	35cef97bd4	Linter fixes PR 3 (#4415 ) * Linter fixes * fix(bsr): update test to register TEST protocol once --------- Co-authored-by: Damian Debkowski <damian.debkowski@hashicorp.com>	2 years ago
Jeff Mitchell	25bce9e8ff	Fix linter warnings	2 years ago
Johan Brandhorst-Satzkorn	d9c6de4e8b	deps: update runc dependency (#4349 ) Fixes GHSA-xr7r-f8xq-vfvv	2 years ago
Jeff Mitchell	18f863514e	Finish sqltests for multi grant scopes (#4320 )	2 years ago
Jeff Mitchell	025fce6f95	Change error message and add beginnings of sql tests (#4319 )	2 years ago
Johan Brandhorst-Satzkorn	133e928c6d	internal/db/schema: update recording indexes (#4278 ) The new index includes the new delete_time and delete_after columns, which are used in the list query.	2 years ago
Jeff Mitchell	f5dd457b34	Multiple grant scope IDs (#4251 ) Co-authored-by: Timothy Messier <tim.messier@gmail.com> Co-authored-by: Todd <github@quaddi.com>	2 years ago
Louis Ruch	fa69ddcb8a	fix(db): Set max storage policy days (#4267 )	2 years ago
Jim	6a37c93d77	fix: properly handle rows.Err() (#4261 ) * fix (auth): properly handle rows.Err() * fix (auth/ldap): properly handle rows.Err() * fix (auth/oidc): properly handle rows.Err() * fix (auth/password): properly handle rows.Err() * fix (authtoken): properly handle rows.Err() * fix (credential): properly handle rows.Err() * fix (credential/static): properly handle rows.Err() * fix (credential/vault): properly handle rows.Err() * fix (db): properly handle rows.Err() * fix (db/schema): properly handle rows.Err() * fix (db/migrations): properly handle rows.Err() * fix (host): properly handle rows.Err() * fix (host/plugin): properly handle rows.Err() * fix (host/static): properly handle rows.Err() * fix (iam): properly handle rows.Err() * fix (kms): properly handle rows.Err() * fix (oplog): properly handle rows.Err() * fix (pagination/purge): properly handle rows.Err() * fix (plugin): properly handle rows.Err() * fix (job/scheduler): properly handle rows.Err() * fix (server): properly handle rows.Err() * fix (session): properly handle rows.Err() * fix (target): properly handle rows.Err()	2 years ago
Johan Brandhorst-Satzkorn	b229ece914	internal/db: use more efficient index for host listing (#4256 ) In tests, the index including the catalog_id performs significantly better.	2 years ago
Louis Ruch	d620dcbd4f	feat(db): Update storage policy override default to false	2 years ago
Louis Ruch	5505833702	chore: reorder db migrations from llb	2 years ago
Danielle Miu	e903947bef	Create delete_session_recording job and associated infra	2 years ago
Damian Debkowski	c27d12c189	fix(db): add target_org_id to session_recording_aggregate table	2 years ago
Johan Brandhorst-Satzkorn	dc4ea67d6e	internal/db: changes for pagination	2 years ago
Danielle Miu	39b3bb2d1d	fix(db): allow session_recording_aggregate to return recordings correctly	2 years ago
Danielle Miu	df02651bb5	feat(db): new cols for retain and delete session recordings, update structs, and tests	2 years ago
Hugo	978db86186	feat(internal/db): Storage policies implementation (#767 ) This commit implements part of the storage policy's database model, focusing specifically on the operational tables for this feature.	2 years ago
Johan Brandhorst-Satzkorn	b6fe9824ab	internal/db: optimize session list indexes	2 years ago
Johan Brandhorst-Satzkorn	69262c386c	internal/db: renumber migrations (#4218 )	2 years ago
Johan Brandhorst-Satzkorn	85ca909720	internal/db: add iam_scope indexes These will help the iam_scope list queries	2 years ago
Johan Brandhorst-Satzkorn	d4b60241db	internal/db: storage bucket pagination migrations (#4213 )	2 years ago
Michael Milton	52b1c1b4c7	Managed group list pagination (#4208 ) * wip * managed groups pagination * address PR comments * missed one change	2 years ago
Johan Brandhorst-Satzkorn	44971b3559	internal/db: migrations for accounts pagination	2 years ago
Johan Brandhorst-Satzkorn	0715003945	internal/db: renumber session recording migration	2 years ago
Johan Brandhorst-Satzkorn	cce52a16e6	internal/db: db migrations for session recordings (#4200 )	2 years ago
Johan Brandhorst-Satzkorn	e9b90d9ce1	internal/db: add schema changes for host catalog pagination	2 years ago
Johan Brandhorst-Satzkorn	f0dbddb9e3	internal/db: ensure new columns are not null (#4197 ) Ensure the new create_time and update_time columns are marked as not null after populating the data. Also ensure that the data is populated safely with an updated query pattern inspired by the auth method pattern.	2 years ago
Todd	40d9c3fd1d	Paginate iam groups (#4194 )	2 years ago
Johan Brandhorst-Satzkorn	8dc431fef5	internal/db: renumber auth methods migration (#4199 )	2 years ago
Michael Milton	8d50a0913f	Auth Method list pagination (#4165 ) * wip * implement list pagination for auth methods * remove unused code * update tests to fix sorting * address PR comments * simplify query, add token tests * change test name * wip sql testing * add boolean column to indicate whether auth methods should be returned to unauthenticated users, remove secrets from api response * is_active_public not null * remove secrets and unwrapping * move not null to immediately after data migration * order public_id descending * fix(db): Fix auth method migrations (#4193) * wip * fix db tests * fix failing api tests * reverse order of create time and update time in test --------- Co-authored-by: Michael Gaffney <mike@gaffney.cc>	2 years ago
Todd	0fad770d8a	Paginate iam user (#4185 ) * paginate iam user	2 years ago
Johan Brandhorst-Satzkorn	b3207d8e1c	internal/host: sort public_id descending The previous sort would run the risk of missing items which collide on update or create times because of the compound comparison using strictly less than for both the timestamp and the public_id. Sorting both descending avoids this issue.	2 years ago
Johan Brandhorst-Satzkorn	d94a25bbd2	internal/host: sort public_id descending The previous sort would run the risk of missing items which collide on update or create times because of the compound comparison using strictly less than for both the timestamp and the public_id. Sorting both descending avoids thi issue.	2 years ago
Johan Brandhorst-Satzkorn	d4f5677858	internal/iam: sort public_id descending The previous sort would run the risk of missing items which collide on update or create times because of the compound comparison using strictly less than for both the timestamp and the public_id. Sorting both descending avoids thi issue.	2 years ago
Johan Brandhorst-Satzkorn	f9950a7e1c	internal/authtoken: sort public_id descending The previous sort would run the risk of missing items which collide on update or create times because of the compound comparison using strictly less than for both the timestamp and the public_id. Sorting both descending avoids thi issue.	2 years ago
Johan Brandhorst-Satzkorn	d4d1efce93	internal/session: sort public_id descending The previous sort would run the risk of missing items which collide on update or create times because of the compound comparison using strictly less than for both the timestamp and the public_id. Sorting both descending avoids thi issue.	2 years ago
Johan Brandhorst-Satzkorn	6238a533b0	internal/credential: sort public_id descending The previous sort would run the risk of missing items which collide on update or create times because of the compound comparison using strictly less than for both the timestamp and the public_id. Sorting both descending avoids thi issue.	2 years ago
Johan Brandhorst-Satzkorn	fc9b26efa7	internal/target: sort public_id descending The previous sort would run the risk of missing items which collide on update or create times because of the compound comparison using strictly less than for both the timestamp and the public_id. Sorting both descending avoids thi issue.	2 years ago
Johan Brandhorst-Satzkorn	27393a8227	internal/db: add host set indexes	2 years ago
Johan Brandhorst-Satzkorn	21c593ab71	internal/db: add host indexes	2 years ago
Todd	5aa560387c	Add pagination for iam Roles (#4161 ) * Add pagination for iam Roles	2 years ago
Johan Brandhorst-Satzkorn	77b76ae685	internal/db: reorder pagination schema files	2 years ago
Johan Brandhorst-Satzkorn	22ce2b4468	internal/db: add cred store pagination schema changes Add create time, update time columns to base table, add indexes used by list queries.	2 years ago
Johan Brandhorst-Satzkorn	780dfe20ad	internal/db: move pagination migrations to same folder (#4142 ) Since rebasing on main, we are no longer working in 79, but 80 instead.	2 years ago
Todd	5daef58f54	SQLite allows having 1 set to MaxOpenConnections (#3941 ) * let sqlite have a single concurrent connection	2 years ago
Jim	fba9d1cdf1	Add client daemon repo, server, and cmd The commands added are: boundary daemon start boundary daemon stop boundary daemon add-token boundary daemon status boundary search If these actions are performed on a platform that doesn't support sqlite natively in go an error is returned. Besides some hooks into the cmd package all the significant logic resides in internal/clientcache feat (cmds/authenticate): output the token name when storing token feat: update go-dbw which now uses pure go sqlite driver refact (base.Command): export FlagOutputCurlString Exporting FlagOutputCurlString allows other commands that embed the base.Command to access the flag as needed feat: add a cache Repository to manage cache storage feat: add a "cache" CLI command This new command will support both managing and searching the local CLI cache daemon. feat: start cache daemon as needed when issue CLI commands Move files to new directories and add persona table (#3526) * Add persona table and move files to new directories Client daemon now uses unix sockets and update pathing (#3535) * Add unix socket listen and dialers with file permissions applied * Update the pathing to match RFC Cache refreshing now uses all stored personas. (#3545) Updating headers for LLB files Decouple daemon and cache server (#3566) * Separate daemon from server * Add persona handler to daemon server (#3579) Moved search to use mql and added tests for the search command (#3589) * search uses hashicorp/mql Windows support for client side daemon (#3618) * exec.Command for background process * Log to stderr when in foreground and file always * Use pid files across platform for start and stop Requests with version info must match the daemon's version (#3640) * requests with version info must match daemons Update dependencies and breakout cgo/pure go sqlite drivers (#3657) upgrade go-dbw deps and add cgo build tags Enable graceful shutdown in windows (#3695) * Enable graceful shutdown in windows Co-authored-by: Jim <jlambert@hashicorp.com> Targets delete when a persona changes address or user id (#3673) * Handle cache refresh in repo Add sessions to the client side cache (#3682) * Add sessions to repository * Search command supports sessions * adding session related tests to db and refresh Change cache schema from personas to tokens+users (#3737) * Change from personas to tokens+users * Separate keyring tokens from auth tokens Add support for keyringless auth tokens (#3765) * Add support for keyringless auth tokens Remove expired auth tokens from cache (#3768) * Remove expired auth tokens from cache Complete the TODO of checking for inflight tx (#3779) Delete tokens that the controller says are invalid (#3782) fix search test with boundary token check (#3799) Collect resource info in repository_<resource>s.go (#3769) * Collect resource info in repository_<resource>s.go * Move search logic into the domain (#3794) Daemon and search restricted to supported platforms (#3801) * Daemon and search restricted to supported platforms Reorganize the clientcache packages (#3814) * Refactor for self contained domain logic * Put clientcache stuff under internal	2 years ago
Michael Milton	6ba3676c37	Auth token pagination (#4113 ) * Add pagination support for auth tokens * change analyze count estimate comment * adjust queries for efficiency and security * even more efficient queries	2 years ago
Johan Brandhorst-Satzkorn	fcb47f3772	internal/db: add cred library pagination schema changes	2 years ago
Johan Brandhorst-Satzkorn	497c158b9d	internal/db: add migration for static credential table The static credential table is used rather than the credential table as all credentials returned by ListCredentials are static at this time.	2 years ago
Michael Milton	9c3eb16046	Sessions list pagination (#4079 ) * Session list pagination refactor * address pr comments, add explanatory comments, clean up sql queries, move tests into proper files * fix build error * fix test error	2 years ago
Johan Brandhorst-Satzkorn	6181f34497	internal/db: move new migrations to new folder	2 years ago
Johan Brandhorst-Satzkorn	f0f7e635a0	internal/db: add target schema for pagination design The new design uses the base table for querying, so the base table needs to have both create_time and update_time. In this commit we add the update_time to the target base table. We also add a number of new indexes to help with looking up targets.	2 years ago
Todd	5bfc8a88b1	Add indexes on update_time (#3974 ) * Add indexes on update_time Co-authored-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com> Co-authored-by: Michael Gaffney <mike@gaffney.cc>	2 years ago
Johan Brandhorst-Satzkorn	43cff11865	internal/db: renumber migration	2 years ago
Johan Brandhorst-Satzkorn	dd2b373866	internal/db: add Now method (#3887 ) The Now() method returns the current transaction timestamp.	2 years ago
Johan Brandhorst-Satzkorn	38536f4eef	internal/db: add deleted tables for all resources (#3885 ) Creates a new foo_deleted table for each subtype of each resource that needs to support pagination. The deleted tables are used to track deletions of resources and return them to users of the new pagination API. We track only the ID and delete_time of the resource, so there's no way to filter out resources that the user could not see before it was deleted. This is intentional to keep the design simple and is not considered a security issue.	2 years ago
Damian Debkowski	2f62f65252	fix(db): make bucket_prefix column immutable for storage buckets (#4108 )	2 years ago
Elim Tsiagbey	8b8d2822df	feat(oidc): OIDC Prompt (#4053 ) Boundary OIDC method does not currently support passing in prompts during authentication. This change adds the capability of passing OIDC prompts. Prompts are optional OIDC parameters that determine the behaviour of the authentication server: https://openid.net/specs/openid-connect-core-1_0.html#AuthRequest ## Changes - New `auth_oidc_prompt` table which contains all the prompts for OIDC auth method - New `auth_oidc_prompt_enm` table which contains possible enum values for a prompt - Currently supported: - `none`: The Authorization Server MUST NOT display any authentication or consent user interface pages - `login`: The Authorization Server SHOULD prompt the End-User for reauthentication - `consent`: The Authorization Server SHOULD prompt the End-User for consent before returning information to the Client - `select_account`: The Authorization Server SHOULD prompt the End-User to select a user account - `oidc_auth_method_with_value_obj` view has been updated to return `prompt` value - Add `prompt` option for OIDC auth method CLI create and update - Pass configured prompt during OIDC authentication - Add `prompt` API validation for create and update	2 years ago
Johan Brandhorst-Satzkorn	940eb61282	sqltest: fix typo (#3886 )	3 years ago
dani	c89777de41	Expose Valid Principals for Vault SSH Signed Certs (#3791 ) * allow additional valid principals when creating vault ssh signed cert cred lib * add additional_valid_principals field to credential libraries documentation page	3 years ago
Jim	1c330543a7	fix (oplog keys): fix issue with migration of oplog keys to new tables (#3868 ) We inadvertently used kms.GetWrapper(ctx, am.ScopeId, kms.KeyPurposeDatabase) which resulted in the use of a database key for an oplog entry. See: https://github.com/hashicorp/boundary/pull/3665/files#diff-bcf83b5283886c98fc45b6435fdc94d87d545b2768c6beac8acd14eb9d6b13f1 This fixes the issue with the missing "Database" purpose key when we converted the oplog keys into their own tables. We chosen to truncate the existing oplog entries to eliminate any references to a Database kms key in existing oplog entries.	3 years ago
Jim	4ed675be16	feat (db): add Writer.IsTx(...) (#3770 )	3 years ago
Jim	0d19892e03	fix (events): convert multierror to stdlib error (#3743 ) * fix (events): convert multierror to stdlib error multierror errors will be converted to a stdlib error (because multierror doesn't support json marshaling). * refact (errors): convert from multierror to stdlib errors.Join(...) * refact: convert from pkg/errors to stdlib error wrapping	3 years ago
Irena Rindos	9733a250fa	refact(event): move observability/event to event (#3704 )	3 years ago
Michael Gaffney	0af36d3d8a	refactor(sql): Consolidate changes to oplog_entry into one migration	3 years ago
Michael Gaffney	3befccffd7	refactor(sql): Reorder and rename oplog kms migrations	3 years ago
Michael Gaffney	d8393e5793	style(sql): fix indenting and remove trailing whitespace	3 years ago
Jim	a16033b73a	fix (kms): move oplog DEK into its own internal kms (#3665 ) * fix (auth/ldap): fix which DEK is used for oplog during authenticate * chore: update go-kms-wrapping dep for new extras/kms changes The latest version supports setting a table name prefix for a kms which is needed to fix our oplog DEK problem * chore: update extras/kms dep This latest version supports an optional TableNamePrefix which is required to fix our existing oplog DEK issue. * fix (oplog): schema oplog DEK fix These changes will allow use to store the oplog DEK in it's own set of tables, so they won't get deleted when a scope is deleted * fix (kms): move oplog DEK into its own internal kms * tests: update tests to be compatible with oplog fix * tests: remove migration tests which are incompatible with oplog fix These tests no longer work because they are missing the required schema changes for the oplog fix	3 years ago
Jim	dc89ca28a1	feat: add API support for additional LDAP auth method fields. (#3679 ) * feat (auth/ldap): add Repo support for max page size and deref aliases (#3670) * chore: update cap/ldap dependency This latest version of cap/ldap adds support for MaxPageSize and DerefAliases * feat (auth/ldap): add MaxPageSize and DerefAliases to storage proto * feat (api/authmethods): add new ldap fields. Add support for maximum_page_size and dereference_aliases * feat (auth/ldap): schema changes to support new LDAP attributes * feat (auth/ldap): add Repo support for max page size and deref aliases * feat (handlers/ldap): add support for new fields (#3674) Added API support for new ldap fields: MaximumPageSize and DereferenceAliases * chore: add CHANGELOG entry for new api auth ldap fields	3 years ago
Jim	d5411a26f6	chore: update protobuf dep (#3669 )	3 years ago
hashicorp-copywrite[bot]	29da0bcb92	[COMPLIANCE] License changes (#3567 ) * Adding explicit MPL license for sub-package. This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository. * Adding explicit MPL license for sub-package This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository. * Updating the license from MPL to Business Source License Going forward, this project will be licensed under the Business Source License v1.1. Please see our blog post for more details at https://hashi.co/bsl-blog, FAQ at www.hashicorp.com/licensing-faq, and details of the license at www.hashicorp.com/bsl. * Update copyright file headers to BUS-1.1 * Rerun make gen This will pick up the last of the license changes * Revert "Rerun make gen" This reverts commit `dbe0968bff`. * Remove copywrite from generation script The current version of copywrite is not sophisticated enough to only apply the copywrite changes to certain subdirectories, so to avoid accidentally relicensing our api and sdk modules, we will stop running the copywrite script until such time that it can do the right thing. * License protobuf API as MPL The protobuf API files are used to generate the SDK, since we want the SDK to be MPL, the protobuf files must also be MPL. This extends to all protobuf files whose sources end up in the SDK directory, including: - internal/proto/controller/api - internal/proto/controller/custom_options - internal/proto/plugin It also includes the generated service directory in internal/gen/controller/api/services, since they are generated from the protobuf sources used for the SDK. * Change website license file name This makes the naming scheme consistent with the rest of the repository. * Add note on licensing to README * Set copywrite license to BUSL * Revert "Add note on licensing to README" This reverts commit `4d865a91a7`. --------- Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com> Co-authored-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>	3 years ago
Damian Debkowski	e285a8ef6f	refactor: make secrets optional for storage bucket (#3353 )	3 years ago
Michael Milton	8527f0305d	Fix issue where snapshot conflicts weren't being handled (#3406 ) * fix issue where conflicts weren't being handled * Instead of using on collision, alter the daily_range * move tests to appropriate file * Apply suggestions from code review Co-authored-by: Michael Gaffney <mike@gaffney.cc> --------- Co-authored-by: Michael Gaffney <mike@gaffney.cc>	3 years ago
Michael Milton	23c40a0bc3	Update census to run less frequently (#3346 ) * update census to run less frequently * Update internal/census/census_job.go Co-authored-by: Michael Gaffney <mike@gaffney.cc> * change nextrunin duration --------- Co-authored-by: Michael Gaffney <mike@gaffney.cc>	3 years ago
Jeff Mitchell	8322916fb4	Remove all deprecated Err/Wrap/E calls (#3286 )	3 years ago
Timothy Messier	603642f271	fix(session): List performance with large number of session connections (#3288 ) As part of the performance improvements for listing sessions for v0.9.0, the view used to retrieve sessions for listing was updated to no longer include a join on session_connection. However, a later migration that was adding additional columns to this view inadvertently re-added the join and session_connection columns. This resulted in requests to list sessions performing this join, and returning additional rows for each connection in a session. The application would ignore this additional data, since it is no longer returned via the API. This would be particularly impactful in cases where there are a large number of connections per session, as this would greatly increase the amount of data returned by the database. This fix removes the session_connection join from this view, which should restore the original performance improvements even in cases where there is a large number of session connections per session. See: https://github.com/hashicorp/boundary/pull/2160 Ref: `32070678dc` Fixes: `1e3c941be1` (cherry picked from commit `7aef341682`)	3 years ago
Timothy Messier	876f9bb07f	fix(session): List performance with large number of session connections As part of the performance improvements for listing sessions for v0.9.0, the view used to retrieve sessions for listing was updated to no longer include a join on session_connection. However, a later migration that was adding additional columns to this view inadvertently re-added the join and session_connection columns. This resulted in requests to list sessions performing this join, and returning additional rows for each connection in a session. The application would ignore this additional data, since it is no longer returned via the API. This would be particularly impactful in cases where there are a large number of connections per session, as this would greatly increase the amount of data returned by the database. This fix removes the session_connection join from this view, which should restore the original performance improvements even in cases where there is a large number of session connections per session. See: https://github.com/hashicorp/boundary/pull/2160 Ref: `32070678dc` Fixes: `1e3c941be1`	3 years ago
Michael Gaffney	99a8777866	fix(db): Add index to session connection table This index improves the performance for two queries on the `session_connection` table. The execution time for the first query went from 40.105 ms to 1.405 ms. The execution time for the second query went from 13.808 ms to 0.217 ms. Query 1 explain plan before index: Update on session_connection (cost=6465.17..6557.68 rows=11 width=50) (actual time=38.868..39.147 rows=3 loops=1) -> Nested Loop (cost=6465.17..6557.68 rows=11 width=50) (actual time=38.471..38.506 rows=3 loops=1) -> HashAggregate (cost=6464.75..6464.86 rows=11 width=40) (actual time=38.447..38.450 rows=3 loops=1) Group Key: (session_connection_1.public_id)::text Batches: 1 Memory Usage: 24kB -> Nested Loop Semi Join (cost=0.42..6464.72 rows=11 width=40) (actual time=29.184..38.434 rows=3 loops=1) -> Seq Scan on session_connection session_connection_1 (cost=0.00..6329.38 rows=16 width=20) (actual time=15.744..38.014 rows=23 loops=1) Filter: ((worker_id)::text = 'w__________k'::text) Rows Removed by Filter: 163607 -> Index Scan using session_connection_state_connection_id_end_time_key on session_connection_state (cost=0.42..8.45 rows=1 width=20) (actual time=0.016..0.016 rows=0 loops=23) Index Cond: (((connection_id)::text = (session_connection_1.public_id)::text) AND (end_time IS NULL)) Filter: ((state = ANY ('{authorized,connected}'::text[])) AND (start_time < (CURRENT_TIMESTAMP - '00:00:05'::interval))) Rows Removed by Filter: 1 -> Index Scan using session_connection_pkey on session_connection (cost=0.42..8.44 rows=1 width=20) (actual time=0.016..0.016 rows=1 loops=3) Index Cond: ((public_id)::text = (session_connection_1.public_id)::text) Planning Time: 0.793 ms Trigger immutable_columns: time=0.474 calls=3 Trigger update_connection_state_on_closed_reason: time=0.159 calls=3 Trigger update_time_column: time=0.034 calls=3 Trigger update_version_column: time=0.041 calls=3 Trigger wh_update_session_connection: time=0.688 calls=3 Execution Time: 40.105 ms Query 1 explain plan after index: Update on session_connection (cost=152.71..219.87 rows=8 width=50) (actual time=0.497..0.683 rows=3 loops=1) -> Nested Loop (cost=152.71..219.87 rows=8 width=50) (actual time=0.266..0.292 rows=3 loops=1) -> HashAggregate (cost=152.29..152.37 rows=8 width=40) (actual time=0.256..0.258 rows=3 loops=1) Group Key: (session_connection_1.public_id)::text Batches: 1 Memory Usage: 24kB -> Nested Loop Semi Join (cost=4.94..152.27 rows=8 width=40) (actual time=0.199..0.251 rows=3 loops=1) -> Bitmap Heap Scan on session_connection session_connection_1 (cost=4.51..50.76 rows=12 width=20) (actual time=0.017..0.035 rows=10 loops=1) Recheck Cond: ((worker_id)::text = 'w__________k'::text) Heap Blocks: exact=9 -> Bitmap Index Scan on session_connection_worker_id_closed_reason_ix (cost=0.00..4.51 rows=12 width=0) (actual time=0.011..0.011 rows=10 loops=1) Index Cond: ((worker_id)::text = 'w__________k'::text) -> Index Scan using session_connection_state_connection_id_end_time_key on session_connection_state (cost=0.42..8.45 rows=1 width=20) (actual time=0.021..0.021 rows=0 loops=10) Index Cond: (((connection_id)::text = (session_connection_1.public_id)::text) AND (end_time IS NULL)) Filter: ((state = ANY ('{authorized,connected}'::text[])) AND (start_time < (CURRENT_TIMESTAMP - '00:00:05'::interval))) Rows Removed by Filter: 1 -> Index Scan using session_connection_pkey on session_connection (cost=0.42..8.44 rows=1 width=20) (actual time=0.010..0.010 rows=1 loops=3) Index Cond: ((public_id)::text = (session_connection_1.public_id)::text) Planning Time: 0.614 ms Trigger immutable_columns: time=0.279 calls=3 Trigger update_connection_state_on_closed_reason: time=0.185 calls=3 Trigger update_time_column: time=0.027 calls=3 Trigger update_version_column: time=0.032 calls=3 Trigger wh_update_session_connection: time=0.450 calls=3 Execution Time: 1.405 ms Query 2 explain plan before index: Limit (cost=1002.40..6147.94 rows=1 width=253) (actual time=12.244..13.726 rows=1 loops=1) -> Nested Loop Left Join (cost=1002.40..6147.93 rows=1 width=253) (actual time=12.243..13.724 rows=1 loops=1) Join Filter: ((w.public_id)::text = (session_connection.worker_id)::text) CTE worker_config_tags -> GroupAggregate (cost=1.71..2.26 rows=22 width=59) (actual time=0.044..0.082 rows=22 loops=1) Group Key: ct_1.worker_id, ct_1.source -> Sort (cost=1.71..1.77 rows=22 width=49) (actual time=0.031..0.034 rows=22 loops=1) Sort Key: ct_1.worker_id, ct_1.source Sort Method: quicksort Memory: 27kB -> Seq Scan on server_worker_tag ct_1 (cost=0.00..1.22 rows=22 width=49) (actual time=0.007..0.010 rows=22 loops=1) -> Nested Loop Left Join (cost=0.14..9.28 rows=1 width=245) (actual time=0.106..0.108 rows=1 loops=1) Join Filter: ((w.public_id)::text = (ct.worker_id)::text) -> Nested Loop Left Join (cost=0.14..8.72 rows=1 width=213) (actual time=0.103..0.104 rows=1 loops=1) Join Filter: ((w.public_id)::text = (wt.worker_id)::text) -> Index Scan using server_worker_pkey on server_worker w (cost=0.14..8.15 rows=1 width=181) (actual time=0.012..0.012 rows=1 loops=1) Index Cond: ((public_id)::text = 'w__________k'::text) -> CTE Scan on worker_config_tags wt (cost=0.00..0.55 rows=1 width=64) (actual time=0.090..0.090 rows=0 loops=1) Filter: (((worker_id)::text = 'w__________k'::text) AND (source = 'api'::text)) Rows Removed by Filter: 22 -> CTE Scan on worker_config_tags ct (cost=0.00..0.55 rows=1 width=64) (actual time=0.001..0.002 rows=1 loops=1) Filter: (((worker_id)::text = 'w__________k'::text) AND (source = 'configuration'::text)) Rows Removed by Filter: 8 -> GroupAggregate (cost=1000.00..6136.36 rows=1 width=21) (actual time=12.133..13.610 rows=0 loops=1) Group Key: session_connection.worker_id -> Gather (cost=1000.00..6136.35 rows=1 width=13) (actual time=12.132..13.608 rows=0 loops=1) Workers Planned: 2 Workers Launched: 2 -> Parallel Seq Scan on session_connection (cost=0.00..5136.25 rows=1 width=13) (actual time=9.012..9.012 rows=0 loops=3) Filter: ((closed_reason IS NULL) AND ((worker_id)::text = 'w__________k'::text)) Rows Removed by Filter: 54544 Planning Time: 0.398 ms Execution Time: 13.808 ms Query 2 explain plan after index: Limit (cost=2.82..20.03 rows=1 width=253) (actual time=0.132..0.134 rows=1 loops=1) -> Nested Loop Left Join (cost=2.82..20.02 rows=1 width=253) (actual time=0.131..0.133 rows=1 loops=1) Join Filter: ((w.public_id)::text = (session_connection.worker_id)::text) CTE worker_config_tags -> GroupAggregate (cost=1.71..2.26 rows=22 width=59) (actual time=0.046..0.086 rows=22 loops=1) Group Key: ct_1.worker_id, ct_1.source -> Sort (cost=1.71..1.77 rows=22 width=49) (actual time=0.033..0.035 rows=22 loops=1) Sort Key: ct_1.worker_id, ct_1.source Sort Method: quicksort Memory: 27kB -> Seq Scan on server_worker_tag ct_1 (cost=0.00..1.22 rows=22 width=49) (actual time=0.007..0.011 rows=22 loops=1) -> Nested Loop Left Join (cost=0.14..9.28 rows=1 width=245) (actual time=0.120..0.120 rows=1 loops=1) Join Filter: ((w.public_id)::text = (ct.worker_id)::text) -> Nested Loop Left Join (cost=0.14..8.72 rows=1 width=213) (actual time=0.117..0.118 rows=1 loops=1) Join Filter: ((w.public_id)::text = (wt.worker_id)::text) -> Index Scan using server_worker_pkey on server_worker w (cost=0.14..8.15 rows=1 width=181) (actual time=0.011..0.012 rows=1 loops=1) Index Cond: ((public_id)::text = 'w__________k'::text) -> CTE Scan on worker_config_tags wt (cost=0.00..0.55 rows=1 width=64) (actual time=0.104..0.105 rows=0 loops=1) Filter: (((worker_id)::text = 'w__________k'::text) AND (source = 'api'::text)) Rows Removed by Filter: 22 -> CTE Scan on worker_config_tags ct (cost=0.00..0.55 rows=1 width=64) (actual time=0.002..0.002 rows=1 loops=1) Filter: (((worker_id)::text = 'w__________k'::text) AND (source = 'configuration'::text)) Rows Removed by Filter: 8 -> GroupAggregate (cost=0.42..8.45 rows=1 width=21) (actual time=0.010..0.010 rows=0 loops=1) Group Key: session_connection.worker_id -> Index Only Scan using session_connection_worker_id_closed_reason_ix on session_connection (cost=0.42..8.44 rows=1 width=13) (actual time=0.010..0.010 rows=0 loops=1) Index Cond: ((worker_id = 'w__________k'::text) AND (closed_reason IS NULL)) Heap Fetches: 0 Planning Time: 0.544 ms Execution Time: 0.217 ms Closes #3281	3 years ago
Johan Brandhorst-Satzkorn	610df9892b	all: propagate more history metadata (#3279 ) A few target and host properties were missed when reading historical data from the history tables.	3 years ago
Michael Gaffney	8159ddb08e	feat(db): Add function to update billing snapshot table This adds the `update_sessions_pending_daily_snapshot` SQL function which updates the `sessions_pending_daily_snapshot` table by inserting values for any days missing from the table. This also adds a check constraint to the `sessions_pending_daily_snapshot` table along with a `comment on table`. This also removes the hcp daily billing views since they are no longer needed. Ref: `ace2def49d`	3 years ago
Michael Gaffney	41bdfb4406	fix(db): Fix time zone consistency issues in monthly billing views The hcp monthly billing views build a range of buckets using the `generate_series` and the `now()` functions along with relative offsets like `+ 1 month`. The calculated start and end times for these views can vary depending on the time zone setting of the PostgreSQL server. This adds 3 table functions that call their respective views with the time zone set to UTC. Calling these functions instead of their underlying views directly prevents the returned values from varying when the server's time zone is not set to UTC. This also drops support for PostgreSQL 11. The included tests use the `date_trunc(field, source [, time_zone ])` function added in PostgreSQL 12. Fixes: `41b1b5b4b7`	3 years ago
Michael Gaffney	47ddd2e40a	fix(db): Fix generated series to yield consistent results The hcp billing `_all` views built a range of buckets using the `generate_series` function using the minimum value of the `session_pending_time` column from the `wh_session_accumulating_fact` table as the start time and `now()` as the end time. It then truncated values in the series using the `date_trunc` function. This allowed the generated series of buckets returned to vary and could result in one additional bucket being included in the series depending on the current time and the start time. As an illustrative example, the `hourly_all` query used: select date_trunc('hour',time) from generate_series( (select min(session_pending_time) -- start time from wh_session_accumulating_fact), now(), '1 hour'::interval ) as time; Substituting the start time with a hard coded value to show the issue: -- Show current time, where the minutes portion is 55 select now(); now ------------------------------- 2023-05-22 14:55:40.567605+00 (1 row) -- generate series with start time's minutes portion at 57 select date_trunc('hour',time) from generate_series( '2023-05-22 01:57:00'::timestamptz, now(), '1 hour'::interval ) as time; date_trunc ------------------------ 2023-05-22 01:00:00+00 2023-05-22 02:00:00+00 2023-05-22 03:00:00+00 2023-05-22 04:00:00+00 2023-05-22 05:00:00+00 2023-05-22 06:00:00+00 2023-05-22 07:00:00+00 2023-05-22 08:00:00+00 2023-05-22 09:00:00+00 2023-05-22 10:00:00+00 2023-05-22 11:00:00+00 2023-05-22 12:00:00+00 2023-05-22 13:00:00+00 (13 rows) -- Wait a few minutes select now(); now ------------------------------ 2023-05-22 14:58:48.75287+00 (1 row) -- Now run again and get an extra result, even though still -- within the same hour select date_trunc('hour',time) from generate_series( '2023-05-22 01:57:00'::timestamptz, now(), '1 hour'::interval ) as time; date_trunc ------------------------ 2023-05-22 01:00:00+00 2023-05-22 02:00:00+00 2023-05-22 03:00:00+00 2023-05-22 04:00:00+00 2023-05-22 05:00:00+00 2023-05-22 06:00:00+00 2023-05-22 07:00:00+00 2023-05-22 08:00:00+00 2023-05-22 09:00:00+00 2023-05-22 10:00:00+00 2023-05-22 11:00:00+00 2023-05-22 12:00:00+00 2023-05-22 13:00:00+00 2023-05-22 14:00:00+00 (14 rows) To fix this, we call the `generate_series` function truncating the start time argument to the bucket size: -- Hourly select bucket from generate_series( date_trunc('hour', select min(session_pending_time) from wh_session_accumulating_fact)), now(), '1 hour'::interval ) as bucket; -- Monthly select bucket from generate_series( date_trunc('month', select min(session_pending_time) from wh_session_accumulating_fact)), now(), '1 month'::interval ) as bucket; Fixes: `41b1b5b4b7`	3 years ago
Michael Gaffney	053e74f66c	feat(db): Add table for census upload job status	3 years ago
Michael Gaffney	8bf80e0544	test(db): Conditionally skip monthly billing view tests Certain tests fail when run on the first day of the month because of edge cases in the setup of test data. This adds conditional logic to skip those tests when run on the first day of a month.	3 years ago
Jim	d4e537c88d	chore: updgrade pgx direct deps to v5 (#3258 )	3 years ago
Jim	536551dc0c	fix: update go-dbw dep and refactor conversion of pgx errors (#3242 )	3 years ago
Timothy Messier	61a37ef4a4	fix(schema): Add missing endpoint column to session recording view	3 years ago
Timothy Messier	f53d65eacd	refact(sql): Rename migrations to avoid number conflicts	3 years ago
Todd	4063880bbe	Add credential history views	3 years ago
Michael Gaffney	a783c91b2d	feat(sql): Add dynamic credential hst tables to session recording This adds the `recording_dynamic_credential` join table and a trigger that runs as an after insert on the `recording_session` table. The join table joins the `recording_session`, `credential_vault_store_hst`, `credential_library_history_base`, and the `credential_purpose_enm` tables.	3 years ago
Michael Gaffney	3769f0851c	test(sql): Refactor the colors persona This changes multiple IDs in the colors persona to be 12 characters. It also renames some IDs in an attempt to make them more consistent. This also changes the order of the columns in some insert statements to follow the pattern of the other insert statements. This also adds documentation to the top of colors persona file in an attempt to explain the general patterns being followed in the file.	3 years ago
Michael Gaffney	03b1291b34	feat(sql): Add history tables for dynamic credentials This adds history tables for `credential_vault_store`, `credential_vault_library`, and `credential_vault_ssh_cert_library`. This also adds a base table for the credential library history tables.	3 years ago
Todd	e1540be9f3	Add host information to session recording view	3 years ago
Michael Gaffney	a8224f4f57	feat(sql): Add endpoint from session table to session recording This adds an `endpoint` column to the `recording_session` table and updates the session recording insert trigger to copy the value of `endpoint` from the `session` table to the `recording_session` table.	3 years ago
Michael Gaffney	06e4e46248	feat(sql): Add static credential hst tables to session recording This adds the `recording_static_credential` join table and a trigger that runs as an after insert on the `recording_session` table. The join table joins the `recording_session`, `credential_static_store_hst`, `credential_static_history_base`, and the `credential_purpose_enm` tables.	3 years ago
Michael Gaffney	3a64fb3d58	feat(sql): Add history tables for static credentials This adds history tables for `credential_static_store`, `credential_static_json_credential`, `credential_static_username_password_credential`, and `credential_static_ssh_private_key_credential`. This also adds a base table for the static credential history tables.	3 years ago
Todd	386eac072c	Add recording_channel_ssh_aggregate view	3 years ago
Johan Brandhorst-Satzkorn	11893a8885	db: add state and error_details to recording	3 years ago

1 2 3 4 5 ...

606 Commits (deeac5c457e86c6c2bf6db50f00a385ecbf4a360)