821 Commits (5fe40bc7bd200788e34d8bcf4ce1dfd1da5bbad2)

Author	SHA1	Message	Date
Jim	5fe40bc7bd	style (events): add changes from `make fmt` in prep for release.	5 years ago
Louis Ruch	4cdce29a84	fix(connect): Remove erroneous event error message (#1500) ... When a session is terminated by the user, the cancel session path is triggered, in this path the websocket connection is closed. If the connection is already closed, an EOF is returned. In this situation an EOF is expected and should not be logged to the user. Example log line: { "id": "ezqk6w9dOQ", "source": "https://hashicorp.com/boundary/dev-controller/boundary-dev", "specversion": "1.0", "type": "error", "data": { "error": "failed to close WebSocket: failed to read frame header: EOF", "error_fields": {}, "id": "e_n60YPsDkaj", "version": "v0.1", "op": "worker.(Worker).handleProxy", "info": { "msg": "error closing client connection" } }, "datacontentype": "text/plain", "time": "2021-09-02T10:05:46.587513-07:00" }	5 years ago
Jeff Mitchell	2c61bd4963	Fix recursive listing when the ingress scope lacks permissions (#1478) ... The existing logic was to ignore Forbidden errors when recursively listing in favor of looking for downstream scopes with appropriate permissions. The problem is that when there is actually no authentication token at all, what comes back is an Unauthenticated error. Therefore if you didn't have permissions on the ingress scope (e.g. the default of `global`) but did downstream for the anonymous user, you'd get back an error. I don't think there's any security issue with doing this; the user could always have simply started the search at a downstream scope to begin with so nothing more is being revealed, this just fixes behavior to match expectations in this case. A test was added for one of the services; the code is exactly the same across them. The reason that the existing tests that were added with the output fields testing didn't catch this is that even though they were specifying `u_anon` as the user, the disabled test context skipped the logic that would return a 401 in this case. It turns out that fixing this is very complicated because we don't have a great way in most tests of defining grants to put onto the generated scopes, so even if a true anonymous context was added each test would require the anonymous user to have list permissions on whatever resource was under test, requiring a lot of role and grant and principal finagling. This is probably a good longer-term TODO but not one that seemed worth it for this fix. Fixes ICU-2301	5 years ago
Jeff Mitchell	781871c30d	Properly accept grant format type=<type>;output_fields=<of> (#1474) ... Although we document that this is allowed, and the ACL code handles it, some of the validation code did not. This would cause a semi intermittent problem because that validation code is used only in some code paths; specifically we skip it (for now) to avoid erroring out on some deprecated grant string formats when reading grants from the DB, but we use it to error out when validating incoming grants.	5 years ago
Louis Ruch	030c09c6b9	bug(targets): Fix panic in authorize-session (#1496) ... When renaming credential libraries -> credential sources the target service added the credential library information to the SessionCredential response, while the target cmd was parsing the credential source. This prevents a panic like the following: boundary targets authorize-session -id ttcp_5PS2dktESb panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x30 pc=0x1f99a31] goroutine 1 [running]: github.com/hashicorp/boundary/internal/cmd/commands/targetscmd.printCustomActionOutputImpl(0xc00046c0a0) /Users/louisruch/boundary/internal/cmd/commands/targetscmd/funcs.go:822 +0x9d1 github.com/hashicorp/boundary/internal/cmd/commands/targetscmd.(*Command).Run(0xc00046c0a0, {0xc000134030, 0x2, 0x2}) /Users/louisruch/boundary/internal/cmd/commands/targetscmd/targets.gen.go:324 +0x188b github.com/mitchellh/cli.(*CLI).Run(0xc0001c6500) /Users/louisruch/go/pkg/mod/github.com/mitchellh/cli@v1.1.2/cli.go:262 +0x5f8 github.com/hashicorp/boundary/internal/cmd.RunCustom({0xc000134010, 0x60, 0x0}, 0x0) /Users/louisruch/boundary/internal/cmd/main.go:186 +0x9d6 github.com/hashicorp/boundary/internal/cmd.Run(...) /Users/louisruch/boundary/internal/cmd/main.go:92 main.main() /Users/louisruch/boundary/cmd/boundary/main.go:13 +0xc9 Fixes #1488	5 years ago
Louis Ruch	db8e9bb0c6	refactor(worker): refactors the proxy handler to use endpoint scheme (#1491)	5 years ago
Jim	5f46207e76	Events: add redact/encrypt node for audit events to Eventer (#1490) ... * feat (events): Add new options needed for AuditConfig WithAuditWrapper and WithFilterOperations all for optional paramters when creating AuditConfig types. * refactor (event): Change EventerConfig.Sinks to a ptr We need to modify the SinkConfig.AuditConfig during the NewEventer factory, so we have to use a ptr not a ref. * feat (events): Add audit configuration types * feat (events): add audit config to sink configuration. * feat (events): add redact/encrypt node for audit events to Eventer Integrate the encrypt.Filter node from the eventlogger into the event.Eventer, so audit events can be redacted/encrypted based on data classifications. * test (events) Add test protobufs for events Add test protobufs along with a test service for testing the audit event encrypt node	5 years ago
Timothy Messier	90e2c871f0	refactor(wh): Remove host prefix from wh_user_dimension org fields ... These fields are being refactored so they conform with the new wh_credential_dimension. This will inform Business Intelligence tools that these attributes conform across dimensions.	5 years ago
Timothy Messier	026f09a9a8	refactor(wh): Rename primary and foreign key columns ... The tables in the operational part of the Boundary database (meaning non-warehouse tables) suffix primary key columns and foreign key columns with `_id`. Many of these columns appear in the warehouse dimension and fact tables as normal attributes, not primary keys or foreign keys. To disambiguate ID columns from operational tables, the primary key column for dimension tables will be renamed from `id` to `key` and the foreign key columns in the fact tables will be renamed to change the suffix from `_id` to `_key`.	5 years ago
Timothy Messier	c8bf1917f5	db: Refactor test setup for pre-populated data ... Now when initializing the test database for sql tests, it will load the colors persona data into the database. This will provide a data set that tests can use without needing to call `wtt_load`. The main motivation is to allow for a separation of setup and test execution that requires crossing a transaction boundary. The pgTap tests are required to exectue as a single transaction, so that the rollback can reset the state, and each test can be isolated. However, can cause issues with some test senarios, particuarly with the use of some times releated functions like `current_timestamp` and `now` which return the time that the current transaction started. This creates an issue if a test relies on some setup that sets a time via `now`, then an update that expects a new time via `now`. See: https://www.postgresql.org/docs/13/functions-datetime.html#FUNCTIONS-DATETIME-CURRENT	5 years ago
Jeff Mitchell	2649d1b966	Move protooptions to sdk/pbs (#1486) ... This prevents a reverse dependency from sdk on the main module.	5 years ago
Jeff Mitchell	b13975cb6a	Move API resource pbs to new location (#1484) ... This moves the generated pb files to the `sdk` module. This allows third parties to make use of them if desired (e.g. in upcoming plugins) and moves away from the `gen` nomenclature in the directory, since we are going to be adding some extra interface functions to the messages.	5 years ago
Michael Gaffney	5f88243ddc	Fix db migration that adds support for OIDC in the data warehouse (#1471) ... Four new columns were added to the `wh_user_dimension` table to support OIDC. The data type for the columns was originally `wh_dim_text` which is a domain type with a `not null` restriction. Adding the new columns caused the migration to fail if the `wh_user_dimension` table had any rows in it because the existing rows would violate the `not null` restriction. Closes #1469	5 years ago
Louis Ruch	31af49e76d	refactor(worker): Move tcp_proxy into its own package (#1458) ... * refactor(worker): Move tcp_proxy into its own package This refactors: - the tcp_proxy.go into worker/proxy/tcp - add a missing unit test for the HandleTcpProxyV1 - refactors session.go into worker/session, needed to avoid import cycle - adds a baseline for a mock controller session client - go lint fixes * Add sleep between spinning listener goroutine and dialing Co-authored-by: Jeff Mitchell <jeffrey.mitchell@gmail.com>	5 years ago
Jim	4d70c049c2	fix: Stop trying to re-write the resp header on event flush errors (#1463)	5 years ago
Timothy Messier	0f19009cea	feat(wh): Add oidc related columns to wh_user_dimension (#1455) ... This adds new columns to the wh_user_dimension to provide additional data from OIDC that does not apply to Password Auth. Note: Given the specifics of the data, it is possible these columns may be used by other auth methods in the future, hence the lack of `oidc` in the column names. The specific columns are: - auth_method_external_id For OIDC this is the Issuer - auth_account_external_id For OIDC this is the Subject - auth_account_full_name For OIDC this is the account's Full Name, and can be `None`. - auth_account_email For OIDC this is the account's Email, and can be `None`. For all of these new columns, they will be `Not Applicable` if the auth method is Password.	5 years ago
Jeff Mitchell	f7894811f5	Update go-bexpr to allow colons in paths; add a test (#1453)	5 years ago
Jim	3bd6a209a4	prevent panic when hclog_formatter_node is operating on a struct (#1456)	5 years ago
Timothy Messier	624bdbf0c2	test(db): Add baseline tests for wh_user_dimension (#1454) ... This sets up the configuration for running a new suite of tests that are written in sql, to test the sql logic implemented in plpgsql functions and triggers. It is primarily designed to provides tests for the data warehouse functionality that is implemented purely in sql. The infrastructure to run these tests leverages docker. It creates one docker container using the `library/postgres` image and provisions the database with the boundary sql migrations and test helper functions. A separate docker container is run that has the pgtap extension and pg_prove cli to execute the tests. As noted, pgtap is used for the tests. This postgresql extension provides a number of helpful assertion functions to make writing sql tests easier. It also, along with the pg_prove cli, provides readable test output. Along with the new testing infrastructure this commit establishes a baseline of tests for the wh_user_dimension: - Unit tests for individual views and functions: - whx_user_dimension_source - whx_user_dimension_target - wh_upsert_user - Integration tests for that verify that the wh_user_dimension gets populated correctly when sessions are inserted or updated. This ensures that: - the trigger runs on session insert - the trigger results in the correct changes to the wh_user_dimension. This commit provides a `Makefile` to run the tests locally that provides support for: - Setting the postgres docker image to easily test multiple versions. - Passing through options to pg_prove - Passing the test list to run individual tests - Additional targets to separately startup the database image to quickly re-run tests while developing new tests. Lastly configures circleci to run this test suite as a matrix build in parallel with the other tests. The matrix build tests multiple versions of postgres: - latest - 13-alpine - 12-alpine - 11-alpine See: https://pgtap.org/	5 years ago
Jim	477b541bf0	Events: ongoing clean up of eventing (#1448)	5 years ago
Jim	00cb18efbe	use parseutil.ParseDurationSecond(...) which allows a value without a suffix to be treated as seconds instead of nanoseconds (#1447)	5 years ago
Jim	acc3dda2df	parse the duration string specified in an event file config into a time.Duration (#1446)	5 years ago
Todd Knight	8496093874	make fmt.	5 years ago
Jeff Mitchell	c75c97a872	Normalize event configuration (#1444) ... * Normalize event configuration This updates event configuration stanzas to more closely match behavior for listeners and kmses: * Singular "sink" instead of "sinks" (probably not the most correct option, but a consistent option, and one that makes sense for HCL) * Allows sinks to have the type defined at the start of the block * Moves type-specific configuration to a specific named block Co-authored-by: Jim <jlambert@hashicorp.com>	5 years ago
s-christoff	00ada3c578	update op (#1442)	5 years ago
Jim	3a3f956615	support new event formats of: hclog-text and hclog-json (#1440)	5 years ago
Jeff Mitchell	e3012afb4f	Add primary auth method ID to default output fields, fix display bug (#1441)	5 years ago
Todd Knight	39cc245966	Make gen after make tools. Includes make fmt of existing non generated files. (#1438)	5 years ago
Todd Knight	03b4c095c4	Fix syntax error in profiling_on.go (#1437)	5 years ago
Chris Marchesi	947dc96b30	server: Add back controller shutdown on worker startup failure (#1432) ... * server: Add back controller shutdown on worker startup failure This adds back the controller shutdown call on worker failure. This was added in a15961e4 to address a panic discovered in testing, and was mistakenly thought to be unnecessary. This, however, covers multi-role scenarios where we have already started the controller, which needs to be shut down gracefully if possible if we can't start the worker. A nil check has been added to ensure the panic does not resurface. Co-authored-by: Jeff Mitchell <jeffrey.mitchell@gmail.com>	5 years ago
Jim	137729b0c0	Event log adpaters (#1434)	5 years ago
Jim	68dd2a66ba	Remove unneeded hclog dependencies as we move forward with events (#1433)	5 years ago
Jeff Mitchell	f8a51b987c	Migrate target host sets -> host sources (#1424) ... This follows the same model as https://github.com/hashicorp/boundary/pull/1413 See there and CHANGELOG for more details.	5 years ago
s-christoff	6b78108ecf	Update primary error functions to take a context, deprecate old functions (#1358) ... * add new error funcs which take a ctx and deprecate existing funcs * use context where available when writing error events * remove circular internal/errors pkg dep * suppress some chatty errors using errors.WithoutEvent() * convert auth oidc funcs to take context where needed to write events Co-authored-by: Jim <jlambert@hashicorp.com>	5 years ago
Susmitha Girumala	868a165a86	Bump admin UI version for credential brokering feature (#1429)	5 years ago
Louis Ruch	76a7ce93e8	Fix race with base context cancel and final run update in scheduler (#1422) ... * Fix race with base contedt cancel and final run update * Use scheduler waitgroup to wait for all running jobs to exit:	5 years ago
Chris Marchesi	a15961e461	Close connections on worker on shutdown, add force shutdown (#1354) ... This adds functionality to ensure that connections are closed on a worker when the server is shut down or reloaded. Connections are closed, and then the shutdown process waits for the next report to the controller to go through, waiting the grace period that was added in #1330. The rest of shutdown is then proceeded with. Listener shutdown has been moved to the start of the process to ensure that new connections can't be made while we are shutting down. Additionally, functionality has been added to force shut down servers after a second interrupt has been received after the first one. This takes the form of code in both the server and dev mode code paths that just performs an os.Exit if another shutdown is received during the normal shutdown process. Additionally, the commit also includes some refactoring to tests so that we could use some of the new E2E testing code to test session connections during shutdown and reload. Additionally, this fixes a panic in worker startup when there are no controllers defined. Looks like the shutdown call was attempting to call into the controller, not the worker. Finally, this also removes the extra shutdown within the outer worker startup. Shutdown is actually handled within the inner StartWorker(), attempting shutdown on errors was actually leading to potential use of an uninitialized worker, which is leading to panics in testing.	5 years ago
Jim	be8c3cb1ea	prevent panics when waitgroup == nil (#1423)	5 years ago
Jim	70ad081e39	Events: convert from hclog to events (#1409)	5 years ago
Todd Knight	10e8aeae4b	Subtype Id registration moved to its own package (#1416) ... * Allow repo packages to register subtypes and id prefixes. * Refactor out request authentication logic to a new package as a sibling to handler.	5 years ago
Jeff Mitchell	ab6f3eaeb4	Migrate credential-library nomenclature around targets to credential-source (#1413) ... See the CHANGELOG entry for more information. Co-authored-by: Jim <jlambert@hashicorp.com> Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com>	5 years ago
Jeff Mitchell	528d54b4ad	Add token/userinfo claims to account read output (#1419) ... This is very useful when trying to write managed groups filters as you can directly look at the structure of the claims for the accounts you're trying to filter. The output of the claims in text format on the CLI currently leaves something to be desired. I started revamping some of the formatting functions and it quickly got very fragile. Fundamentally at this point we're likely better off -- now that we have a good idea what our output format/standards are like -- rewriting the formatting code. I'll take that as a tasker as I have a lot of ideas...	5 years ago
Michael Gaffney	d4fc2efa53	Move base interfaces of the domain into a boundary package (#1415)	5 years ago
Jim	11124794bf	Events: add node for formatting/filtering cloudevents (#1390)	5 years ago
Chris Marchesi	1ebb8efab6	Apply extra review feedback from #1340 (#1400) ... * internal/servers/controller: refactor WaitForNextWorkerStatusUpdate * internal/servers/worker: use connId instead of conn.GetConnectionId() * internal/servers/worker: remove Worker.logClose * internal/session: complement versus inclusive state search * internal/session: use ScanRows instead of Scan * internal/session: make closeConnectionsForDeadServersCte results gormable * internal/tests/cluster: rename TestWorkerSessionCleanup to TestSessionCleanup * Simplify SQL query for CloseConnectionsForDeadWorkers (#1410) Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com>	5 years ago
Jim	cab3dd6986	update eventlogger dep which should eliminate a possible panic with the gated.Broker is nil (#1411)	5 years ago
Chris Marchesi	aae0c29f88	internal/servers/worker: fix TestWorkerWaitForNextSuccessfulStatusUpdate (#1402) ... This fixes this test which was failing intermittently. * Ups the grace period timeout to 2s. * Re-positions the testify require.New() so that incorrect code path warnings are not given.	5 years ago
Louis Ruch	2dae498123	Ensure job progress is always reported (#1396) ... * Update completed and total when job finishes * Remove unused option in test * Add comment * Use writer for updates and inserts	5 years ago
Louis Ruch	74aba58c73	Remove Telemetry from config tests (#1401)	5 years ago
Jeff Mitchell	ed6c59a053	Update the rest of the shared libs locations	5 years ago