53 Commits (02ff2db66e015aed51a2b9d1735dcc8036d9bfba)

Author	SHA1	Message	Date
Chris Marchesi	a15961e461	Close connections on worker on shutdown, add force shutdown (#1354) ... This adds functionality to ensure that connections are closed on a worker when the server is shut down or reloaded. Connections are closed, and then the shutdown process waits for the next report to the controller to go through, waiting the grace period that was added in #1330. The rest of shutdown is then proceeded with. Listener shutdown has been moved to the start of the process to ensure that new connections can't be made while we are shutting down. Additionally, functionality has been added to force shut down servers after a second interrupt has been received after the first one. This takes the form of code in both the server and dev mode code paths that just performs an os.Exit if another shutdown is received during the normal shutdown process. Additionally, the commit also includes some refactoring to tests so that we could use some of the new E2E testing code to test session connections during shutdown and reload. Additionally, this fixes a panic in worker startup when there are no controllers defined. Looks like the shutdown call was attempting to call into the controller, not the worker. Finally, this also removes the extra shutdown within the outer worker startup. Shutdown is actually handled within the inner StartWorker(), attempting shutdown on errors was actually leading to potential use of an uninitialized worker, which is leading to panics in testing.	5 years ago
Jim	70ad081e39	Events: convert from hclog to events (#1409)	5 years ago
Chris Marchesi	1ebb8efab6	Apply extra review feedback from #1340 (#1400) ... * internal/servers/controller: refactor WaitForNextWorkerStatusUpdate * internal/servers/worker: use connId instead of conn.GetConnectionId() * internal/servers/worker: remove Worker.logClose * internal/session: complement versus inclusive state search * internal/session: use ScanRows instead of Scan * internal/session: make closeConnectionsForDeadServersCte results gormable * internal/tests/cluster: rename TestWorkerSessionCleanup to TestSessionCleanup * Simplify SQL query for CloseConnectionsForDeadWorkers (#1410) Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com>	5 years ago
Jeff Mitchell	140c54e319	Repoint base62 import	5 years ago
Chris Marchesi	5a70875726	internal/servers/controller: Worker failure connection cleanup (#1340) ... This commit adds the support to do the following: * Mark connections for non-reporting workers as closed. This is the controller counterpart to the worker functionality (see #1330). This is written as a scheduled job that does the work DB-side in a single atomic query. * Works to reconcile states if such a broken controller-worker connection resumes and a worker reports a connection as connected that should be disconnected. In this case, the controller will send an update request, and the worker will honor it and terminate the connection. * Further refinement of the grace period setting has been added here. We have converged on the current server "liveness" setting as our default here, which is half of the previous 30s (15 seconds, in other words). Additionally, this is now configurable on the controller and worker side, with the caveat that it's currently impossible to do so in config as the setting has been untagged in HCL. This is exposed so that we can run some sophisticated testing scenarios where we skew the grace period to either the controller or worker to ensure the aforementioned reconciliation works. * Some repository functions have been added to support the new functionality, in addition to some test code to the worker to allow querying of session state while testing. * Finally, we've added some add timestamp subtraction functions as well, basically serving as the opposite of the addition functions.	5 years ago
Jeff Mitchell	2cbcf9a563	Update usage of shared-secure-libs (#1393)	5 years ago
Chris Marchesi	48e55f156a	worker: ensure connections are closed in local state on controller fail (#1369) ... This backports some test work that is being added in #1340 to assert that the worker is actually marking connections closed in its *local* state on a failure to contact the controller. This was missed in #1357 during some late-stage refactoring. Summary of the changes: * If we can't actually contact the controller during connection closure to mark the connections as closed, we actually construct a "fake" response that just simply includes a closed state for all connections requested. This is then used to update the local connection state. * Added a 3 second timeout to the CloseConnection request to the controller to ensure the call to close connections completed in a timely manner and did not hang indefinitely. Testing enhancements: * Added a new helper in TestHelper called LookupSession that allows the lookup of a session from the worker's internal session state. * This is used in the new ExpectConnectionStateOnWorker check in the E2E test, which is part of a larger amount of enhancements to the E2E test that are coming in #1340. * Added a ConnectionStatusFromProtoVal helper function to the session package to allow the conversion of protobuf connection status values to their higher-level counterparts; essentially it is a reverse of ConnectionStatus.ProtoVal. This is used in some of our tests (with more coming in #1340) to just help with keeping things more idiomatic. Finally, there is some minor error refactoring, and we also change the timeout on internal status and connection close requests to the controller to 5 seconds.	5 years ago
Chris Marchesi	9f9acce972	[Merge to main] internal/servers/worker: Controller failure connection cleanup (#1330) (#1357) ... This commit adds some rudimentary functionality for cleaning up connections on controller failure, or more specifically, a failure to communicate with the controllers. In this scenario, when a status cannot be sent for any reason for longer than 30 seconds, the worker will close all connections that are normally not cleaned up (so for sessions that currently are *not* in the progress of being canceled or terminated). Status requests now time out after 10 seconds if they are not made successfully. Session status is not touched during this process, but natural cleanup or connectionless sessions will still occur. This should be okay, as the session should be added on re-connect if need be. A few other changes are made to cancelConnections: first, error logging has been moved to the function so that we can handle errors requesting cancellation directly within the function; this entails simply just locally marking these connections as closed with a warning. Second, there was a minor logging bug where closed connections were not properly being noted in the last trace log, this has been fixed. Additional changes: * Circle: Set acceptance test no_output_timeout to 30m	5 years ago
Jeff Mitchell	86c9a90554	Update target credential library service, API, SDK, CLI (#1343)	5 years ago
Michael Gaffney	df35699c4e	Integrate with Vault to retrieve and manage per session credentials (#1308) ... * Fix down migrations * Udpates due to merge * Sentinel values only have to be greater than 0 * Create credential related sdk structs and methods. * Post merge interface updating. * Removing scratch code that was not cleaned up. * updates * Convert client certificate and client certificate keys into pem blocks and validate. * Add newline at end of file. * Faster check for "s" at the end of a resource name when generating from templates. * make fmt * updates * Add dynamic credentials to a session * Refactor: move contents of file * updates * Updates * Add InvalidDynamicCredential error * Assign credentials * Allow updating client certificates seperate from the client certificate key. * Replace application purpose string with enum value * PR feedback / fix go test . * Delete dead code * Use common view * Refactor: extract interface and rename method * Refactor: move code to eliminate privPurpLibrary * Add comment to requestMap * Refactor: do not export methods on an unexported struct * Update internal/db/schema/migrations/postgres/10/03_vault_credential.up.sql Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com> * Add comments to credential Purpose constants * Reorder switch statements to handle errors first * PR feedback * update * fix merge * Organizing the scheduler code and extract the individual store creation test helper function. * Run 'make fmt' * Create certificate when creating store. Allow vault address to be passed in as option * Refactor: rename database views and associated structs This change renames two database views and associated go structs to use a consistent naming pattern. Views ending in 'private' contain encrypted credentials needed for connecting to Vault. Views ending in 'public' mirror the 'private' views but do not include any encrypted values. This change does not include any changes to functionality or behavior. * Adding SAD target operations for credential libraries to the SDK. * Add status field and enumeration table for credential status * Refactor: rename Status to TokenStatus * Add status to vault credentials * Creating the template for the CLI and helper functions. * Add private database view for vault credentials * Add scope_id to view and cleanup TODO comment * Reformat embedded sql queries * Add vault client helpers * updates * white space * Add LookupLease test * CLI commands for Store work. Renamed fields in proto w/ vault prefix. * Test issue credentials with client TLS * Add additional checks to vault ping * Add an error for vault token capabilities and create a VaultToken error kind * Add support for working with Vault policies * Use Vault 1.7.2 for testing * Add testing helper for creating a vault client with a non-root token * Fix Vault renew lease test The Vault renew lease endpoint accepts an optional duration parameter for requesting the number of seconds to extend the lease by. However, Vault is not required to honor this request. * Add a var for the required capabilities of a boundary Vault token * Enhance the test helper for adding a policy to Vault * Allow the test role in postgresql to revoke credentials * Remove out of date comments * Enhance Vault test helpers for the database secrets engine * Reformat comments * Add a method for getting the capabilities of the current Vault token * Add revoke lease to internal Vault client * Reformat: combine err return line with check for nil line * Remove redundant checks in test code Remove checks in test code that are redundant with checks in the test harness. * Refactor CreateToken test helper to return the secret and the token * Fix build failures caused by signature change in vault.CreateToken * Fix Vault version and Vault API link in comments * ran `make gen` after merging from mgaffney-vault. * Remove Vault prefix from attribute fields in API, create a base CLI option for adding field prefixes. format now uses that prefix for error reporting. * Template now allows attaching attribute field prefixes. * Credential store uses PrefixAttributeFieldErrorsWithSubactionPrefix. * Fixing naming of vault flags and helper flags. * Updating more references to generated api methods. * Make the capabilities String method more readable * Add comment outlining PrintApiError usage of Options. * Refactor test to move declarations closer to first use * Fix comment * Implement revoke method on vault repository * Tests now verify certs, CA, and tokens, addresses can be updated in a store. * Added test for creating store with pk/cert in same field. * Allo updating certificates seperate from private key. Only contact backing vault service when updating token. * Fixing bad tests and removing unused code. * Adding tests for various ways of updating tokens. * Add private credential * Add token revocation job and cred renewal/revocation job * Regen after merge. * Fixing field mask related merge error. * Create Library CLI. Removes vault prefix from path. Adds a WithoutCleanup testOption for vault. * Set all credentials to revoked when a token is revoked * Fixing missed renaming update. * fix copy/paste error. * Fixing code generation errors after merge. * make gen after merge. * Update internal/cmd/commands/credentialstorescmd/vault_funcs.go Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com> * Update internal/cmd/commands/credentialstorescmd/vault_funcs.go Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com> * Update internal/cmd/commands/credentialstorescmd/vault_funcs.go Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com> * Refactor: move migrations from 10 to 12 * Removing logic from transaction. * wrapping client errors in the calling op for CredentialStore's client() method. * Redact token * redact client key * updates * Add trigger to revoke credentials when session is canceled or terminated * Fixing some help text for some flags. * Add ability to load values from files (file://) or environment variables (env://). * Updating comments * fix bad merge * feedback * Adding issued credentials to sessions. * Error on capabilities * updates * Expose NewVaulTestServer * Add godoc * Adding generated SessionCredential struct for API. * Progress on building handler test for Authorize Session. * feedback * Update policies * Add 'revoke' as a status for Vault Tokens * Add delete_time to vault credential store * Create dynamic credentials at session creation time. * Tests pass with credentials attached to AuthorizedSession result. * Only call issue if there are credentials to issue. * Fixing output only issues. Adding type to credential library. * Fix test to clear un/pw generated. confirm generated un/pw changes from 1 authorize session call to another. * Update lookupTokenError * Add delete_time to private credential store * Refactor: move private store to separate file * Fixing tests for AuthorizeSession and verifying error cases. * Only create the credential repo if there are credentials to issue. * Rename Library field to CredentialLibrary. * Fixing missing change of Library to CredentialLibrary. * Removing Default SDK functions for fields which are required and update CLI to match. * Replace resource specific id variables with "id" and the input parmater for paths to be just a single string and the resource name as it should show up in the path. * Soft delete vault credential stores * Prevent new SQL functions from silently overwriting existing functions * Rename "Hmac" to "HMAC" in the human readable cli response table. * Fix tests * Add vault database cleanup jobs * Add ability to set the token period in the vault test harness * updates * Add not_null_columns func test * remove unused struct * Up no output timeout * Validate current vault token on addr update * Fix client cert change test * add cert test * Update revocation job where * Fix comment * Use Vault's default port in CLI help strings * Remove down migrations * Increase migration file numbers by 1 * Run make migrations * Move migrations from 12 to 10 * Increase Circle CI timeout for acceptance tests * Fix spelling mistake in error messages * ASD cmds for Target and returning credentials to AuthSession request (#1338) * Add Add/Set/Remove commands for libraries on Targets. Return credentials on AuthorizeSession request. Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com> Co-authored-by: Louis Ruch <louisruch@gmail.com> Co-authored-by: Todd Knight <T.Alan.Knight@gmail.com> Co-authored-by: Jeff Mitchell <jeffrey.mitchell@gmail.com>	5 years ago
Jeff Mitchell	054c0aa538	Update password generated account IDs (#1290)	5 years ago
Jeff Mitchell	7da892a054	Managed Groups OIDC Repo (#1254)	5 years ago
Jeff Mitchell	7ee8ddba6a	True up some spelling	5 years ago
Jim	c7bac26e1a	Fix a broken auth token test and support conditional tests for "versioned" releases. (#1195)	5 years ago
Jeff Mitchell	e6af51943d	Add read:self and delete:self to auth tokens and add logout command (#1162)	5 years ago
Jim	75108cbc8b	Ongoing OIDC: return the primary account info along with the user. (#1145)	5 years ago
Todd Knight	71673ea161	Pass disable_discovered_config_validation in the API change-state call (#1100)	5 years ago
Jim	dd0f34bc35	Add new OIDC auth method. (#1090) ... Co-authored-by: Todd Knight <T.Alan.Knight@gmail.com> Co-authored-by: Jeff Mitchell <jeffrey.mitchell@gmail.com> Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com>	5 years ago
Jeff Mitchell	6605b4c9f8	Revamp authenticate (#1025)	5 years ago
Jeff Mitchell	c63529e020	Fix WithKeyId after recent PR (#978) ... Fixing the session passing around its key ID (which had previously been blank) exposed another issue which was caught during bats tests: WithKeyId statements using the wrong key, which prior to now had been ignored because the argument was empty, resulting in an ineffective option. This fixes those situations. It also updates a bats test that didn't get updated in the CLI return status PR.	5 years ago
Jeff Mitchell	9649a19d23	Update CLI error codes to be consistent with `connect` (#976)	5 years ago
Jeff Mitchell	72aa110f72	CLI JSON updates (#962) ... See Changelog for details.	5 years ago
Jeff Mitchell	341e7aff0d	CLI Generation (#953)	5 years ago
Todd Knight	909026ed3d	API filter tests (#958) ... * API filter tests. Requires that PR https://github.com/hashicorp/boundary/pull/957 is merged and tagged first. * Updating /api dependency.	5 years ago
Jeff Mitchell	22462cdd84	Add read:self and cancel:self to sessions; add dev unpriv user (#923) ... * Sessions support for read:self and cancel:self permissions. This is automatic with fallback; no API change is needed. * Strips out values from listing if only self actions are allowed on the resource and the user ID is different. This prevents a default ACL of id=*;type=session;actions=read:self,cancel:self from causing all values to be visible when listing. * Adds an unprivileged user to development/testing instances. This user has only default permissions plus the ability to authorize sessions in the default target.	5 years ago
Jeff Mitchell	5266eb3269	Don't error worker filters if a key isn't found, treat it as a non-match (#900)	5 years ago
Louis Ruch	6e40d8e483	Update anon listing test (#917)	5 years ago
Jeff Mitchell	b8a953ba53	Add anon listing test	5 years ago
Jeff Malnick	10a5f1b832	test: add tests for authorized-action (#893)	5 years ago
Jeff Mitchell	717a3b52ee	Add worker tagging (#862) ... This allows workers to have tags assigned to them, which can then be filtered to control which targets they are allowed to serve.	5 years ago
Todd Knight	aa4157639c	Change Format of API Error (#784) ... This is a wire breaking change for json errors since we have removed and renamed a few fields, and changed part of the structure of the returned json error. * Remove redundant fields in Status, details.error_id, details.TraceId and details.request_id from API errors. * Added wrapped errors and "ops" field to indicate the internal operation of the errors returned by the API. * Added error cli error reporting of Op and wrapped errors.	5 years ago
Jeff Mitchell	e2e4631929	Add ability to set public cluster address (#761)	6 years ago
Jeff Mitchell	1f48d97327	Support for Unix domain socket listeners (#705) ... The client already supports protocol-agnostic transport so handles unix:/// schemes.	6 years ago
Jeff Malnick	76a891cfe2	tests: remove testing of revision output for now (#667)	6 years ago
Jeff Malnick	e08ab98a2b	tests: add CLI tests for version output (#664)	6 years ago
Jeff Mitchell	13631064fc	Don't create a default role when the scope being created is a project (#658) ... * Don't create a default role when the scope being created is a project * Fix up some more tests	6 years ago
Jeff Malnick	dfa94c7990	tests: allow overriding auth variables in CLI tests (#647) ... * tests: ensure default user can connect to default target and ensure created target has intended host sets * docs: update readme for CLI tests with how to run against arbitrary clusters * feat: override auth variables in CLI tests	6 years ago
Jeff Malnick	7f48e2edc6	tests: ensure default user can connect to default target and ensure created target has intended host sets (#646)	6 years ago
Jeff Malnick	3c1670c0d0	tests: add scopes CLI tests (#645)	6 years ago
Jeff Malnick	70e696a446	tests: add roles CLI tests (#644)	6 years ago
Jeff Malnick	cad8949ac8	tests: add host sets CLI tests (#643)	6 years ago
Jeff Malnick	bee1412785	tests: add host catalog CLI tests (#639)	6 years ago
Jeff Malnick	0f1fbea8f7	tests: add hosts CLI tests (#638)	6 years ago
Jeff Malnick	c4c1e108b0	test: add groups resource CLI tests (#633)	6 years ago
Jeff Malnick	90e1dbe444	tests: add bats tests for testing CLI on arbitrary Boundary deployments (#614)	6 years ago
Jeff Mitchell	5bc8c8d921	Fix API test	6 years ago
Jeff Mitchell	5cbfcf8eb6	Shuffle things around out of new modules	6 years ago
Jeff Mitchell	20aef738c4	Add host-catalogs CLI command. (#312)	6 years ago
Jeff Mitchell	8f579c75c3	paum -> ampw (#303)	6 years ago
Jeff Mitchell	b47cca0329	Add (non-db aspects of) the recovery key workflow (#286)	6 years ago