70 Commits (c8ee312ae2e145afdc28ac4abb16d68d1ac2000d)

Author	SHA1	Message	Date
Jeff Mitchell	8322916fb4	Remove all deprecated Err/Wrap/E calls (#3286)	3 years ago
Jim	e4d6742514	feat (action/resource): add Download and SessionRecording	3 years ago
Louis Ruch	8ea61054cd	feat(storage): Add support for Storage Bucket API	3 years ago
Jeff Mitchell	fc664eb35f	Improve grant validation (#3081) ... This adds several relationships into the resource package that are then made use of to improve the grant validation that runs during Parse. By understanding mappings between ID and resource type the validation checks can far better align with actual allowed grant formats so we can find several common misunderstandings before they simply result in an authorization failure. In the future some of these functions could be used in the logic in the actual ACL checks but it's not currently necessary and I don't want to lump that change in with this. Note that the tests have not meaningfully changed, but they _have_ had to be updated because the stricter validation is actually catching issues.	3 years ago
Jeff Mitchell	d391dc6ab7	Add top level resource prefix function in globals (#3075)	3 years ago
Jeff Mitchell	b76b24a4ad	Move prefixes for many packages into the globals package (#3069) ... This is a prerequisite for some enhancements to grant validation	3 years ago
Johan Brandhorst-Satzkorn	3c29308673	chore: Add license headers to all files	3 years ago
Johan Brandhorst-Satzkorn	edd323b73a	Key Rotation/Destruction (#2477) (#2607) ... * Key Rotation/Destruction (#2477) * fix: Correct kms.CreateKeys comment This method does not return anything. * feat(kms): Add new ListKeys method The new ListKeys method wraps the underlying KMS wrapper, returning all the keys in the scope specified. * feat(scopes): Add new scope actions for key management Adds list-keys, rotate-keys and revoke-keys actions for scopes. * feat(scopes): Add ListKeys API endpoint This new endpoint lists all keys in a scope * feat(api): Add ListKeys to Scopes client This has to be a custom function because it is a custom action and doesn't map well to any of the existing templates, or generalises well in a way that would make it reasonable to create a new template. * feat(cli): Add new scopes list-keys command * add RotateKeys function to the kms repository * Add Rotate Keys Endpoint to Scopes Api (#2360) * add rotate keys endpoint * add tests for RotateKeys endpoint * remove that one comment I forgot about * addressing more PR comments * Add Rotate Keys CLI Command (#2395) * add cli command and client * add bats tests * fix some info text and pr comments * write a new rotate keys description * Add Missing DEK Foreign Keys (#2408) * add missing fks * mark key as nullable in gorm * update tests to use a new wrapper with a real key_id * fix formatting in test and add keys for auth_token test * replace key values * revert postgres_40_01_test.go * style: reformat sql for readability * refactor: change type of key_id columns to kms_private_id * Store key_id and scope_id with oplog entries, re-type columns referencing data key version (#2431) * fix(db): Correct types of data key version referencers * fix(oplog): Fix missing error handling * feat(oplog): Store key_id and scope_id with oplog entries This migration truncates all existing oplog entries, as migrating existing data would have been complex and likely unnecessary. * Update view references * Always delete oplog entries when dropping keys * Fix rebase issues * Fix sql tests * Add schema for new kms destruction jobs (#2479) * feat: Add schema for new kms destruction jobs The schema lets us manage destruction jobs per-table while providing a guarantee that only one run is running at a time. * Apply suggestions from code review Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com> * Fix table reference and tests Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com> * update domain, api, and cli to all include key versions in outputs (#2472) * update domain, api, and cli to all include key versions in outputs * fix go sum * add migration to fix immutable error * fix final test issue as well as make gen issue * rebase and fix * address johan pr comments * make gen, didn't realize comments were included * update migrations based on new migration 55 file 03 * Move migrations to 56 * feat: Add key version rewrapping function registry (#2478) The new RegisterTableRewrapFn can be used by a domain to register a callback to use when rewrapping data in a specific table name. * feat(kms): Add scopeId to RewrapFn (#2539) This makes it much easier for rewrap functions to look up the correct wrapper and limit the number of rows searched. * Update Root Cert Proto Definition (#2542) * update root cert proto definition to properly identify public_key as the table primary key * add the test fix as well * Rewrap Functions for Encrypted Tables (#2532) * add oidc rewrap * add argon2 config rewrap * add auth token rewrapping * add worker auth cert rewrapping * check the err * add username password credential rewrapping * add ssh private key credential rewrapping * add vault client certificate rewrap function * add host catalog secret rewrap function * cleanup a little * add host vault token rewrap function * add session credential rewrap function * add session rewrap function * add worker auth rewrap function * update test to include full assert gamut * add session rewrap function TEST * bring all tests up to current standards, make more readable, and include full assert gamut * rework all rewrap functions to utilize scope id * update comments, queries, sql refs, etc, in response to PR comments * forgot a comment * remove hmac updates and fix a couple comments again * Rewrapping Registered Tables Test (#2555) * add the registered table test as well as the two missing rewrap functions * address pr comments, add db r/w conversions, cmp.diff, sql comment, remove (now) faulty immutable test * Add ability to list key version destruction jobs, recurring jobs and destroying key versions (#2498) * feat(kms): Add ability to list data key version destruction jobs * feat(scopes): Add ListKeyVersionDestructionJobs * feat(cli): Add list-key-version-destruction-jobs * feat(kms): Add recurring job that performs table rewrapping The new recurring job runs for each job table with a registered rewrapping callback. It attempts to become the running run and start its rewrapping, gracefully handling sudden interruptions by resuming it's work if it finds evidence of sudden interruption. * feat(kms): Add recurring job for destroying key versions The new job monitors the database for data key version destruction jobs that have finished rewrapping all its data, performing the final data key version revocation. * feat(kms): Add DestroyKeyVersion DestroyKeyVersion immediately destroys a key version if it is a root key version or a data key version which encrypts no data. If the data key version encrypts data which needs to be rewrapped, it queues an asynchronous job to complete the rewrapping operation. * feat(scopes): Add DestroyKeyVersion API endpoint * feat(cli): Add scopes destroy-key-version CLI command * Ensure all secret updates include key ID (#2556) * feat(all): Ensure all secret updates include key ID When updating a secret, it is paramount that the key ID is also updated, as it is the only means we have of ensuring that we only destroy keys once there is no more data associated with the key. * Remove fix to session repository for now This breaks the current private key derivation method * add new encrypt/decrypt funcs and made necessary proto changes (#2557) * add new encrypt/decrypt funcs and made necessary proto changes * address all PR comments * fix test panic * add param checking to rewraps * fix(schema): Correctly organize migrations again * Review comments part 1 * Review comments part 2 * Review comments part 3 * Store cert private key, encrypt tofu token (#2583) * fix(session): store cert private key, encrypt tofu token Previously, we would derive a session certificate private key from the session key used in each project, and not store the key. We would also encrypt the tofu token with the database key but not store a reference to this key in the database. This change fixes this by replacing our key derivation with a key generation step, and instead store the generated key, encrypted, in the database. We also ensure that any new tofu tokens are encrypted with the same key. To handle existing sessions, we lazily rewrite the sessions whenever a user lists them. There is a minor risk that a user could end up destroying the database key that encrypts the tofu token before that session has been rewrapped, but this is going to be rare and temporary. * feat(session): Allow the random source for secrets to be configured * Review comments * Minor fixes * More minor fixes * More small fixes Co-authored-by: Danielle Miu <dani.miu@hashicorp.com> Co-authored-by: Danielle <29378233+DanielleMiu@users.noreply.github.com> Co-authored-by: Michael Gaffney <mike@gaffney.cc> Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com> * fix(session): Unset session key ID when scope is deleted Unsetting the key ID allows the session to live beyond the lifetime of the scope, while allowing the scope to cascade delete its keys. * fix(migrations): Rewrite migration 56 tests These tests can no longer use the normal test helpers as they try to use the new oplog table structure, so we have to manually write all the interactions. * fix(migrations): Rename migration 58 file name This was the name used in the release branch, update it to the new migration number. * fix(e2e): Add some comments and debug to kms tests * fix(session): Handle empty project and user IDs in read When a project or user is deleted, the session is automatically canceled and the respective field on the session is unset. LookupSession did not handle this case gracefully, and would error on decryption in this state. Skip decrypting sessions that do not have either a user or project to avoid this error. The decrypted values are not used for a canceled session. * Fix worker test * Increase test timeout Co-authored-by: Danielle Miu <dani.miu@hashicorp.com> Co-authored-by: Danielle <29378233+DanielleMiu@users.noreply.github.com> Co-authored-by: Michael Gaffney <mike@gaffney.cc> Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com>	3 years ago
Johan Brandhorst-Satzkorn	f57454b6b9	Rewrite interface{} to any (#2535) ... * chore: Rewrite interface{} to any * Add ignore revs	4 years ago
Jeff Mitchell	53b5e532d5	Remove deprecated methods/fields on targets (#2393)	4 years ago
Irena Rindos	18dff62b7b	Merge BYOW GA branch to main (#2398) ... * feat(workers): Add ability to read and reinitialize cert authority (#2312) * feat(workers): Return Release Version on worker list and read (#2377) * feat(cli): Read and reinitialize worker certificate authority (#2387)	4 years ago
Jeff Mitchell	01fb949d0b	Add controller-led worker auth flow (#2413) ... This adds an action to the API that allows pulling out an activation token that can be given to a worker in its config (directly or via env or file). The worker can pass this along with its normal request in place of its normal nonce and the server will automatically accept it. Co-authored-by: Irena Rindos <irenarindos@users.noreply.github.com>	4 years ago
Johan Brandhorst-Satzkorn	aef9073fa6	Upgrade to Go 1.19 (#2347)	4 years ago
Haotian	28f53a64b4	feat(workers): implement worker service add/set/remove api tags	4 years ago
Jeff Mitchell	704d68848c	Merge remote-tracking branch 'origin/main' into llb-byow	4 years ago
Louis Ruch	68eb6e2bed	chore(targets): remove deprecated credential libraries on target resources (#1533)	4 years ago
Louis Ruch	58d546cdd4	feat(credential): Add static credential store and username_password credential	4 years ago
Damian Debkowski	3441fa0447	(fix) added more context to error details for cli command (#2170) ... * (test) added more context to error details for cli commands * (fix) refactor solution to utilize custom error type casting * (test) added api unit test to validate InvalidArgumentError return message * Update internal/types/subtypes/error.go Co-authored-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com> * (refactor) rename InvalidArgumentError into UnknownSubtypeIDError Co-authored-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>	4 years ago
Jeff Mitchell	5d3facf561	Merge remote-tracking branch 'origin/main' into llb-byow	4 years ago
Jeff Mitchell	d6362f9e5e	Restrict anonymous user capabilities (#2156)	4 years ago
Jim	fdf43fc4fe	feature (worker): add CreateWorker(...) service (aka API) (#2143)	4 years ago
Jeff Mitchell	c59d0df611	Add worker creation ACLs (#2138) ... This is a bit more than simply adding the action; we had some checks to ensure that list/create could only happen/not happen in some cases and those had to be modified to account for subactions.	4 years ago
Todd	b58dada40d	Move the server daemons into their own package (#2061) ... * Move the server daemons into their own package With this change the internal/server package now is dedicated to the server domain (workers and controllers) and not to the server daemons. The hope is that this will make it easier to keep the daemons decoupled from the representation of the servers in the database.	4 years ago
Timothy Messier	c8be4a9890	refact(target): Switch subtype attributes to oneof ... This updates the target resources to use a oneof for attributes. This allows requests to the targets service to leverage the new subtypes.AttributeTransformerInterceptor so that attributes are strongly typed prior to the service handler.	4 years ago
Johan Brandhorst-Satzkorn	5b71aa6212	refact(subtype): Remove copied structToProto methods ... The original idea was for these methods to eventually become entirely encapsulated by the subtype package, but as it turns out we still have use for them in custom transformation functions, so move the subtype package to use these shared functions instead.	4 years ago
Johan Brandhorst-Satzkorn	6d56fcd26f	feat(subtypes): Add new transformation registration option ... This allows the user to register a custom transformation callback for a specific protobuf message, in cases where the transformation is sufficiently complex for it to be cumbersome to formalise the logic through protobuf options.	4 years ago
Johan Brandhorst-Satzkorn	1fd44ee0f1	refact(subtypes): Remove return from transformations ... This return value is just a reference to the input, which is being mutated. We may as well make that clear via the signature, which will also simplify the new custom transformation function registration logic.	4 years ago
Johan Brandhorst-Satzkorn	53481146fd	feat(authmethods): add well typed attribute options (#1972) ... Add well typed attribute options to the authmethods proto definitions, while hiding them from being rendered in the openapiv2 spec.	4 years ago
Timothy Messier	3d58ce5f5c	feat(subtypes): Add function to convert protobuf for filtering ... This will transform a protobuf message with a oneof for attributes into the API format to support filtering for list endpoints.	4 years ago
Timothy Messier	198a39ef1e	feat(subtypes): Add grpc interceptor for transforming attributes ... This interceptor is responsible for maintaining compatability with the JSON API with the change to the protobuf messages. The messages are changing from a single attribue field to a oneof to explicitly associate subtype attribute messages with the resource message. The interceptor transforms the request prior to the auditReqeust interceptor, and transforms the response after the auditResponse interceptor. This allows the audit interceptors to use all the necessary classification information when creating the audit events.	4 years ago
Timothy Messier	af93f7fccb	feat(subtypes): Attribute field registration and lookup	4 years ago
Timothy Messier	31ef40ec5a	feat(subtype): Add global registry	4 years ago
Timothy Messier	1ae388438f	feat(subtypes): Add function to provide all known prefixes	4 years ago
Timothy Messier	02b2291375	feat(subtypes): Make Registry thread-safe	5 years ago
Jeff Mitchell	1f7999ed97	Add authorized collection actions output for credential stores (#1530) ... Add authorized collection actions output for credential stores	5 years ago
Jeff Mitchell	92809b733a	Fix unknown permission issue with credential-store (#1524) ... This also affected credential libraries and managed groups. It seems that any end-to-end tests of these grants currently had them included in wildcards instead of individually, but these types were never added to the valid list for resources. Additionally, credential store wasn't added as a valid top level type for pinning purposes.	5 years ago
Jeff Mitchell	f8a51b987c	Migrate target host sets -> host sources (#1424) ... This follows the same model as https://github.com/hashicorp/boundary/pull/1413 See there and CHANGELOG for more details.	5 years ago
s-christoff	6b78108ecf	Update primary error functions to take a context, deprecate old functions (#1358) ... * add new error funcs which take a ctx and deprecate existing funcs * use context where available when writing error events * remove circular internal/errors pkg dep * suppress some chatty errors using errors.WithoutEvent() * convert auth oidc funcs to take context where needed to write events Co-authored-by: Jim <jlambert@hashicorp.com>	5 years ago
Jim	70ad081e39	Events: convert from hclog to events (#1409)	5 years ago
Todd Knight	10e8aeae4b	Subtype Id registration moved to its own package (#1416) ... * Allow repo packages to register subtypes and id prefixes. * Refactor out request authentication logic to a new package as a sibling to handler.	5 years ago
Jeff Mitchell	ab6f3eaeb4	Migrate credential-library nomenclature around targets to credential-source (#1413) ... See the CHANGELOG entry for more information. Co-authored-by: Jim <jlambert@hashicorp.com> Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com>	5 years ago
Michael Gaffney	df35699c4e	Integrate with Vault to retrieve and manage per session credentials (#1308) ... * Fix down migrations * Udpates due to merge * Sentinel values only have to be greater than 0 * Create credential related sdk structs and methods. * Post merge interface updating. * Removing scratch code that was not cleaned up. * updates * Convert client certificate and client certificate keys into pem blocks and validate. * Add newline at end of file. * Faster check for "s" at the end of a resource name when generating from templates. * make fmt * updates * Add dynamic credentials to a session * Refactor: move contents of file * updates * Updates * Add InvalidDynamicCredential error * Assign credentials * Allow updating client certificates seperate from the client certificate key. * Replace application purpose string with enum value * PR feedback / fix go test . * Delete dead code * Use common view * Refactor: extract interface and rename method * Refactor: move code to eliminate privPurpLibrary * Add comment to requestMap * Refactor: do not export methods on an unexported struct * Update internal/db/schema/migrations/postgres/10/03_vault_credential.up.sql Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com> * Add comments to credential Purpose constants * Reorder switch statements to handle errors first * PR feedback * update * fix merge * Organizing the scheduler code and extract the individual store creation test helper function. * Run 'make fmt' * Create certificate when creating store. Allow vault address to be passed in as option * Refactor: rename database views and associated structs This change renames two database views and associated go structs to use a consistent naming pattern. Views ending in 'private' contain encrypted credentials needed for connecting to Vault. Views ending in 'public' mirror the 'private' views but do not include any encrypted values. This change does not include any changes to functionality or behavior. * Adding SAD target operations for credential libraries to the SDK. * Add status field and enumeration table for credential status * Refactor: rename Status to TokenStatus * Add status to vault credentials * Creating the template for the CLI and helper functions. * Add private database view for vault credentials * Add scope_id to view and cleanup TODO comment * Reformat embedded sql queries * Add vault client helpers * updates * white space * Add LookupLease test * CLI commands for Store work. Renamed fields in proto w/ vault prefix. * Test issue credentials with client TLS * Add additional checks to vault ping * Add an error for vault token capabilities and create a VaultToken error kind * Add support for working with Vault policies * Use Vault 1.7.2 for testing * Add testing helper for creating a vault client with a non-root token * Fix Vault renew lease test The Vault renew lease endpoint accepts an optional duration parameter for requesting the number of seconds to extend the lease by. However, Vault is not required to honor this request. * Add a var for the required capabilities of a boundary Vault token * Enhance the test helper for adding a policy to Vault * Allow the test role in postgresql to revoke credentials * Remove out of date comments * Enhance Vault test helpers for the database secrets engine * Reformat comments * Add a method for getting the capabilities of the current Vault token * Add revoke lease to internal Vault client * Reformat: combine err return line with check for nil line * Remove redundant checks in test code Remove checks in test code that are redundant with checks in the test harness. * Refactor CreateToken test helper to return the secret and the token * Fix build failures caused by signature change in vault.CreateToken * Fix Vault version and Vault API link in comments * ran `make gen` after merging from mgaffney-vault. * Remove Vault prefix from attribute fields in API, create a base CLI option for adding field prefixes. format now uses that prefix for error reporting. * Template now allows attaching attribute field prefixes. * Credential store uses PrefixAttributeFieldErrorsWithSubactionPrefix. * Fixing naming of vault flags and helper flags. * Updating more references to generated api methods. * Make the capabilities String method more readable * Add comment outlining PrintApiError usage of Options. * Refactor test to move declarations closer to first use * Fix comment * Implement revoke method on vault repository * Tests now verify certs, CA, and tokens, addresses can be updated in a store. * Added test for creating store with pk/cert in same field. * Allo updating certificates seperate from private key. Only contact backing vault service when updating token. * Fixing bad tests and removing unused code. * Adding tests for various ways of updating tokens. * Add private credential * Add token revocation job and cred renewal/revocation job * Regen after merge. * Fixing field mask related merge error. * Create Library CLI. Removes vault prefix from path. Adds a WithoutCleanup testOption for vault. * Set all credentials to revoked when a token is revoked * Fixing missed renaming update. * fix copy/paste error. * Fixing code generation errors after merge. * make gen after merge. * Update internal/cmd/commands/credentialstorescmd/vault_funcs.go Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com> * Update internal/cmd/commands/credentialstorescmd/vault_funcs.go Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com> * Update internal/cmd/commands/credentialstorescmd/vault_funcs.go Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com> * Refactor: move migrations from 10 to 12 * Removing logic from transaction. * wrapping client errors in the calling op for CredentialStore's client() method. * Redact token * redact client key * updates * Add trigger to revoke credentials when session is canceled or terminated * Fixing some help text for some flags. * Add ability to load values from files (file://) or environment variables (env://). * Updating comments * fix bad merge * feedback * Adding issued credentials to sessions. * Error on capabilities * updates * Expose NewVaulTestServer * Add godoc * Adding generated SessionCredential struct for API. * Progress on building handler test for Authorize Session. * feedback * Update policies * Add 'revoke' as a status for Vault Tokens * Add delete_time to vault credential store * Create dynamic credentials at session creation time. * Tests pass with credentials attached to AuthorizedSession result. * Only call issue if there are credentials to issue. * Fixing output only issues. Adding type to credential library. * Fix test to clear un/pw generated. confirm generated un/pw changes from 1 authorize session call to another. * Update lookupTokenError * Add delete_time to private credential store * Refactor: move private store to separate file * Fixing tests for AuthorizeSession and verifying error cases. * Only create the credential repo if there are credentials to issue. * Rename Library field to CredentialLibrary. * Fixing missing change of Library to CredentialLibrary. * Removing Default SDK functions for fields which are required and update CLI to match. * Replace resource specific id variables with "id" and the input parmater for paths to be just a single string and the resource name as it should show up in the path. * Soft delete vault credential stores * Prevent new SQL functions from silently overwriting existing functions * Rename "Hmac" to "HMAC" in the human readable cli response table. * Fix tests * Add vault database cleanup jobs * Add ability to set the token period in the vault test harness * updates * Add not_null_columns func test * remove unused struct * Up no output timeout * Validate current vault token on addr update * Fix client cert change test * add cert test * Update revocation job where * Fix comment * Use Vault's default port in CLI help strings * Remove down migrations * Increase migration file numbers by 1 * Run make migrations * Move migrations from 12 to 10 * Increase Circle CI timeout for acceptance tests * Fix spelling mistake in error messages * ASD cmds for Target and returning credentials to AuthSession request (#1338) * Add Add/Set/Remove commands for libraries on Targets. Return credentials on AuthorizeSession request. Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com> Co-authored-by: Louis Ruch <louisruch@gmail.com> Co-authored-by: Todd Knight <T.Alan.Knight@gmail.com> Co-authored-by: Jeff Mitchell <jeffrey.mitchell@gmail.com>	5 years ago
Jeff Mitchell	179a49657e	Managed Groups service (#1262)	5 years ago
Jeff Mitchell	e6af51943d	Add read:self and delete:self to auth tokens and add logout command (#1162)	5 years ago
Jeff Mitchell	727d1c7085	Add "no-op" action and use it as default for scope permissions (#1138) ... Because of listing visibility, if no actions are assigned to a user for an individual resource, it's excluded from the list. This is fine and good. However, when we made a change in 0.2.0 to restrict the information returned from listing for the anonymous user for scopes and auth methods, it exposed the fact that because of the way this visibility works we had to add `read` to scopes by default -- thereby rather defeating the purpose of restricting what goes back to `u_anon` requests. The purpose of "no-op" is to never be used for real actions within the API, but as something that can be granted to specifically enable visibility for the targeted user. This makes the controls over list visibility much more granular.	5 years ago
Jim	dd0f34bc35	Add new OIDC auth method. (#1090) ... Co-authored-by: Todd Knight <T.Alan.Knight@gmail.com> Co-authored-by: Jeff Mitchell <jeffrey.mitchell@gmail.com> Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com>	5 years ago
Dane Harrigan	433998494a	Use uint for internal types (#926)	5 years ago
Jeff Mitchell	cb3980cb8c	Remove list:self from main (#915) ... * Remove list:self from main As per futher discussions, we won't be supporting this, in favor of simply read:self and cancel:self (since listing now already filters out items on which you have no permissions). This doesn't remove list:self entirely from the code base as much of that will simply be converted to read:self and cancel:self -- that will be in a later PR. However, with this removed if we want to release we won't expose an API publicly that isn't going to stick around.	5 years ago
Jeff Mitchell	0629526319	Add read/list/cancel:self actions and support for ACL subactions. (#882) ... Update perms table in anticipation of session support for these.	5 years ago
Jeff Mitchell	dcb15cffbd	Add authorized actions output on resources (#870)	5 years ago