boundary

Commit Graph

Author	SHA1	Message	Date
Jeff Mitchell	0fd906d964	Merge branch 'main' into plugin-hostcatalogs	5 years ago
Jim	769416c2ff	refactor (controller): Use a grpc server for the grpc-gateway (#1576 ) * refactor (controller): Use a grpc server for the grpc-gateway This starts a grpc server for the grpc-gateway. The new gateway server uses an in-memory listener which also supports "dialing", so the http proxy can connect to the in-memory listener. This change allows us to use all grpc server features and was in part motivated by a need for an interceptor of inbound requests. With that said, having a "proper" grpc server for the gateway will provide a lot more flexibility and capabilities moving forward. The request context will still be verified via middleware in the http proxy (see wrapHandlerWithCommonFuncs). This verified context will be serialized into a protobuf and sent along to the grpc-gateway via metadata where an interceptor will add the verified request info to the request's context (see requestCtxInterceptor). The interceptor will verify the serialized request info via a shared ticket with it's companion in-memory http proxy. FYI: the ticket is simply random base62 string (see db.NewPrivateId). This change also necessitated a minor change to services. We need to return an empty response instead of nil, nil when there are no errors and the response is empty. * refactor (handlers): Rename gRPC Gateway filter Originally, this was called an OutgoingInterceptor, which is an unfortunate name since it's not an interceptor. Now that we're refactoring the gateway stuff and have proper interceptors, renaming this should remove ambiguity * refactor: Rework how 204 with empty msg is handled Recent changes to grpc gateway has necessitated a change to how 204s are handled by the gateway * tests (bats): Check status codes for delete operations	5 years ago
Jeff Mitchell	1c8e0ed1e1	Add scaffolding for external host plugins and convert aws/azure over (#1581 )	5 years ago
Jeff Mitchell	70e6536919	Swap server interface for client (#1580 )	5 years ago
Todd	e11a481256	Register Azure and AWS at controller start up (#1570 ) * Register Plugins. * Pass catalogs and persisted data when listing hosts for host sets. Register azure plugin at startup if not present.	5 years ago
Jeff Mitchell	7d71618df0	Add more host-catalog and host-set CLI support (#1567 ) Add more host-catalog and host-set CLI support Co-authored-by: Todd <T.Alan.Knight@gmail.com>	5 years ago
Todd	bbfef2802f	Placeholder Plugin Providers for Repos and added testPlugin struct (#1532 ) * Create minimalistic plugin map and test plugin. * Pass secrets and attributes to the plugin when creating catalogs or sets. Add tests for CreateSet and CreateCatalog repo methods. * make fmt and extract out getPrivateCatalog and getCatalog helper methods. * Exporting TestPluginServer so it can be used in the handlers. * Keep separate persisted data from host catalogs lookups which facilitate passing it onto the plugins. * Add TODO to merge get Catalog and get Persisted data. * Adding tests for unimplemented being returned by the plugin. * Use GetFields() instead of AsMap() to confirm attributes should be populated. Make clone consistent between catalog and sets. Co-authored-by: Jeff Mitchell <jeffrey.mitchell@gmail.com>	5 years ago
Jeff Mitchell	01d4383c1b	Add preferred endpoints logic to API/handler/DB and some CLI bits (#1529 )	5 years ago
Todd Knight	f2236482ad	Creating plugin catalog handler (#1521 ) The handler only supports create, lookup, and list for plugin based host catalogs.	5 years ago
Todd Knight	715982148f	Create Plugin Host Set Handler for Create, List, Get (#1518 )	5 years ago
Jim	68dd2a66ba	Remove unneeded hclog dependencies as we move forward with events (#1433 )	5 years ago
s-christoff	6b78108ecf	Update primary error functions to take a context, deprecate old functions (#1358 ) * add new error funcs which take a ctx and deprecate existing funcs * use context where available when writing error events * remove circular internal/errors pkg dep * suppress some chatty errors using errors.WithoutEvent() * convert auth oidc funcs to take context where needed to write events Co-authored-by: Jim <jlambert@hashicorp.com>	5 years ago
Jim	70ad081e39	Events: convert from hclog to events (#1409 )	5 years ago
Jim	11124794bf	Events: add node for formatting/filtering cloudevents (#1390 )	5 years ago
Jeff Mitchell	44329d4ff9	Repoint mlock import	5 years ago
Jeff Mitchell	140c54e319	Repoint base62 import	5 years ago
Chris Marchesi	5a70875726	internal/servers/controller: Worker failure connection cleanup (#1340 ) This commit adds the support to do the following: * Mark connections for non-reporting workers as closed. This is the controller counterpart to the worker functionality (see #1330). This is written as a scheduled job that does the work DB-side in a single atomic query. * Works to reconcile states if such a broken controller-worker connection resumes and a worker reports a connection as connected that should be disconnected. In this case, the controller will send an update request, and the worker will honor it and terminate the connection. * Further refinement of the grace period setting has been added here. We have converged on the current server "liveness" setting as our default here, which is half of the previous 30s (15 seconds, in other words). Additionally, this is now configurable on the controller and worker side, with the caveat that it's currently impossible to do so in config as the setting has been untagged in HCL. This is exposed so that we can run some sophisticated testing scenarios where we skew the grace period to either the controller or worker to ensure the aforementioned reconciliation works. * Some repository functions have been added to support the new functionality, in addition to some test code to the worker to allow querying of session state while testing. * Finally, we've added some add timestamp subtraction functions as well, basically serving as the opposite of the addition functions.	5 years ago
Jeff Mitchell	2cbcf9a563	Update usage of shared-secure-libs (#1393 )	5 years ago
Jim	1daa3ebefe	integrate events into the servers (#1355 )	5 years ago
Michael Gaffney	df35699c4e	Integrate with Vault to retrieve and manage per session credentials (#1308 ) * Fix down migrations * Udpates due to merge * Sentinel values only have to be greater than 0 * Create credential related sdk structs and methods. * Post merge interface updating. * Removing scratch code that was not cleaned up. * updates * Convert client certificate and client certificate keys into pem blocks and validate. * Add newline at end of file. * Faster check for "s" at the end of a resource name when generating from templates. * make fmt * updates * Add dynamic credentials to a session * Refactor: move contents of file * updates * Updates * Add InvalidDynamicCredential error * Assign credentials * Allow updating client certificates seperate from the client certificate key. * Replace application purpose string with enum value * PR feedback / fix go test . * Delete dead code * Use common view * Refactor: extract interface and rename method * Refactor: move code to eliminate privPurpLibrary * Add comment to requestMap * Refactor: do not export methods on an unexported struct * Update internal/db/schema/migrations/postgres/10/03_vault_credential.up.sql Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com> * Add comments to credential Purpose constants * Reorder switch statements to handle errors first * PR feedback * update * fix merge * Organizing the scheduler code and extract the individual store creation test helper function. * Run 'make fmt' * Create certificate when creating store. Allow vault address to be passed in as option * Refactor: rename database views and associated structs This change renames two database views and associated go structs to use a consistent naming pattern. Views ending in 'private' contain encrypted credentials needed for connecting to Vault. Views ending in 'public' mirror the 'private' views but do not include any encrypted values. This change does not include any changes to functionality or behavior. * Adding SAD target operations for credential libraries to the SDK. * Add status field and enumeration table for credential status * Refactor: rename Status to TokenStatus * Add status to vault credentials * Creating the template for the CLI and helper functions. * Add private database view for vault credentials * Add scope_id to view and cleanup TODO comment * Reformat embedded sql queries * Add vault client helpers * updates * white space * Add LookupLease test * CLI commands for Store work. Renamed fields in proto w/ vault prefix. * Test issue credentials with client TLS * Add additional checks to vault ping * Add an error for vault token capabilities and create a VaultToken error kind * Add support for working with Vault policies * Use Vault 1.7.2 for testing * Add testing helper for creating a vault client with a non-root token * Fix Vault renew lease test The Vault renew lease endpoint accepts an optional duration parameter for requesting the number of seconds to extend the lease by. However, Vault is not required to honor this request. * Add a var for the required capabilities of a boundary Vault token * Enhance the test helper for adding a policy to Vault * Allow the test role in postgresql to revoke credentials * Remove out of date comments * Enhance Vault test helpers for the database secrets engine * Reformat comments * Add a method for getting the capabilities of the current Vault token * Add revoke lease to internal Vault client * Reformat: combine err return line with check for nil line * Remove redundant checks in test code Remove checks in test code that are redundant with checks in the test harness. * Refactor CreateToken test helper to return the secret and the token * Fix build failures caused by signature change in vault.CreateToken * Fix Vault version and Vault API link in comments * ran `make gen` after merging from mgaffney-vault. * Remove Vault prefix from attribute fields in API, create a base CLI option for adding field prefixes. format now uses that prefix for error reporting. * Template now allows attaching attribute field prefixes. * Credential store uses PrefixAttributeFieldErrorsWithSubactionPrefix. * Fixing naming of vault flags and helper flags. * Updating more references to generated api methods. * Make the capabilities String method more readable * Add comment outlining PrintApiError usage of Options. * Refactor test to move declarations closer to first use * Fix comment * Implement revoke method on vault repository * Tests now verify certs, CA, and tokens, addresses can be updated in a store. * Added test for creating store with pk/cert in same field. * Allo updating certificates seperate from private key. Only contact backing vault service when updating token. * Fixing bad tests and removing unused code. * Adding tests for various ways of updating tokens. * Add private credential * Add token revocation job and cred renewal/revocation job * Regen after merge. * Fixing field mask related merge error. * Create Library CLI. Removes vault prefix from path. Adds a WithoutCleanup testOption for vault. * Set all credentials to revoked when a token is revoked * Fixing missed renaming update. * fix copy/paste error. * Fixing code generation errors after merge. * make gen after merge. * Update internal/cmd/commands/credentialstorescmd/vault_funcs.go Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com> * Update internal/cmd/commands/credentialstorescmd/vault_funcs.go Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com> * Update internal/cmd/commands/credentialstorescmd/vault_funcs.go Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com> * Refactor: move migrations from 10 to 12 * Removing logic from transaction. * wrapping client errors in the calling op for CredentialStore's client() method. * Redact token * redact client key * updates * Add trigger to revoke credentials when session is canceled or terminated * Fixing some help text for some flags. * Add ability to load values from files (file://) or environment variables (env://). * Updating comments * fix bad merge * feedback * Adding issued credentials to sessions. * Error on capabilities * updates * Expose NewVaulTestServer * Add godoc * Adding generated SessionCredential struct for API. * Progress on building handler test for Authorize Session. * feedback * Update policies * Add 'revoke' as a status for Vault Tokens * Add delete_time to vault credential store * Create dynamic credentials at session creation time. * Tests pass with credentials attached to AuthorizedSession result. * Only call issue if there are credentials to issue. * Fixing output only issues. Adding type to credential library. * Fix test to clear un/pw generated. confirm generated un/pw changes from 1 authorize session call to another. * Update lookupTokenError * Add delete_time to private credential store * Refactor: move private store to separate file * Fixing tests for AuthorizeSession and verifying error cases. * Only create the credential repo if there are credentials to issue. * Rename Library field to CredentialLibrary. * Fixing missing change of Library to CredentialLibrary. * Removing Default SDK functions for fields which are required and update CLI to match. * Replace resource specific id variables with "id" and the input parmater for paths to be just a single string and the resource name as it should show up in the path. * Soft delete vault credential stores * Prevent new SQL functions from silently overwriting existing functions * Rename "Hmac" to "HMAC" in the human readable cli response table. * Fix tests * Add vault database cleanup jobs * Add ability to set the token period in the vault test harness * updates * Add not_null_columns func test * remove unused struct * Up no output timeout * Validate current vault token on addr update * Fix client cert change test * add cert test * Update revocation job where * Fix comment * Use Vault's default port in CLI help strings * Remove down migrations * Increase migration file numbers by 1 * Run make migrations * Move migrations from 12 to 10 * Increase Circle CI timeout for acceptance tests * Fix spelling mistake in error messages * ASD cmds for Target and returning credentials to AuthSession request (#1338) * Add Add/Set/Remove commands for libraries on Targets. Return credentials on AuthorizeSession request. Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com> Co-authored-by: Louis Ruch <louisruch@gmail.com> Co-authored-by: Todd Knight <T.Alan.Knight@gmail.com> Co-authored-by: Jeff Mitchell <jeffrey.mitchell@gmail.com>	5 years ago
Louis Ruch	649ed32504	Scheduler updates (#1311 ) * Scheduler updates * Add convenience methods to db/timestamp.Timestamp * Remove recursive loop Co-authored-by: Michael Gaffney <mike@gaffney.cc>	5 years ago
Louis Ruch	feb5d7385d	Louis job scheduler (#1197 ) * Adds job repo * Adds scheduler * Adds plugin table which is a place holder for future expansion of the scheduler. Currently all jobs that are created are linked to the pi_system plugin which are jobs that are held in a local registeredJobs map populated at controller startup.	5 years ago
Jim	dd0f34bc35	Add new OIDC auth method. (#1090 ) Co-authored-by: Todd Knight <T.Alan.Knight@gmail.com> Co-authored-by: Jeff Mitchell <jeffrey.mitchell@gmail.com> Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com>	5 years ago
Jeff Mitchell	6c48f6ea77	Use pointers for any Uber atomic values (#1044 ) BK clued me into the fact that for some types, similar alignment problems exist as with the normal atomic package (e.g. 64-bit types on 32-bit architectures) if they are not pointer values. So change the few places they weren't pointers to pointers. None of these are 64-bit, but it feels like best practice for using the library and should just standardize on it.	5 years ago
Jeff Malnick	11c7c8dbe8	feat: make auth token staleness and duration configurable (#777 ) * feat: make auth token time to live and staleness configurable * docs: add configuration docs for auth token TTL and staleness Co-authored-by: Jeff Mitchell <jeffrey.mitchell@gmail.com>	5 years ago
Jeff Mitchell	e2e4631929	Add ability to set public cluster address (#761 )	5 years ago
Jim	19aecfefae	terminate "completed" sessions (#477 )	6 years ago
Todd Knight	e937b0ea27	Sessions Read/List/Cancel API and SDK (#369 )	6 years ago
Jeff Mitchell	07a7e9750a	Tie together the database-driven session handling with the worker and add relevant CLI comands (#370 )	6 years ago
Jeff Mitchell	23156afa11	Add in most of the proxy flow (#326 )	6 years ago
Todd Knight	544e78b593	Target Handler and SDK CRUDL & add\|set\|remove-host-sets methods. (#310 )	6 years ago
Jeff Mitchell	ac4d9fa311	Add nonce storage and replay prevention test (#293 )	6 years ago
Jeff Mitchell	b47cca0329	Add (non-db aspects of) the recovery key workflow (#286 )	6 years ago
Jim	9570897032	basic keys mgmt repo (#264 )	6 years ago
Jeff Mitchell	203e2b5dc2	Initial worker porting steps (#232 )	6 years ago
Jeff Mitchell	6661117d4c	The name. The name. The name!	6 years ago
Todd Knight	e5ec1f48b2	Account API handler for CRDL operations (#228 )	6 years ago
Todd Knight	c44a4c4795	Tools for auth token interception and authentication (#152 )	6 years ago
Todd Knight	6804c98b0b	Applying Host Catalog comments to Projects (#105 ) * NewService for the project service now returns a non pointer and an error. * Replacing t.Fatal calls with require.NoError calls. * Initialize empty slices following the recommended way. * Don't inline the prefix strings when validating requests. * Check error when cleaning up the test. * Add check that update updates the time. * Use google.golang.org/protobuf where possible instead of github.com/golang/protobuf. * Shorten the validateX function names. * Inline API responses. * Change test cleanup error logging to t.Errorf. * Remove unneeded creation of an extra project. * Inline regex check for invalid id. * Build the pb.Project inline when converting from db to wire proto. * Change stored repos into repo creating functions. * Using iam.TestScopes instead of creating them myself. * Remove out of date comments. * Update name to match go test naming standards. * Adding project service comments. * Using scopeType's new prefix method inside scope.go * Setting test project's name and desc to default to pass tests. * Change host catalog to project in test. * Fixing merge errors. * Fixing some more merge errors. * Fix typo in comments. * Fix typos in project_service.go. * Update error indicating provided id and the expected prefix. * Make the RepoFactory types in a common directory.	6 years ago
Todd Knight	7f6e78e66e	Handler hostcatalogs (#86 ) * Created host catalog service. * Fixing uri generation for create. * Use newly submitted static host catalog. * Copying over tests from projects. * Fix UpdateHostCatalogRequest merging proto. * Properly use static host catalog prefixes in tests. * Correct test to verify the expected uri and id prefix is set. * Add test for unsetting descriptions. * Correct bad spelling in comment. * Tweaking variable names. * Adding Stringer to the catalog type and setting the type on the response. * Setting up the catalogType to contain more info regarding its type. * Validate the type field in requests. * Remove unused method. * Comment describing invariants around types. * Wiring in the repo into the static catalog handler. * Updated comments to the exported methods. * fix: return the result of handlers.NotFoundError instead of just calling it. * Fixing typos referencing just hosts instead of host catlogs. * Update NewService returns an error and a non pointer type. * Pull "org_id" and "project_id" into constants and the validation of them into the ValidateAncestors. * Check errors for all RegisterHandlerServer calls. fix: Remove left over reference that could cause a panic. * fix: Changing "x := []string{}" to "var x []string" * fix: Log errors in test cleanup. * Changing the code to use google.golang.org/protobuf instead of the older github based protobuf library. * Changing all t.Fatal checks to require.NoError calls. * Changing unrecognized prefixes to Invalid Argument instead of Not Found. * fix: toDbUpdateMask now returns error if it was unable to find any valid paths. * Define CreatedTime, UpdatedTime, and Type at initialization time in toProto. * Remove unused host catalog from test. * Change a t.Fatal to a require.NoError that I missed earlier. * Change shutdown error logging to t.Errorfs. * Shorten validateX function names. * Inline API responses. * Add todo to make a new repo per request. * Inline regex check for invalid id. * Change stored repos into repo creating functions which satisfies ICU-223. * Wrap proto comments to be under 80 characters long. * Remove the exception on the read only fields that can be set in an Update request. * Removing out of date comment. * Use iam.TestScopes instead of making the project myself. * Renaming files to be consistant with go test standards and resource naming standards. * Remove TODO related to a task completed in an earlier commit. * Fix the previous commit which removed the wrong TODO.	6 years ago
Todd Knight	c0c6a9b0b8	Handler project (#44 ) * Renamed fakeRepo to mockRepo * Changing around TODOs to capture current plans with the code. * Remove the repo interface and use a real DB for tests. * Fixed a few tests. * Adding checks for properly formatted identifiers. * Updating code to account for recent merges. Relaxing some restrictions on what an id can look like. * Added small comment explaining why RegisterGrpcGatewayer interface exists. * Move project service up a call level. Long term I think it would make sense to define these services even higher up the call stack, potentially at the initialization of the controller server, near where the DB is initialized. * Making these fields align with those in the resources under hosts/... * Hard code the service initialization in the handler registration code. * Regenerate project related code proto code. * Removing type field for project. * Adds basic project handling in CLI/SDK (#47) * Adds SDK for creating and reading projects * Reorganizes a bit * Adds CLI for creating and reading projects * Fixes some some generation bugs * Adds redirect support for default org ID * Fix protobuf import references being moved. * Update client test to look for "orgs" and "projects" instead of "org" and "project" * Update expected segments test * Fix setAddr handling when default organization is in use * Set client in target after create or read Co-authored-by: Jim Lambert <jimlambrt@Jims-MBP-3.home> Co-authored-by: Jim <jlambert@hashicorp.com> Co-authored-by: Michael Gaffney <mgaffney@users.noreply.github.com> Co-authored-by: Jeff Mitchell <jeffrey.mitchell@gmail.com>	6 years ago
Jeff Mitchell	4d1b916348	Remove AllLoggers (#39 ) This was copied from Vault code but shouldn't be necessary any more as a change in hclog a while back meant that derived loggers share the log level of the parent.	6 years ago
Jeff Mitchell	5e82db8a2d	Minor bugfixes and cleanup	6 years ago
Michael Gaffney	5e5a56cac0	Refactor names to avoid stutter (#1 ) Since client code uses the package name as a prefix when referring to the package contents, the names for those contents need not repeat the package name. - Rename base.BaseCommand to base.Command - Rename base.BaseServer to base.Server - Rename config.DevConfig() to config.Dev() - Rename config.LoadConfigFile() to config.LoadFile() - Rename config.NewConfig() to config.New() - Rename config.ParseConfig() to config.Parse() - Rename controller.ControllerCommand to controller.Command - Rename controller.NewController() to controller.New() - Rename dev.DevCommand to dev.Command - Rename listener.ListenerFactory to listener.Factory - Rename listener.NewListener() to listener.New() - Rename version.GetVersion() to version.Get() - Rename version.VersionInfo to version.Info See: - https://golang.org/doc/effective_go.html#package-names - https://blog.golang.org/package-names#TOC_3. - https://github.com/golang/go/wiki/CodeReviewComments#package-names	6 years ago
Jeff Mitchell	482b4d56dd	Initial commit	6 years ago

45 Commits (1cf94d0e3edd25f412ffdb618b0e2116a7c343ea)