packer

Commit Graph

Author	SHA1	Message	Date
Jenna Goldstrich	8dcd9fe1a5	Ensure org ID is set and move UploadSbom to api package	1 year ago
Jenna Goldstrich	347c57306c	hcp: use enum for HCP SBOM upload Since the protos for uploading an SBOM for a build have been changed to use an enumeration instead of a plain string with the latest revisions to the HCP Packer SBOM support feature, we update how we reference those values for the SBOM format to use that enum instead.	1 year ago
Lucas Bajolet	9b3f29faf9	hcp: wrap completeBuild to mark as failed on error When a build cannot be completed without errors, the build state was left as running, unless the build explicitly failed, which meant that HCP Packer would be responsible for changing the status after the heartbeats for the build stopped being sent for two 5m periods. This commit changes this behaviour, by explicitly marking the build as failed if something did not work while trying to complete a build on HCP Packer, even if the local Packer core build succeeded before that.	1 year ago
Jenna Goldstrich	362c547211	hcp: integrate SBOM upload to HCP code Since packer now supports keeping track of SBOMs produced during a build, we add the code to integrate those changes into the internal/hcp package, so we do upload them on build completion.	1 year ago
Jenna Goldstrich	cf6a82fae8	Make error less brittle to fix upcoming error with HCP SDK Go changing how we return 404 error	1 year ago
Lucas Bajolet	9076c7b24a	internal/dag: remove unused code Since the DAG package was lifted from Terraform, its contents are more than what we need for now, so this commit cleans-up the package to keep only the currently needed parts of code. If we need to support more in the future, we can revert this commit, or pickup the changes again from Terraform.	1 year ago
Lucas Bajolet	09774aaeb8	internal/dag: adapt Validate to not check for Root The implementation of the DAG as extracted from Terraform relied on a Root vertex being injected into the graph as the last node to visit. This is used as a sanity check for Terraform, but doesn't apply to our use-case for now, as we are always executing everything and have no need for this root node. Instead, we change how Validate operates so it does not error in case there is no valid root node for the graph, but enables us calling it to check for self-referencing edges, and circular dependencies.	1 year ago
Lucas Bajolet	673c13ebe7	internal: add dag package The dag package is a port over from Terraform to Packer, changing what little there was to fit our current dependency ecosystem. Most of the changes are on the type of diagnostics returned, as Terraform has its own type for them, while we rely on hcl's Diagnostics. Other than that, the functionality is essentially equivalent, and the code was barely touched.	1 year ago
Lucas Bajolet	69ec781ee2	hcp: don't continue if HEAD doesn't exist In HCP's metadata package, especially the VCS/git parts, we keep the current HEAD for a repository, along with the state it is in, in order to report it to HCP Packer when the build completes. However, when a build is run on a template from an empty Git repository, and HCP Packer is enabled, the code would crash when trying to get the information on the current HEAD, as it doesn't exist. The git library we use returns an error in such a case, but this was ignored, leading to a crash when attempting to get the hash to this reference later on. This commit fixes the problem by NOT ignoring the error to get the head, and immediately stop processing the git data as it doesn't yet exist.	1 year ago
Devashish	265dfe105a	Remove suffix from CICD platform type (#13121 )	2 years ago
Devashish	aa6c5f8405	feat: Phase 2 Extension of Build Metadata (#13092 )	2 years ago
Jenna Goldstrich	078ad45860	HCP Packer Buckets: Change UpsertBucket to call GetBucket (#13059 ) * Update UpsertBucket to first call GetBucket, this will allow bucket level role based authentication, as CreateBucket uses project level auth * Fix one incorrect test failure message	2 years ago
Devashish	f19b9b7369	Remove GetMetadata from StartBuild and add to CompleteBuild for HCL (#12926 )	2 years ago
Devashish	70d2efec4d	Report metadata only when the build completes (#12923 )	2 years ago
Devashish	edc8a4dc0d	Use latest version of hcp-sdk-go Remove loggers and move AddMetadataToBuild method to Version	2 years ago
Devashish	8e92027546	HPR-1537: Packer Core sends metadata to HCP Packer in UpdateBuild API (#12891 )	2 years ago
Devashish	595b45e67c	Packer tracks Version and Plugins Metadata (#12860 )	2 years ago
Lucas Bajolet	32f89015fe	hcp: fix hcp artifact extraction method With Packer 1.10.1 we started warning when a build failed to complete because of a potential incompatibility with the builder being used. This led to cases in which the build failed for other reasons, and Packer would still warn of potential incompatibilities, even if the builder was in effect HCP compatible. We attempted to fix this issue by introducing a new error type, and checks when we read the artifacts linked to a build, however this loop would fail when any one of the artifacts is not compatible with HCP Packer, leading to false failures. To avoid this problem, we log incompatibilities to the verbose logger, and only signal the problem if all the artifacts could not be used to upload data to HCP Packer, in which case it's almost certain that if the build succeeded and no artifacts are registered to the build, that all the components used are not compatible with HCP, and should be reported as such to users.	2 years ago
Lucas Bajolet	548893bbee	build: don't suggest lack of HCP support on fail When running a build with HCP Packer enabled, Packer attempts to push the build status to HCP. If the build fails, we update the status to BUILD_FAILED, and that's the end of it. If however the build succeeds, Packer attempts to get the HCP artifact from the builder, which will only succeed if the builder supports it. Otherwise, we'll get either nil, or an artifact type that is not compatible with what is expected for HCP support. When either of those happens, we warn that the builder may not support HCP Packer at all, so users are aware of the problem. However, when the error was introduced, it only looked at the fact that an error was produced, independently of the type of error. This caused legitimate errors while building to be reported as potential incompatibility between the builder and HCP, which was confusing to users. This commit changes this by introducing a new error type, only produced when the artifact either is nil, or failed to be deserialised into a HCP artifact, which lets us produce the incompatibility warning with more accuracy.	2 years ago
Devashish	91ce35f439	Expose versionFingerprint instead of versionId from packer template	2 years ago
Wilken Rivera	12b8c885a3	Update error messaging for HCP incompatible plugin builds (#12800 ) * Update error messaging for HCP incompatible plugin builds * Update command/build.go Co-authored-by: Lucas Bajolet <105649352+lbajolet-hashicorp@users.noreply.github.com> --------- Co-authored-by: Lucas Bajolet <105649352+lbajolet-hashicorp@users.noreply.github.com>	2 years ago
Devashish	3fa637c608	Migrate HCP Packer to API V2 and new nomenclature changes (#12794 ) * add hcp packer new nomenclature updates * Update docs with nomenclature changes * Update navigation bar links * Bump github.com/hashicorp/hcp-sdk-go from 0.81.0 to 0.82.0 * fix acceptance test template --------- Co-authored-by: sylviamoss <moss@hashicorp.com> Co-authored-by: Wilken Rivera <dev@wilkenrivera.com>	2 years ago
Wilken Rivera	c245b1fb7c	deps: Bump github.com/hashicorp/hcp-sdk-go to v0.73.0 * Updates cloud-resource-management API to use sable * Removes duplicate imports of the same SDK packages	2 years ago
hashicorp-copywrite[bot]	19055df3ec	[COMPLIANCE] License changes (#12568 ) * Updating the license from MPL to Business Source License Going forward, this project will be licensed under the Business Source License v1.1. Please see our blog post for more details at https://hashi.co/bsl-blog, FAQ at https://hashi.co/license-faq, and details of the license at www.hashicorp.com/bsl. * Update copyright file headers to BUSL-1.1 --------- Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com>	3 years ago
Wilken Rivera	60c66fdf9f	Add additional validation for manually set project ids When setting a project id via the HCP_PROJECT_ID env the client will try to validate the project by checking that it has an associated registry. If the project is invalid or not a valid UUID an error will be displayed to the user * Add comment to clarify usage of SDK loaded env. variables	3 years ago
Wilken Rivera	38101a64c9	hcp: Add support for project level service principals HCP supports two types of service principals: Organization-level and project-level. When a user tries to publish to an active HCP Packer registry using a plsp the client fails when configuring the client due to a API permission error; namely plsp do not have the permissions to query an org for a list of projects. Setting the HCP_PROJECT_ID does not resolve the issue because the call to ListProjects is still executed. This changes updates the client configuration params to obtain both the HCP Organization and Project IDs that will be used for connecting to the HCP Packer registry. With this change if a user provides a project Id via the HCP_PROJECT_ID environment variable no call to ListProjects will be made. Instead the value will be take as is and used to create the connection. A user connecting with a project level service principals must provide a valid HCP_PROJECT_ID in order to connect.	3 years ago
hashicorp-copywrite[bot]	d563a3c623	[COMPLIANCE] Add Copyright and License Headers	3 years ago
Lucas Bajolet	925cb5e541	hcp: support HCP_PROJECT_ID environment variable With HCP supporting multi-projects now, Packer needs to take it into account when picking a project from an organisation. This commit adds two cases: 1. multiple projects are defined, none is supplied through HCP_PROJECT_ID: in this case we will default to the oldest project defined for the organisation. 2. we supply HCP_PROJECT_ID: in this case, we pick the project with the corresponding ID, and use it for publishing metadata.	3 years ago
Wilken Rivera	eca75a6847	Run make fmt to fix checks	3 years ago
hashicorp-copywrite[bot]	b7df3ca36f	[COMPLIANCE] Add Copyright and License Headers (#12254 ) Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com>	3 years ago
Wilken Rivera	d880d1bca7	Update ReadFromHCLBuildBlock to use the hcp_packer_registry.Description In packer v1.8.5, the bucket's description was not properly set in the bucket object we use for HCP, therefore all the buckets created by Packer did not have their description updated. Before the change ``` --- FAIL: TestReadFromHCLBuildBlock (0.00s) --- FAIL: TestReadFromHCLBuildBlock/configure_bucket_using_only_hcp_packer_registry_block (0.00s) types.bucket_test.go:380: expected the build to to have contents of hcp_packer_registry block but it does not: &registry.Bucket{ Slug: "hcp_packer_registry-block-test", - Description: "", + Description: "description from hcp_packer_registry block", Destination: "", BucketLabels: {"org": "test"}, ... // 5 identical fields } FAIL FAIL github.com/hashicorp/packer/internal/hcp/registry 1.072s FAIL ``` After Change ``` ~> go test ./... ? github.com/hashicorp/packer/internal/hcp/api [no test files] ok github.com/hashicorp/packer/internal/hcp/env (cached) ok github.com/hashicorp/packer/internal/hcp/registry 1.130s ```	3 years ago
Lucas Bajolet	ec1d2e68f5	hcp: generate fingerprints on each new build Fingerprints are how we link a packer build to an iteration on HCP. These are computed automatically from the Git SHA in the current state, and are unique to the bucket/iteration. The main problem with this approach is that while sound in theory, it quickly falls apart when users want to run the same build configuration twice, but expect a new image to be created. With the current model, this fails, as the iteration with the current SHA already exists. While this is solvable through environment variables, or by committing a change to the repository, we think this is not clear enough, and causes an extra step to what should otherwise be a simple process. Therefore, to lower the barrier of entry into HCP, we change this behaviour with this commit. Now, fingerprints are randomly generated ULIDs instead of a git SHA, and a new one is always generated, unless one is already specified in the environment. This makes continuation of an existing iteration a conscious choice rather than something automatic, and virtually eliminates conflicts such as the ones described above.	3 years ago
Lucas Bajolet	73482bb636	internal: remove unused Labels on Iteration The iteration structure that we use for linking a packer build to an iteration on HCP defines a `Labels' attribute, which is never set nor read from at any point. Since it is unused, we remove it in this commit.	3 years ago
Lucas Bajolet	ca197afe9b	hcp: remove superfluous return value on GetBuilds This commit irons out one of the pain points of the HCP rework by introducing a HCPPublisher interface, implemented both by the JSON Core, and the HCL2 PackerConfig, which keeps a map of the build names used by Packer to the build names pushed on HCP. This in turn lets us go back to the old implementation of the GetBuilds function, which returns a list of (filtered) builds, and eventually an error if something went wrong while processing.	3 years ago
Lucas Bajolet	2200cb14ce	hcp: set the status to CANCELLED when a build is Related to the previous behaviour problem, this changes how the status of a build is set on HCP Packer, as a cancelled build is different from a failed one, we can set it to the appropriate state on cancellation.	3 years ago
Lucas Bajolet	4846301ff4	hcp: reject template type changes between builds As a follow-up to tracking the template type for an iteration, we actively check that it remains consistent between two invocations of packer, and reject the update if it is not the case.	3 years ago
Lucas Bajolet	6d0045607c	hcp: record template type when creating iterations In order to ensure users keep the same templates between two packer invocations for a single iteration, we start tracking the type of template used when building images.	3 years ago
Lucas Bajolet	08f1e85fb1	hcp: fix payloads for new SDK version The grpc_gateway v2 changes had some repercussions in the payloads of the API calls, making the current code non-functional against the latest SDK.	3 years ago
Lucas Bajolet	ccb5a22bb4	hcp: remove duplicated build registration for JSON In JSON templates, we register the builds to process on HCP at the initialisation of the Registry. However, during the HCP breakout, this code was duplicated with a slight difference for the JSON templates, and the build names were registered once during initialisation, with the expected full name, and once at the start of PopulateIteration, with the truncated name. This caused a bug when processing a template that contains a builder with both a name and a type, as both would be registered, but only one would complete, leading to the iteration being unable to complete.	3 years ago
Lucas Bajolet	719c8681cd	registry: don't use datasource output for ancestry The ancestry inferral code relied on HCP datasource outputs for deciding what would be the ancestry for an image built with HCP. This brings a dependency into the hcp package, which is not really necessary as we only need a few information from those entities, hence this commit removes the full dependency on the structures, in favour of more focused data structures where we only cherry pick what is necessary for this code.	3 years ago
Wilken Rivera	606e6c48f1	internal: reorganise registry/HCP code This commit reorganises the code for both the registry/API and the Orchestrator/Registry. The main difference with the previous version is how stuff is exposed. Now we only expose a Registry interface to the outside (previously named Orchestrator), which has several implementations: null is the default, and is returned if HCP is not enabled. The other implementations being HCL/JSON, both private to the hcp sub-package. The api (previously `registry') is the set of functionality that abstracts and calls the HCP API. It was meant to be merged with the `hcp' package, but because of a dependency loop with the datasources, both are separated for now.	3 years ago
Lucas Bajolet	1cee460d0d	hcp: extract all HCP-related code to hcp package As part of the work to displace everything related to HCP from scattered places around the Packer code, we move it all to an hcp package. This in turn reduces the amount of code that the commands have to integrate, and leaves the HCP details to its own enclave.	3 years ago
Lucas Bajolet	f3e4c2c3bb	tmp: move HCP logic to unwrap builds/post	3 years ago
Lucas Bajolet	f0d0a0e6c5	internal/registry: reverse image check condition In order to determine whether or not a build is done or needs processing, we check 3 things: 1. that its status is not done 2. that its ID is not empty 3. that it has no images linked to it The was the image check was phrased indicated that we checked for the reverse, so we reverse the naming to make it less confusing.	3 years ago
Lucas Bajolet	61c810e720	command: rework HCP-related error messages The HCP error messages were sometimes a bit terse in terms of the information it bubbles up to the user. This made them useful for people who already knew what to look for, but newcomers would almost certainly be lost because of the lack of information in those. To improve on this situation, we reword most of those error messages in this commit.	3 years ago
Lucas Bajolet	c2dc140896	hcp: read fingerprint from env during initialize Previously, we'd get the fingerprint for an iteration when creating the bucket. Since we moved the remainder of the logic for getting the iteration to a separate Initialize method, we also move the logic that reads the fingerprint from the environment to this function.	3 years ago
Wilken Rivera	6693128d73	Read loading over bucket name from env var	3 years ago
Wilken Rivera	148d5e3a2d	Add initialize method to Iteration Fingerprint initialize was previously occurring during the creation of a bucket. We want to be able to initialize a bucket and defer the setting of a fingerprint to a later point. * Update test to reflect new function signatures for Bucket and Iteration	3 years ago
Wilken Rivera	0efd064d29	Update IsHCPExplicitelyEnabled logic	3 years ago
Wilken Rivera	a427094426	Add env for HCP_PACKER_BUILD_FINGERPRINT	3 years ago

1 2

79 Commits (v1.12.0)