packer

Commit Graph

Author	SHA1	Message	Date
Lucas Bajolet	c2dc140896	hcp: read fingerprint from env during initialize Previously, we'd get the fingerprint for an iteration when creating the bucket. Since we moved the remainder of the logic for getting the iteration to a separate Initialize method, we also move the logic that reads the fingerprint from the environment to this function.	3 years ago
Wilken Rivera	6693128d73	Read loading over bucket name from env var	3 years ago
Wilken Rivera	148d5e3a2d	Add initialize method to Iteration Fingerprint initialize was previously occurring during the creation of a bucket. We want to be able to initialize a bucket and defer the setting of a fingerprint to a later point. * Update test to reflect new function signatures for Bucket and Iteration	3 years ago
Wilken Rivera	0efd064d29	Update IsHCPExplicitelyEnabled logic	3 years ago
Wilken Rivera	a427094426	Add env for HCP_PACKER_BUILD_FINGERPRINT	3 years ago
Lucas Bajolet	b3188637bf	registry: load bucket slug in command/registry Loadgin the slug from the environment within the Bucket implied that the Bucket was successfully created, which may fail outside of a Git directory when the iteration fingerprint is not set through environment variables. To make it possible to print as much information regarding the environment to the users at once rather than stopping immediately when this step fails, we move the logic to read the default value from the environment to the HCP setup code.	3 years ago
Lucas Bajolet	ae15ed339c	registry: add function to detect explicit HCP The HCP_PACKER_REGISTRY environment variable had its behaviour changed recently, as prior versions of Packer expected the attribute to be forcefully set to a value that is neither "off" nor "0" in order to get HCP integration to work, or that a "hcp_packer_registry" block was defined in an HCL template. Now, this environment variable defaults to not explicitely enable, but instead to explicitely disable HCP integration, and the feature switch fall upon HCP_PACKER_BUCKET_NAME. As an extra feature, we keep the prior behaviour alive when it is explicitely defined as a value to enable it. That way we can report errors if the rest is not defined, rather than silently ignore it. This function we add to env is the first stone to enable this behaviour.	3 years ago
Eng Zer Jun	f5ec0f1c57	test: use `T.Setenv` to set env vars in tests This commit replaces `os.Setenv` with `t.Setenv` in tests. The environment variable is automatically restored to its original value when the test and all its subtests complete. Reference: https://pkg.go.dev/testing#T.Setenv Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>	4 years ago
Lucas Bajolet	779a31a25f	hcp: use HCP_PACKER_REGISTRY to disable HCP The current behaviour for HCP integration is based on the presence of the HCP_PACKER_REGISTRY environment variable, where if it is either not "0" or "off", the HCP integration is enabled. This commit changes the behaviour to only use this variable to explicitely disable HCP, and the HCP_PACKER_BUCKET_NAME variable will condition if the HCP integration is enabled for a build or not in JSON templates. For HCL templates, the integration can also be disabled if the HCP_PACKER_REGISTRY variable is set to "0" or "off", and will be considered enabled if the HCP_PACKER_BUCKET_NAME is set, or there is a "hcp_packer_registry" block in a build.	4 years ago
Sylvia Moss	5c3d5c1295	Send channel id to HCP Packer (#11861 )	4 years ago
Lucas Bajolet	0a41694ee5	registry: add heartbeat call for running builds (#11846 ) * registry: add heartbeat call for running builds When a build is running, we send periodic heartbeats to HCP Packer's API. This will be used on the service side to detect if a build is stalled on the core side because of a crash or any other malfunction that caused Packer not to send an update on the status of a build. * registry: only update status on status update Prior to this commit, we'd send updates to both the labels and the cloud-provider whenever an update to the status of a build would be sent. This would cause a bug in which once a build reached the post-processing state, and its cloud-provider was set, the status could not be updated anymore, as the cloud-provider would be set and further updates are rejected by the platform. To avoid this problem, we only transmit the status when doing a status update, and no other fields along with it. * internal: publicise CompleteBuild function The markBuildComplete private function used to be called from a final status update to DONE, through the UpdateBuildStatus function. The problem being that the (now) CompleteBuild function not only updates the status of the build, but also all its metadata. For consistency, we remove the indirection, and explicitely call CompleteBuild when we want to finish a build. * internal: block status updates on DONE builds	4 years ago
Jenna Goldstrich	1e316c9076	Remove HCP Packer acceptance tests (#11842 ) * remove acceptance tests that have been moved to private repo * Remove HCP env vars from Acceptance Test nightly run	4 years ago
Wilken Rivera	846e4e6443	Associate Packer build to HCP Packer parent iteration (#11832 ) * Add support for data source lookup * Update failing test * Fix typo for HCP_PACKER_BUILD_FINGERPRINT env in test * Add test Fingerprint where needed; failing tests were not showing because they are properly set for acctest * Update to use BuildContext	4 years ago
Jenna Goldstrich	4449328786	Revert "Migrate to internal Registry tests (#11806 )" (#11817 ) This reverts commit `961c1d7046`.	4 years ago
Jenna Goldstrich	961c1d7046	Migrate to internal Registry tests (#11806 ) * Implement HPATS in Nightly build, delete registry par test * Update .github/workflows/acceptance-test.yml * add workflow-dispatch	4 years ago
Sylvia Moss	5c0988acbd	bump hcp-sdk-go and fix import path (#11763 ) * bump hcp-sdk-go and fix import path * bump hcp-sdk-go 0.19.0	4 years ago
Sylvia Moss	c15d3a2e89	remove iteration revocation check (#11624 )	4 years ago
Sylvia Moss	5135d68b5f	do not fail for scheduled revocation (#11619 )	4 years ago
Wilken Rivera	dd525fb814	Update HCP Packer build labels when re-running Packer on an incomplete build (#11584 ) * Update HCP Packer build labels argument Previously when running a partial build on multi-cloud build template it was found that build labels were only being applied at the creation for the partially executed build. Leaving all other completed builds with no HCP Packer build labels. This updates how incomplete builds are loaded from the registry and ensure that any defined hcp_packer_registry.build_labels are assigned to the build before starting an actual Packer build. Related to: #11573 * Add test case for overwriting build labels * Update tests to call CreateInitialBuild for non-existing builds * Rename test case to TestBucket_PopulateIteration * Fix data race in PopulateIteration against mock service Before Change ``` WARNING: DATA RACE Write at 0x00c0005421b0 by goroutine 47: github.com/hashicorp/packer/internal/registry.(MockPackerClientService).PackerServiceCreateBuild() /Users/scrubbed/Development/packer/internal/registry/mock_service.go:173 +0x2b6 github.com/hashicorp/packer/internal/registry.(Client).CreateBuild() /Users/scrubbed/Development/packer/internal/registry/service.go:169 +0x592 github.com/hashicorp/packer/internal/registry.(Bucket).CreateInitialBuildForIteration() /Users/scrubbed/Development/packer/internal/registry/types.bucket.go:107 +0x204 github.com/hashicorp/packer/internal/registry.(Bucket).PopulateIteration.func1() /Users/scrubbed/Development/packer/internal/registry/types.bucket.go:368 +0x14e github.com/hashicorp/packer/internal/registry.(Bucket).PopulateIteration·dwrap·1() /Users/scrubbed/Development/packer/internal/registry/types.bucket.go:376 +0x58 Previous write at 0x00c0005421b0 by goroutine 46: github.com/hashicorp/packer/internal/registry.(MockPackerClientService).PackerServiceCreateBuild() /Users/scrubbed/Development/packer/internal/registry/mock_service.go:173 +0x2b6 github.com/hashicorp/packer/internal/registry.(Client).CreateBuild() /Users/scrubbed/Development/packer/internal/registry/service.go:169 +0x592 github.com/hashicorp/packer/internal/registry.(Bucket).CreateInitialBuildForIteration() /Users/scrubbed/Development/packer/internal/registry/types.bucket.go:107 +0x204 github.com/hashicorp/packer/internal/registry.(Bucket).PopulateIteration.func1() /Users/scrubbed/Development/packer/internal/registry/types.bucket.go:368 +0x14e github.com/hashicorp/packer/internal/registry.(Bucket).PopulateIteration·dwrap·1() /Users/scrubbed/Development/packer/internal/registry/types.bucket.go:376 +0x58 Goroutine 47 (running) created at: github.com/hashicorp/packer/internal/registry.(Bucket).PopulateIteration() /Users/scrubbed/Development/packer/internal/registry/types.bucket.go:362 +0x5c7 github.com/hashicorp/packer/internal/registry.TestBucket_UpdateLabelsForBuild_withMultipleBuilds() /Users/scrubbed/Development/packer/internal/registry/types.bucket_test.go:179 +0xf7 testing.tRunner() /usr/local/go/src/testing/testing.go:1259 +0x22f testing.(T).Run·dwrap·21() /usr/local/go/src/testing/testing.go:1306 +0x47 Goroutine 46 (finished) created at: github.com/hashicorp/packer/internal/registry.(Bucket).PopulateIteration() /Users/scrubbed/Development/packer/internal/registry/types.bucket.go:362 +0x5c7 github.com/hashicorp/packer/internal/registry.TestBucket_UpdateLabelsForBuild_withMultipleBuilds() /Users/scrubbed/Development/packer/internal/registry/types.bucket_test.go:179 +0xf7 testing.tRunner() /usr/local/go/src/testing/testing.go:1259 +0x22f testing.(T).Run·dwrap·21() /usr/local/go/src/testing/testing.go:1306 +0x47 ================== ``` * Add methods for managing builds on an Iteration * Update Mock Service to be a bit more concurrent It is a simple set of changes to make it work for the current testing use cases. If we need to we can move the Called fields to counters or re-architect the mock. This change also fixes a race condition when appending to the Slice of errs when calling PopulateIteration. * Update test case descriptions * Apply suggestions from code review Only append to errs if err is not nil Co-authored-by: Sylvia Moss <moss@hashicorp.com> * Fix linting issues Co-authored-by: Sylvia Moss <moss@hashicorp.com>	4 years ago
Wilken Rivera	119a825296	Update CreateIntialBuildForIteration to initialize each build its own map (#11574 ) * Rename mock bucket test file * Add failing tests for reproducing #11573 ``` === RUN TestBucket_CreateInitialBuildForIteration 2022/02/16 16:22:21 [TRACE] creating initial build for component happycloud.image --- PASS: TestBucket_CreateInitialBuildForIteration (0.00s) === RUN TestBucket_UpdateLabelsForBuild 2022/02/16 16:22:21 [TRACE] creating initial build for component happycloud.image types.bucket_test.go:87: expected the initial build to have an additional build label but thee is no diff: "" --- FAIL: TestBucket_UpdateLabelsForBuild (0.00s) === RUN TestBucket_UpdateLabelsForBuild_withMultipleBuilds 2022/02/16 16:22:21 [TRACE] creating initial build for component happycloud.image 2022/02/16 16:22:21 [TRACE] creating initial build for component happycloud.image2 types.bucket_test.go:125: Comparing component build labels: map[based_off:alpine source_image:another-happycloud-image version:1.7.0] against global build labels: map[based_off:alpine source_image:another-happycloud-image version:1.7.0] types.bucket_test.go:128: expected the initial build to have an additional build label but they are equal types.bucket_test.go:125: Comparing component build labels: map[based_off:alpine source_image:another-happycloud-image version:1.7.0] against global build labels: map[based_off:alpine source_image:another-happycloud-image version:1.7.0] types.bucket_test.go:128: expected the initial build to have an additional build label but they are equal --- FAIL: TestBucket_UpdateLabelsForBuild_withMultipleBuilds (0.00s) FAIL FAIL github.com/hashicorp/packer/internal/registry 0.646s ``` * Update CreateIntialBuildForIteration to initialize each build with a new map Previously upon creating the initial build the same map, which was initialized for the build_labels argument was being shared across all build images. This was causing an issue with labels being backed by the same map for all builds. This change ensures that all builds get their own map with any global build labels copied over during the initial creation. Closes #11573 Passing tests with changes on branch ``` RUN TestBucket_CreateInitialBuildForIteration 2022/02/16 16:37:40 [TRACE] creating initial build for component happycloud.image --- PASS: TestBucket_CreateInitialBuildForIteration (0.00s) === RUN TestBucket_UpdateLabelsForBuild 2022/02/16 16:37:40 [TRACE] creating initial build for component happycloud.image --- PASS: TestBucket_UpdateLabelsForBuild (0.00s) === RUN TestBucket_UpdateLabelsForBuild_withMultipleBuilds 2022/02/16 16:37:40 [TRACE] creating initial build for component happycloud.image 2022/02/16 16:37:40 [TRACE] creating initial build for component happycloud.image2 types.bucket_test.go:125: Comparing component build labels: map[based_off:alpine source_image:another-happycloud-image version:1.7.0] against global build labels: map[based_off:alpine version:1.7.0] types.bucket_test.go:125: Comparing component build labels: map[based_off:alpine source_image:the-original-happycloud-image version:1.7.0] against global build labels: map[based_off:alpine version:1.7.0] --- PASS: TestBucket_UpdateLabelsForBuild_withMultipleBuilds (0.00s) ``` * Handle errors from bucket methods * Update test cases Initialize maps for bucket when calling NewBucketWithIteration	4 years ago
Wilken Rivera	45beb94442	Update test client to use Org/Project ID from client; not environment variables (#11502 ) On a local developer machine there is a good chance the HCP_ORG_ID and HCP_PROJECT_ID are not set since the HCP Packer client can derive this information using the token/secret. This change initializes the location structure with the information obtained from the client and not the envs. Results of tests before change with no HCP_ORG_ID or HCP_PROJECT_ID set ``` par_test.go:185: (packer_service.PackerServiceGetIterationOK)(nil) par_test.go:185: &{0 [] } (models.GrpcGatewayRuntimeError) is not supported by the TextConsumer, can be resolved by supporting TextUnmarshaler interface --- FAIL: TestAcc_PAR_pkr_build (3.94s) FAIL FAIL github.com/hashicorp/packer/internal/registry/acctest 6.446s testing: warning: no tests to run PASS ok github.com/hashicorp/packer/internal/registry/env 0.318s [no tests to run] FAIL ``` Results of tests after change with no HCP_ORG_ID or HCP_PROJECT_ID set ``` 2022/01/18 12:23:48 ui: --> null.example: Published metadata to HCP Packer registry packer/pkr-acctest-temp-2/iterations/01FSQ3EPPY69PMXJSTTFYDPQ5R --- PASS: TestAcc_PAR_pkr_build (3.20s) PASS ok github.com/hashicorp/packer/internal/registry/acctest 5.121s testing: warning: no tests to run PASS ok github.com/hashicorp/packer/internal/registry/env 0.155s [no tests to run] ```	4 years ago
Sylvia Moss	9ffd7ce938	Fail for revoked iterations (#11492 ) * Fail for revoked iterations * check if iteration is revoked on GetIteration * fix fmt	4 years ago
Sylvia Moss	ef41e7506e	add packer user agent to HCP client (#11455 )	4 years ago
Adrien Delorme	3278c1a817	Packer SVC acc tests (#11274 )	4 years ago
Wilken Rivera	06b35c39de	hcp packer registry build labels (#11401 ) * Add basic support for build_labels argument * Update support for build_labels configuration argument * Update complete test-fixture with a build_labels configuration * Add test for deprecated labels argument * Add deprecation for hcp_packer_registry.labels When using the now deprecated labels argument of the new bucket_labels a Warning will be presented to the user. ``` ~> HCP_PACKER_BUILD_FINGERPRINT=356786543567865456789656789 packer build source.pkr.hcl Warning: the argument hcp_packer_registry.labels has been deprecated and will be removed in a future release; please use hcp_packer_registry.bucket_labels ``` When trying to use both bucket_labels and labels together an error is presented to the user. ``` ~> HCP_PACKER_BUILD_FINGERPRINT=ss6786543567865456789656789 packer build source.pkr.hcl Error: hcp_packer_registry.labels and hcp_packer_registry.bucket_labels are mutely exclusive; please use the recommended argument hcp_packer_registry.bucket_labels on source.pkr.hcl line 17: (source code not available) ``` * Update documentation for build_labels * Apply suggestions from code review Co-authored-by: Adrien Delorme <azr@users.noreply.github.com> * Update hcl2template/types.build.hcp_packer_registry.go Co-authored-by: Adrien Delorme <azr@users.noreply.github.com>	4 years ago
Megan Marsh	b763b79d9f	implement contextual variable packer.iteration_id	4 years ago
Megan Marsh	5d1f8e1424	fix mocks	5 years ago
Megan Marsh	a28a865ce5	add ID	5 years ago
Megan Marsh	3c5c61cd2a	add source image id to build from PARtifact	5 years ago
Megan Marsh	06fb597185	import new sdk and fix compatability issues	5 years ago
Wilken Rivera	b10df3abb7	Update HCP Packer registry image extraction, validation, publishing logic for registry capable plugins (#11221 ) * Update handling of registry artifacts This change uses the github.com/hashicorp/packer-plgin-sdk/packer/registryimage for querying Artifact State for HCP Registry Image metadata. To handle the conversion of the RPC response, mapstructure was introduced to conversion state data into an registryimage.Image before publishing to a image bucket. * Update to use registry image from packersdk * Rename internal registry service pkg * Update vendored plugins to latest version * The latest release of Amazon, GoogleCompute, and Azure have support for publishing images to the HCP Packer registry.	5 years ago
Megan Marsh	fd80f8da8c	Add two new data sources -- hcp-packer-iteration and hcp-packer-image. These data sources together will allow users to query hcp_packer for the image_ids they need to use as source images to their builds, with a simple UI and clear outputs. add quick and dirty acceptance test for hcp packer image iteration and hcp packer image data sources PR review linting	5 years ago
Megan Marsh	980deeb684	Implement hcp packer data source. This is a rough copy of the terraform provider's data source, to make beta user experience consistent across both sources	5 years ago
Wilken Rivera	a534c743cf	HCP Packer registry enabled Packer client (#11154 ) * Add working registry pkg * Add custom error for handling the loading of PAR environment variables * Working Publish to Build, with proper error handling for bucket names * Update hcp-sdk-go to use branch instead of mod replace directive * Update Packer build status configuration * Add support for HCP_PACKER_BUILD_FINGERPRINT env * Add support for publishing one or more PARtifacts from a single build * add git shas to this branch * Add ability to set provider name if available * Add working RegistryBuilder type * Add RegistryPostProcessor as wrapper post-processor * When in PAR mode a empty RegistryPostProcessor is added to the end of the post-processor list to publish all final image data. * Add support for updating a build from PAR that is not in a DONE state * Fix a small issue with creation the initial builds for an empty iteration. * Add PAR URL to post-processor display * Implement hcp_packer_registry block (#11168) * Update vendored Amazon plugin to v1.0.1-dev * Fix panic when running a Packer registry build in a clean directory * Remove the publishing of post-processor metadata from the registry post-processor. * Remove metadata add from registry_builder * Update registry builder to skip a build that was found to be DONE Co-authored-by: Megan Marsh <megan@hashicorp.com> Co-authored-by: Sylvia Moss <moss@hashicorp.com>	5 years ago

34 Commits (0eb25b577582ffc0761b79373959fa187173d018)