packer

Commit Graph

Author	SHA1	Message	Date
Wilken Rivera	dd525fb814	Update HCP Packer build labels when re-running Packer on an incomplete build (#11584 ) * Update HCP Packer build labels argument Previously when running a partial build on multi-cloud build template it was found that build labels were only being applied at the creation for the partially executed build. Leaving all other completed builds with no HCP Packer build labels. This updates how incomplete builds are loaded from the registry and ensure that any defined hcp_packer_registry.build_labels are assigned to the build before starting an actual Packer build. Related to: #11573 * Add test case for overwriting build labels * Update tests to call CreateInitialBuild for non-existing builds * Rename test case to TestBucket_PopulateIteration * Fix data race in PopulateIteration against mock service Before Change ``` WARNING: DATA RACE Write at 0x00c0005421b0 by goroutine 47: github.com/hashicorp/packer/internal/registry.(MockPackerClientService).PackerServiceCreateBuild() /Users/scrubbed/Development/packer/internal/registry/mock_service.go:173 +0x2b6 github.com/hashicorp/packer/internal/registry.(Client).CreateBuild() /Users/scrubbed/Development/packer/internal/registry/service.go:169 +0x592 github.com/hashicorp/packer/internal/registry.(Bucket).CreateInitialBuildForIteration() /Users/scrubbed/Development/packer/internal/registry/types.bucket.go:107 +0x204 github.com/hashicorp/packer/internal/registry.(Bucket).PopulateIteration.func1() /Users/scrubbed/Development/packer/internal/registry/types.bucket.go:368 +0x14e github.com/hashicorp/packer/internal/registry.(Bucket).PopulateIteration·dwrap·1() /Users/scrubbed/Development/packer/internal/registry/types.bucket.go:376 +0x58 Previous write at 0x00c0005421b0 by goroutine 46: github.com/hashicorp/packer/internal/registry.(MockPackerClientService).PackerServiceCreateBuild() /Users/scrubbed/Development/packer/internal/registry/mock_service.go:173 +0x2b6 github.com/hashicorp/packer/internal/registry.(Client).CreateBuild() /Users/scrubbed/Development/packer/internal/registry/service.go:169 +0x592 github.com/hashicorp/packer/internal/registry.(Bucket).CreateInitialBuildForIteration() /Users/scrubbed/Development/packer/internal/registry/types.bucket.go:107 +0x204 github.com/hashicorp/packer/internal/registry.(Bucket).PopulateIteration.func1() /Users/scrubbed/Development/packer/internal/registry/types.bucket.go:368 +0x14e github.com/hashicorp/packer/internal/registry.(Bucket).PopulateIteration·dwrap·1() /Users/scrubbed/Development/packer/internal/registry/types.bucket.go:376 +0x58 Goroutine 47 (running) created at: github.com/hashicorp/packer/internal/registry.(Bucket).PopulateIteration() /Users/scrubbed/Development/packer/internal/registry/types.bucket.go:362 +0x5c7 github.com/hashicorp/packer/internal/registry.TestBucket_UpdateLabelsForBuild_withMultipleBuilds() /Users/scrubbed/Development/packer/internal/registry/types.bucket_test.go:179 +0xf7 testing.tRunner() /usr/local/go/src/testing/testing.go:1259 +0x22f testing.(T).Run·dwrap·21() /usr/local/go/src/testing/testing.go:1306 +0x47 Goroutine 46 (finished) created at: github.com/hashicorp/packer/internal/registry.(Bucket).PopulateIteration() /Users/scrubbed/Development/packer/internal/registry/types.bucket.go:362 +0x5c7 github.com/hashicorp/packer/internal/registry.TestBucket_UpdateLabelsForBuild_withMultipleBuilds() /Users/scrubbed/Development/packer/internal/registry/types.bucket_test.go:179 +0xf7 testing.tRunner() /usr/local/go/src/testing/testing.go:1259 +0x22f testing.(T).Run·dwrap·21() /usr/local/go/src/testing/testing.go:1306 +0x47 ================== ``` * Add methods for managing builds on an Iteration * Update Mock Service to be a bit more concurrent It is a simple set of changes to make it work for the current testing use cases. If we need to we can move the Called fields to counters or re-architect the mock. This change also fixes a race condition when appending to the Slice of errs when calling PopulateIteration. * Update test case descriptions * Apply suggestions from code review Only append to errs if err is not nil Co-authored-by: Sylvia Moss <moss@hashicorp.com> * Fix linting issues Co-authored-by: Sylvia Moss <moss@hashicorp.com>	4 years ago
Wilken Rivera	119a825296	Update CreateIntialBuildForIteration to initialize each build its own map (#11574 ) * Rename mock bucket test file * Add failing tests for reproducing #11573 ``` === RUN TestBucket_CreateInitialBuildForIteration 2022/02/16 16:22:21 [TRACE] creating initial build for component happycloud.image --- PASS: TestBucket_CreateInitialBuildForIteration (0.00s) === RUN TestBucket_UpdateLabelsForBuild 2022/02/16 16:22:21 [TRACE] creating initial build for component happycloud.image types.bucket_test.go:87: expected the initial build to have an additional build label but thee is no diff: "" --- FAIL: TestBucket_UpdateLabelsForBuild (0.00s) === RUN TestBucket_UpdateLabelsForBuild_withMultipleBuilds 2022/02/16 16:22:21 [TRACE] creating initial build for component happycloud.image 2022/02/16 16:22:21 [TRACE] creating initial build for component happycloud.image2 types.bucket_test.go:125: Comparing component build labels: map[based_off:alpine source_image:another-happycloud-image version:1.7.0] against global build labels: map[based_off:alpine source_image:another-happycloud-image version:1.7.0] types.bucket_test.go:128: expected the initial build to have an additional build label but they are equal types.bucket_test.go:125: Comparing component build labels: map[based_off:alpine source_image:another-happycloud-image version:1.7.0] against global build labels: map[based_off:alpine source_image:another-happycloud-image version:1.7.0] types.bucket_test.go:128: expected the initial build to have an additional build label but they are equal --- FAIL: TestBucket_UpdateLabelsForBuild_withMultipleBuilds (0.00s) FAIL FAIL github.com/hashicorp/packer/internal/registry 0.646s ``` * Update CreateIntialBuildForIteration to initialize each build with a new map Previously upon creating the initial build the same map, which was initialized for the build_labels argument was being shared across all build images. This was causing an issue with labels being backed by the same map for all builds. This change ensures that all builds get their own map with any global build labels copied over during the initial creation. Closes #11573 Passing tests with changes on branch ``` RUN TestBucket_CreateInitialBuildForIteration 2022/02/16 16:37:40 [TRACE] creating initial build for component happycloud.image --- PASS: TestBucket_CreateInitialBuildForIteration (0.00s) === RUN TestBucket_UpdateLabelsForBuild 2022/02/16 16:37:40 [TRACE] creating initial build for component happycloud.image --- PASS: TestBucket_UpdateLabelsForBuild (0.00s) === RUN TestBucket_UpdateLabelsForBuild_withMultipleBuilds 2022/02/16 16:37:40 [TRACE] creating initial build for component happycloud.image 2022/02/16 16:37:40 [TRACE] creating initial build for component happycloud.image2 types.bucket_test.go:125: Comparing component build labels: map[based_off:alpine source_image:another-happycloud-image version:1.7.0] against global build labels: map[based_off:alpine version:1.7.0] types.bucket_test.go:125: Comparing component build labels: map[based_off:alpine source_image:the-original-happycloud-image version:1.7.0] against global build labels: map[based_off:alpine version:1.7.0] --- PASS: TestBucket_UpdateLabelsForBuild_withMultipleBuilds (0.00s) ``` * Handle errors from bucket methods * Update test cases Initialize maps for bucket when calling NewBucketWithIteration	4 years ago
Wilken Rivera	45beb94442	Update test client to use Org/Project ID from client; not environment variables (#11502 ) On a local developer machine there is a good chance the HCP_ORG_ID and HCP_PROJECT_ID are not set since the HCP Packer client can derive this information using the token/secret. This change initializes the location structure with the information obtained from the client and not the envs. Results of tests before change with no HCP_ORG_ID or HCP_PROJECT_ID set ``` par_test.go:185: (packer_service.PackerServiceGetIterationOK)(nil) par_test.go:185: &{0 [] } (models.GrpcGatewayRuntimeError) is not supported by the TextConsumer, can be resolved by supporting TextUnmarshaler interface --- FAIL: TestAcc_PAR_pkr_build (3.94s) FAIL FAIL github.com/hashicorp/packer/internal/registry/acctest 6.446s testing: warning: no tests to run PASS ok github.com/hashicorp/packer/internal/registry/env 0.318s [no tests to run] FAIL ``` Results of tests after change with no HCP_ORG_ID or HCP_PROJECT_ID set ``` 2022/01/18 12:23:48 ui: --> null.example: Published metadata to HCP Packer registry packer/pkr-acctest-temp-2/iterations/01FSQ3EPPY69PMXJSTTFYDPQ5R --- PASS: TestAcc_PAR_pkr_build (3.20s) PASS ok github.com/hashicorp/packer/internal/registry/acctest 5.121s testing: warning: no tests to run PASS ok github.com/hashicorp/packer/internal/registry/env 0.155s [no tests to run] ```	4 years ago
Sylvia Moss	9ffd7ce938	Fail for revoked iterations (#11492 ) * Fail for revoked iterations * check if iteration is revoked on GetIteration * fix fmt	4 years ago
Sylvia Moss	ef41e7506e	add packer user agent to HCP client (#11455 )	4 years ago
Adrien Delorme	3278c1a817	Packer SVC acc tests (#11274 )	4 years ago
Wilken Rivera	06b35c39de	hcp packer registry build labels (#11401 ) * Add basic support for build_labels argument * Update support for build_labels configuration argument * Update complete test-fixture with a build_labels configuration * Add test for deprecated labels argument * Add deprecation for hcp_packer_registry.labels When using the now deprecated labels argument of the new bucket_labels a Warning will be presented to the user. ``` ~> HCP_PACKER_BUILD_FINGERPRINT=356786543567865456789656789 packer build source.pkr.hcl Warning: the argument hcp_packer_registry.labels has been deprecated and will be removed in a future release; please use hcp_packer_registry.bucket_labels ``` When trying to use both bucket_labels and labels together an error is presented to the user. ``` ~> HCP_PACKER_BUILD_FINGERPRINT=ss6786543567865456789656789 packer build source.pkr.hcl Error: hcp_packer_registry.labels and hcp_packer_registry.bucket_labels are mutely exclusive; please use the recommended argument hcp_packer_registry.bucket_labels on source.pkr.hcl line 17: (source code not available) ``` * Update documentation for build_labels * Apply suggestions from code review Co-authored-by: Adrien Delorme <azr@users.noreply.github.com> * Update hcl2template/types.build.hcp_packer_registry.go Co-authored-by: Adrien Delorme <azr@users.noreply.github.com>	4 years ago
Megan Marsh	b763b79d9f	implement contextual variable packer.iteration_id	5 years ago
Megan Marsh	5d1f8e1424	fix mocks	5 years ago
Megan Marsh	a28a865ce5	add ID	5 years ago
Megan Marsh	3c5c61cd2a	add source image id to build from PARtifact	5 years ago
Megan Marsh	06fb597185	import new sdk and fix compatability issues	5 years ago
Wilken Rivera	b10df3abb7	Update HCP Packer registry image extraction, validation, publishing logic for registry capable plugins (#11221 ) * Update handling of registry artifacts This change uses the github.com/hashicorp/packer-plgin-sdk/packer/registryimage for querying Artifact State for HCP Registry Image metadata. To handle the conversion of the RPC response, mapstructure was introduced to conversion state data into an registryimage.Image before publishing to a image bucket. * Update to use registry image from packersdk * Rename internal registry service pkg * Update vendored plugins to latest version * The latest release of Amazon, GoogleCompute, and Azure have support for publishing images to the HCP Packer registry.	5 years ago

13 Commits (bdaa03be315102400a43bbb1e6c52240d3f26bce)