Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(cli): unhandled nextToken returned by listImagesCommand in garbage collector for ECR #32679

Merged
merged 2 commits into from
Dec 29, 2024

Conversation

sakurai-ryo
Copy link
Contributor

@sakurai-ryo sakurai-ryo commented Dec 28, 2024

Issue # (if applicable)

Closes #32498

Reason for this change

When listImagesCommand returns nextToken in the readRepoInBatches function, nextToken is not passed as an argument for the subsequent listImagesCommand execution, causing listImagesCommand to continue executing.
https://github.com/aws/aws-cdk/blob/v2.173.4/packages/aws-cdk/lib/api/garbage-collection/garbage-collector.ts#L621

According to the listImagesCommand documentation, if maxResults is not specified, a maximum of 100 images will be returned, so this bug requires at least 100 images in the asset repository.
https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/Package/-aws-sdk-client-ecr/Interface/ListImagesCommandInput/

Reproduction Steps

The following bash script and Dockerfile saved locally and executed, will push 120 container images to the asset repository.

#!/usr/bin/env bash

set -eu

ACCOUNT_ID="your account id"
REGION="your region"
REPO_NAME="cdk-hnb659fds-container-assets-${ACCOUNT_ID}-${REGION}"
IMAGE_NAME="test-image"
AWS_PROFILE="your AWS profile"

echo "Logging in to ECR..."
aws ecr get-login-password --region "${REGION}" --profile "${AWS_PROFILE}" \
| docker login --username AWS --password-stdin "${ACCOUNT_ID}.dkr.ecr.${REGION}.amazonaws.com"

for i in $(seq 1 120); do
  hash=$(head -c 32 /dev/urandom | xxd -p -c 64)
  echo "Building and pushing image with tag: ${hash}"
  touch "${i}.txt"

  docker build \
    --build-arg BUILD_NO="${i}" \
    -t "${IMAGE_NAME}:${i}" \
    .

  docker tag "${IMAGE_NAME}:${i}" \
    "${ACCOUNT_ID}.dkr.ecr.${REGION}.amazonaws.com/${REPO_NAME}:${hash}"

  docker push \
    "${ACCOUNT_ID}.dkr.ecr.${REGION}.amazonaws.com/${REPO_NAME}:${hash}"

  rm "${i}.txt"

  sleep 0.01
done

echo "Done!"
FROM scratch

ARG BUILD_NO
ENV BUILD_NO=${BUILD_NO}

COPY ${BUILD_NO}.txt /

You can reproduce this bug by running the following command after the images have been pushed.

$ cdk gc aws://{account id}/{region} --type ecr --unstable=gc --created-buffer-days 0 --action full --confirm=true

Description of changes

Fix the problem of correctly handling nextToken when executing listImagesCommand in the readRepoInBatches function.

Describe any new or updated permissions being added

Nothing.

Description of how you validated changes

Verifying that this bug has been fixed using the CLI integration tests is difficult, so only unit tests are added.

Checklist


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

@sakurai-ryo sakurai-ryo requested a review from a team as a code owner December 28, 2024 09:47
@aws-cdk-automation aws-cdk-automation requested a review from a team December 28, 2024 09:47
@github-actions github-actions bot added bug This issue is a bug. effort/small Small work item – less than a day of effort p2 star-contributor [Pilot] contributed between 25-49 PRs to the CDK labels Dec 28, 2024
Copy link
Collaborator

@aws-cdk-automation aws-cdk-automation left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pull request linter has failed. See the aws-cdk-automation comment below for failure reasons. If you believe this pull request should receive an exemption, please comment and provide a justification.

A comment requesting an exemption should contain the text Exemption Request. Additionally, if clarification is needed add Clarification Request to a comment.

@aws-cdk-automation aws-cdk-automation added the pr/needs-cli-test-run This PR needs CLI tests run against it. label Dec 28, 2024
Copy link

codecov bot commented Dec 28, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 80.69%. Comparing base (318e725) to head (6fa2328).
Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main   #32679   +/-   ##
=======================================
  Coverage   80.69%   80.69%           
=======================================
  Files         108      108           
  Lines        7019     7019           
  Branches     1299     1299           
=======================================
  Hits         5664     5664           
  Misses       1175     1175           
  Partials      180      180           
Flag Coverage Δ
suite.unit 80.69% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
packages/aws-cdk 80.69% <ø> (ø)

@sakurai-ryo
Copy link
Contributor Author

Exemption Request:

I'm waiting for the CLI integration test to run.

@aws-cdk-automation aws-cdk-automation added pr-linter/exemption-requested The contributor has requested an exemption to the PR Linter feedback. pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. labels Dec 28, 2024
@kaizencc kaizencc added pr-linter/exempt-integ-test The PR linter will not require integ test changes and removed pr-linter/exemption-requested The contributor has requested an exemption to the PR Linter feedback. labels Dec 29, 2024
@aws-cdk-automation
Copy link
Collaborator

➡️ PR build request submitted to test-main-pipeline ⬅️

A maintainer must now check the pipeline and add the pr-linter/cli-integ-tested label once the pipeline succeeds.

Copy link
Contributor

@kaizencc kaizencc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Crap. Thank you for the repro steps and the fix. Super helpful. I am running this through the test pipeline and then will approve.

@aws-cdk-automation aws-cdk-automation removed the pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. label Dec 29, 2024
@kaizencc kaizencc added pr-linter/cli-integ-tested Assert that any CLI changes have been integ tested and removed pr/needs-cli-test-run This PR needs CLI tests run against it. labels Dec 29, 2024
@aws-cdk-automation aws-cdk-automation dismissed their stale review December 29, 2024 19:43

✅ Updated pull request passes all PRLinter validations. Dismissing previous PRLinter review.

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
  • Commit ID: 6fa2328
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Copy link
Contributor

mergify bot commented Dec 29, 2024

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@mergify mergify bot merged commit d9346bc into aws:main Dec 29, 2024
24 of 25 checks passed
Copy link

Comments on closed issues and PRs are hard for our team to see.
If you need help, please open a new issue that references this one.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Dec 29, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug This issue is a bug. effort/small Small work item – less than a day of effort p2 pr-linter/cli-integ-tested Assert that any CLI changes have been integ tested pr-linter/exempt-integ-test The PR linter will not require integ test changes star-contributor [Pilot] contributed between 25-49 PRs to the CDK
Projects
None yet
3 participants