fix(provider/ecs): fetch all images for ECR repos with >100 images #2948
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ECR repositories may contain many images, but the `ListImages` operation will retrieve at most 100 at a time [1]. For repositories with more than 100 images this used to cause pipelines depending on that repository to randomly fail when you start getting over the limit (as the desired image would not be on the first page of results). This commit builds upon the `ListImages` operation and the `nextToken` value to fetch all images available in the repository. And only images matching the desired tag/digest are sent to `DescribeImages`. Some tests were slightly modified to include an empty list instead of `null` in the `imageIds` field as: 1. that's what the `ListImages` returns in case there are no images in the repository 2. the code now uses the results from the `ListImages` operation to filter the images instead of the results of `DescribeImages` as previously done [1]: https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/ecr/model/ListImagesRequest.html#getMaxResults--
|
@clareliguori @BrunoCarrier can either of you LGTM this PR? |
radubutnaru
pushed a commit
to radubutnaru/clouddriver
that referenced
this pull request
Sep 19, 2018
…pinnaker#2948) ECR repositories may contain many images, but the `ListImages` operation will retrieve at most 100 at a time [1]. For repositories with more than 100 images this used to cause pipelines depending on that repository to randomly fail when you start getting over the limit (as the desired image would not be on the first page of results). This commit builds upon the `ListImages` operation and the `nextToken` value to fetch all images available in the repository. And only images matching the desired tag/digest are sent to `DescribeImages`. Some tests were slightly modified to include an empty list instead of `null` in the `imageIds` field as: 1. that's what the `ListImages` returns in case there are no images in the repository 2. the code now uses the results from the `ListImages` operation to filter the images instead of the results of `DescribeImages` as previously done [1]: https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/ecr/model/ListImagesRequest.html#getMaxResults--
clanesf
pushed a commit
to clanesf/clouddriver
that referenced
this pull request
Dec 8, 2018
…pinnaker#2948) ECR repositories may contain many images, but the `ListImages` operation will retrieve at most 100 at a time [1]. For repositories with more than 100 images this used to cause pipelines depending on that repository to randomly fail when you start getting over the limit (as the desired image would not be on the first page of results). This commit builds upon the `ListImages` operation and the `nextToken` value to fetch all images available in the repository. And only images matching the desired tag/digest are sent to `DescribeImages`. Some tests were slightly modified to include an empty list instead of `null` in the `imageIds` field as: 1. that's what the `ListImages` returns in case there are no images in the repository 2. the code now uses the results from the `ListImages` operation to filter the images instead of the results of `DescribeImages` as previously done [1]: https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/ecr/model/ListImagesRequest.html#getMaxResults--
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
ECR repositories may contain many images, but the
ListImagesoperation will retrieve at most 100 at a time [1]. For repositories with
more than 100 images this used to cause pipelines depending on that
repository to randomly fail when you start getting over the limit (as
the desired image would not be on the first page of results).
This commit builds upon the
ListImagesoperation and thenextTokenvalue to fetch all images available in the repository. And only images
matching the desired tag/digest will be sent to
DescribeImages.Some tests were slightly modified to include an empty list instead of
nullin theimageIdsfield as:ListImagesreturns in case there are no images in the repositoryListImagesoperation to filter the images instead of the results ofDescribeImagesas previously done[1]: https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/ecr/model/ListImagesRequest.html#getMaxResults--