-
Notifications
You must be signed in to change notification settings - Fork 40k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
make docker-checker more robust #23662
Conversation
partly addresses #23556 |
Labelling this PR as size/S |
GCE e2e build/test passed for commit 2e8d4cababa76b2d38a10f1bda8d17c753a22501. |
2e8d4ca
to
4507197
Compare
GCE e2e build/test passed for commit 450719719152b500a1831d2df042fd5232a014dc. |
@@ -35,7 +46,7 @@ echo "waiting 30s for startup" | |||
sleep 30 | |||
|
|||
while true; do | |||
if ! sudo timeout 10 docker version > /dev/null; then | |||
if ! sudo timeout 10 docker ps > /dev/null; then |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
10 second is not enough for docker ps
at heavy load based on the past experiences (we changed it to docker ps
and then back to docker version
once). We'd need a longer timeout. cc @dchen1107
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, why are we using sudo.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, please increase the timeout value to 30 second here. sudo can be removed.
4507197
to
736c02c
Compare
PTAL |
GCE e2e build/test passed for commit 736c02c. |
LGTM |
Manually merging to reduce flakiness |
The cherrypick process is defined here. Hoping to get this PR merged today, but the details are here. You need to run the cherry pick tool yourself to get these onto the branch. @bgrant0607 will then approve them for merge. |
Every cherrypick needs a release note. |
This PR has no description and no reference to a P0 issue. |
Fixes the "docker flakes on nodes and masters" part of #23556. |
Internal bugref: 27875273 |
@bgrant0607 #23556 is p0 issue. I have a small summary related to the issue at: #23556 (comment) |
…23662-upstream-release-1.2 Automated cherry pick of #23662
Commit found in the "release-1.2" branch appears to be this PR. Removing the "cherrypick-candidate" label. If this is an error find help to get your PR picked. |
…-pick-of-#23662-upstream-release-1.2 Automated cherry pick of kubernetes#23662
…-pick-of-#23662-upstream-release-1.2 Automated cherry pick of kubernetes#23662
Partly addresses: #23556 by forcefully kill problematic docker processes.