Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kubelet sync pod throws more detailed events #53857

Merged
merged 1 commit into from
Oct 13, 2017

Conversation

derekwaynecarr
Copy link
Member

@derekwaynecarr derekwaynecarr commented Oct 13, 2017

What this PR does / why we need it:
If there are errors in the kubelet sync pod iteration, it is difficult to determine the problem.

This provides more specific events for errors that occur in the syncPod iteration to help perform problem isolation.

Fixes #53900

Special notes for your reviewer:
It is safer to dispatch more specific events now that we have an event budget per object enforced via #47367

Release note:

kubelet provides more specific events when unable to sync pod

@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Oct 13, 2017
@derekwaynecarr derekwaynecarr added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Oct 13, 2017
@derekwaynecarr
Copy link
Member Author

marking do-not-merge while i perform some tests to ensure this will not introduce problems, but now that the client side event spam merged, it should be safer to provide more granular events to help users understand and report problems.

/cc @sjenning @smarterclayton @eparis

@sjenning
Copy link
Contributor

/lgtm

I don't see how a user could conjure a pod to hit these and create a bunch of spam so that's good.

This will be very helpful 👍

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 13, 2017
@k8s-github-robot k8s-github-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 13, 2017
@derekwaynecarr
Copy link
Member Author

before merging, i need to make sure long tail of events doesnt change from prior measurements, and i would like to verify the errors are not hideous.

@derekwaynecarr
Copy link
Member Author

testing this out, prefer we keep all events from containerruntime.SyncPod local to that operation.

@derekwaynecarr derekwaynecarr force-pushed the sync-event branch 2 times, most recently from 1126891 to be45d31 Compare October 13, 2017 14:14
@k8s-ci-robot k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Oct 13, 2017
@smarterclayton
Copy link
Contributor

smarterclayton commented Oct 13, 2017 via email

@derekwaynecarr derekwaynecarr removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Oct 13, 2017
@derekwaynecarr
Copy link
Member Author

i think this is good to go

@joelsmith
Copy link
Contributor

joelsmith commented Oct 13, 2017

/lgtm
(and testing to see if my lgtm has any effect)

@eparis
Copy link
Contributor

eparis commented Oct 13, 2017

/lgtm
/approve no-issue

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 13, 2017
@k8s-github-robot k8s-github-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 13, 2017
@derekwaynecarr
Copy link
Member Author

added the issue to make bot happy

/approve

@k8s-github-robot
Copy link

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: derekwaynecarr, eparis, joelsmith, sjenning

Associated issue: 53900

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these OWNERS Files:

You can indicate your approval by writing /approve in a comment
You can cancel your approval by writing /approve cancel in a comment

@k8s-github-robot
Copy link

Automatic merge from submit-queue (batch tested with PRs 51840, 53542, 53857, 53831, 53702). If you want to cherry-pick this change to another branch, please follow the instructions here.

@k8s-github-robot k8s-github-robot merged commit e6e23ae into kubernetes:master Oct 13, 2017
openshift-merge-robot added a commit to openshift/origin that referenced this pull request Oct 17, 2017
Automatic merge from submit-queue (batch tested with PRs 16889, 16865).

UPSTREAM: 53857: kubelet sync pod throws more detailed events

Also includes the following upstream dependant PRs:

UPSTREAM: 50350: Wait for container cleanup before deletion
UPSTREAM: 48970: Recreate pod sandbox when the sandbox does not have an IP address.
UPSTREAM: 48589: When faild create pod sandbox record event.
UPSTREAM: 48584: Move event type
UPSTREAM: 47599: Rerun init containers when the pod needs to be restarted

xrefs:
kubernetes/kubernetes#53857
kubernetes/kubernetes#50350
kubernetes/kubernetes#48970
kubernetes/kubernetes#48589
kubernetes/kubernetes#48584
kubernetes/kubernetes#47599
@zhangxiaoyu-zidif
Copy link
Contributor

It is very valuable for troubleshooting.
Great!
+1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants