Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Periodically update pod status from kubelet. #5555

Merged
merged 1 commit into from
Mar 18, 2015

Conversation

fgrzadkowski
Copy link
Contributor

This PR is based on #5205 with two bug fixes:

  • deadlock in syncPodStatus
  • in master.go some go-routines were referencing variables that went out of scope

This fixes #4561

@fgrzadkowski
Copy link
Contributor Author

@wojtek-t

@vmarmol
Copy link
Contributor

vmarmol commented Mar 17, 2015

LGTM @fgrzadkowski, but it'll need a rebase :P

// Create the list of pods.
podFullNames := make(map[types.UID]string)
func() {
kl.podLock.Lock()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are not writing to kl.pods, so you can use GetPods() (which does RLock()) to get a copy of kl.pods.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@yujuhong
Copy link
Contributor

@fgrzadkowski, you'll need to rebase because my PR (#5401) was just merged. I left some in-line comments to help you with that.

One nit. Now that we have two locks podStatusLock and podLock, which are used in multiple places, it'd be nice to point out in the comment that some use patterns may lead to deadlocks. E.g. if you grab the locks in the order of podStatusLock and podLock, it'd cause deadlock.

@yujuhong
Copy link
Contributor

In fact, now that kubelet post pod statuses directly, we should be able to simply the lock usage. I'll write up another PR after this is merged, so don't worry about it.

@yujuhong
Copy link
Contributor

As mentioned in #5538, you should add new integration tests for pod status, and make sure that all integration tests pass.

@yujuhong
Copy link
Contributor

Thanks for rebasing! The code LGTM. I'd like to see the integration test be fixed to reflect how we sync the pod status, but that could also be addressed in another PR. I'll leave it to @vmarmol to decide.

Did the PR pass e2e?

@fgrzadkowski
Copy link
Contributor Author

Re integration tests - I'd prefer to add them in a separate PR (I already assigned myself to that issue I filed)

Re e2e tests - they pass. I checked this morning and everything worked as expected.

@vmarmol
Copy link
Contributor

vmarmol commented Mar 18, 2015

Another PR for the tests SGTM (we know where to find you and collect on our integration test ;) ).

Thanks for taking a look @yujuhong!

vmarmol added a commit that referenced this pull request Mar 18, 2015
Periodically update pod status from kubelet.
@vmarmol vmarmol merged commit 9586b39 into kubernetes:master Mar 18, 2015
@fgrzadkowski
Copy link
Contributor Author

I've just noticed - that's a pretty PR number to have :D 5555

@vmarmol
Copy link
Contributor

vmarmol commented Mar 18, 2015

Haha you're right I didn't even notice! A good PR for that number.


// UpdateStatus takes the name of the pod and the new status. Returns the server's representation of the pod, and an error, if it occurs.
func (c *pods) UpdateStatus(name string, newStatus *api.PodStatus) (result *api.Pod, err error) {
result = &api.Pod{}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is the right implementation - a client should be required to send in an api.Pod to do this.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason for this is that you want to be able to force clients to deal with specifying a resource version, and this doesn't allow it (it's basically an overwrite).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Going to submit a pull that deals with this.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SGTM, thanks @smarterclayton

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@smarterclayton, could you hold off on the PR to fix this? UpdateStatus needs to update the status of the mirror pods too. We'll need keep the mirror pods objects and use them here. I am working on a pod manager refactoring which will take care of this.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wondered why the mirror status test is failing. I'll wait until after your change to merge #5738. Note my pull request changes SimpleRunKubelet so we can dial down the sync selector in the integration test.

----- Original Message -----

@@ -113,3 +114,15 @@ func (c *pods) Watch(label labels.Selector, field
fields.Selector, resourceVersi
func (c *pods) Bind(binding *api.Binding) error {
return
c.r.Post().Namespace(c.ns).Resource("pods").Name(binding.Name).SubResource("binding").Body(binding).Do().Error()
}
+
+// UpdateStatus takes the name of the pod and the new status. Returns the
server's representation of the pod, and an error, if it occurs.
+func (c *pods) UpdateStatus(name string, newStatus *api.PodStatus) (result
*api.Pod, err error) {

  • result = &api.Pod{}

@smarterclayton, could you hold off on the PR to fix this? UpdateStatus needs
to update the status of the mirror pods too. We'll need keep the mirror pods
objects and use them here. I am working on a pod manager refactoring which
will take care of this.


Reply to this email directly or view it on GitHub:
https://github.com/GoogleCloudPlatform/kubernetes/pull/5555/files#r26880558

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Kubelet to POST pod status to apiserver
5 participants