This would be a good project for someone who wants to get started in kubelet.

You could e2e test this as follows. Create a pod in the API that requests the same hostPort as is already used by the file-based cadvisor container. The pod should fail and there should be an appropriate reason.

@dchen1107 we just talked about this.

in future we might use this to fail due to out of resources, or other setup errors.

Might need some way to poison the node from future scheduling decisions - tricky.

+1 to this, do want
On Thu, Feb 19, 2015 at 5:42 PM Tim Hockin notifications@github.com wrote:

Might need some way to poison the node from future scheduling decisions -
tricky.

—
Reply to this email directly or view it on GitHub
#4623 (comment)
.

@thockin
there are two cases:

1. two schedulers racing to start a pod on the same node
   • one pod will get the the node first,
   • the second will be kicked off by the kubelet
   • the rc for the second pod makes a new pod
   • there should not be a race the second time because all the schedulers know about the first pod now.
2. file pod versus apiserver pod on the same node.
   • once Node should sync back to master with allocated Pods through file source #4090 is fixed, this won't happen because the schedulers will see the file pod hostPorts.

Oh, I thought this was getting rid of HostPort conflict checking entirely?
Is that ALSO happening somewhere else?

On Thu, Feb 19, 2015 at 4:30 PM, Eric Tune notifications@github.com wrote:

@thockin https://github.com/thockin
there are two cases:

1. two schedulers racing to start a pod on the same node
   • one pod will get the the node first,
   • the second will be kicked off by the kubelet
   • the rc for the second pod makes a new pod
   • there should not be a race the second time because all the
     schedulers know about the first pod now.
     1. file pod versus apiserver pod on the same node.
   • once Node should sync back to master with allocated Pods through file source #4090
     Node should sync back to master with allocated Pods through file source #4090 is
     fixed, this won't happen because the schedulers will see the file pod
     hostPorts.

Reply to this email directly or view it on GitHub
#4623 (comment)
.

This is needed in order to eliminate BoundPods.

kubelet now sorts the pods before filtering, and also records an event on host port conflict. To actually reject such pods, we will need to kubelet to post pod status back to apiserver (#4561)

The scheduler still checks. This handles parallel scheduler races and file
based pods.
On Feb 27, 2015 4:15 PM, "Yu-Ju Hong" notifications@github.com wrote:

kubelet now sorts the pods before filtering, and also records an event on
host port conflict. To actually reject such pods, we will need to
kubelet to post pod status back to apiserver (#4561
#4561)

—
Reply to this email directly or view it on GitHub
#4623 (comment)
.

There should be no parallel scheduler races, we use resource versions and
optimistic concurrency to ensure correctness.

Brendan
On Feb 27, 2015 4:55 PM, "Eric Tune" notifications@github.com wrote:

The scheduler still checks. This handles parallel scheduler races and file
based pods.
On Feb 27, 2015 4:15 PM, "Yu-Ju Hong" notifications@github.com wrote:

kubelet now sorts the pods before filtering, and also records an event on
host port conflict. To actually reject such pods, we will need to
kubelet to post pod status back to apiserver (#4561
#4561)

—
Reply to this email directly or view it on GitHub
<
#4623 (comment)

.

—
Reply to this email directly or view it on GitHub
#4623 (comment)
.

Re. atomicity -- I don't buy that we need it. I commented here: #2483 (comment)

@erictune, do you mean that we should still set the pod status to "failed" so that when the scheduler check, it can get the accurate pod status back? I had thought about it, but the pod statuses are computed on the fly (by checking the containers) as of now. Without the ability to directly set the status, we need to cache this information somewhere. Since there are people touching the same code/file, I'd rather wait until #4561 is fixed.

@brendandburns
perhaps parallel was the wrong word, but a scheduler race appears possible. the cache of pods it uses is not updated atomically with binding pods. It appears the scheduler could:

1. successfully do a /binding of pod A with hostPort N to host H
2. read the cache and see A not bound
3. consider binding B with hostPort N to host H, and not see a conflict.
4. successfully do a /binding of pod B with hostPort N to host H
5. later, the cache notices that A is bound to H

@erictune and I discussed offline, and we decided to create a map to store the rejected pod->status, so that the failed status will be correctly reports when polling happens. By doing this, this issue is no longer blocked by any other issues.

I will write up a PR to implement this.