@polvi as mentioned last week, DaemonSet can be used to start a process on every machine in a cluster, thus simplifying setup of things like monitoring and local-storage-daemons. Documented at https://github.com/kubernetes/kubernetes/blob/ba89c98fc7e892e816751a95ae0ee22f4266ffa5/docs/admin/daemons.md

That is a good list of changes that you have thought of.

The best way to find changes that we haven't thought of yet is to use the feature ourselves.
I propose it doesn't graduate from beta until we are using it as part of the kube-up process.
I'll file an issue to that effect.

Filed #15324

I disagree. If we think it is feasible for KubeUp, then it should definitely be in beta. If its useful for us, its useful for customers.

You have said that before it is beta, we need to be pretty sure we aren't going to change anything. I don't think that we get to that level of sureness without actually using the interface ourselves.

Also, see description in #15324 about how to stage the work.

I might be misquoting @brendandburns, but we have definitely talked about a bar for beta stability internally, and I don't see how we reach that bar without using the feature in earnest.

cc @kubernetes/rh-cluster-infra

DaemonSet won't be ready in 1.1.

In addition to the other issues mentioned, we need to think about interplay between DaemonSet updates and node upgrades.

We need to revert and fix #17318 before graduation.

We need to fix #16967 (via #12744) before graduation.

Either integration into the node controller in order to integrate with node lifecycle, such as creation before other pods can be scheduled and implicit forgiveness behavior (#1574), or use of initializers and finalizers (#3585) to achieve the same

Proposal for how to do this (depends on #18263 being implemented)

1. Admission controller intercepts Node object creation and adds a taint "node being configured"
2. DaemonSet controller automatically adds toleration for "node being configured" taint to all pods it creates
3. When all DaemonSets that should schedule onto the node have started, the taint is removed

(3) is the only tricky step. One idea is that we could have NodeController iterate through all DaemonSet objects, figure out which ones should result in a pod on the given node (due to label selectors not all daemons are supposed to run on all nodes), and when all of those are running remove the taint.

This doesn't require moving DaemonSet controller into NodeController.

Of course this is all related to #3885 (comment)

For the implicit forgiveness part, the idea is that all DaemonSet-managed pods implicitly forgive the node going unreachable, but not any other type of node deletion or cascading delete event.

Correct?

For the implicit forgiveness part, the idea is that all DaemonSet-managed pods implicitly forgive the node going unreachable, but not any other type of node deletion or cascading delete event.

Yeah, this is just addressing what happens when NodeController deletes pods because the NodeReady became False or Unknown. But did you have something specific in mind when you said "other type of node deletion or cascading delete event" ?

I can't think of a cascading delete that would result in deleting a node, but if the admin manually deletes a node with kubectl, we'd still delete the pod, right?

Also to be clear, I'm not planning to do this by exposing the Forgiveness concept anywhere in the API (it's not well-enough formed yet), but just by hardcoding this behavior into the NodeController when the pod in question is DaemonSet-managed.

I feel like there were two kubectl and one server side. Server side one here: #19627

@bgrant0607 @erictune can me make sure that before going to GA someone (fluentd or node-problem-dectector) will actually use this for a quarter or two, to make sure that for example upgrades are handled properly?

@piosz I agree. I don't want to rush any of the controller APIs to GA. I wish Job hadn't been rushed to GA.

/sig apps

cc @kow3ns

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

/remove-lifecycle stale

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

DaemonSet is v1.
cc @kow3ns @janetkuo