Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

scheduler volumebinding: leverage PreFilterResult for Bound local PersistentVolumes #109876

Closed
yibozhuang opened this issue May 6, 2022 · 9 comments · Fixed by #109877
Closed

scheduler volumebinding: leverage PreFilterResult for Bound local PersistentVolumes #109876

yibozhuang opened this issue May 6, 2022 · 9 comments · Fixed by #109877
Assignees
@yibozhuang
Labels
needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. sig/scheduling Categorizes an issue or PR as relevant to SIG Scheduling. sig/storage Categorizes an issue or PR as relevant to SIG Storage.

Comments

@yibozhuang
Copy link
Contributor

What would you like to be added?

Similar to #109475

Today, in the scheduler volumebinding plugin, the bound PV node affinity check is done during Filter stage which means that all nodes are considered.

For the case of bound local PV, the PV node affinity spec looks like

nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - node1

Today, if the node which the local PV with affinity tied to is tainted and/or out of cpu/memory, etc. we still run through Filter and consider all possible nodes which for a large cluster of n nodes, (n-1)/n nodes are irrelevant.

So, under such case, we see something along the lines of

  Type     Reason            Age   From               Message
  ----     ------            ----  ----               -------
  Warning  FailedScheduling  3s    default-scheduler  0/100 nodes are available: 1 node(s) had untolerated taint {key-fake-0: value-fake-0}, 1 node(s) had untolerated taint {key-fake-10: value-fake-10}, 1 node(s) had untolerated taint {key-fake-11: value-fake-11}, 1 node(s) had untolerated taint {key-fake-12: value-fake-12}, 1 node(s) had untolerated taint {key-fake-13: value-fake-13}, 1 node(s) had untolerated taint {key-fake-14: value-fake-14}, 1 node(s) had untolerated taint {key-fake-15: value-fake-15}, 1 node(s) had untolerated taint {key-fake-16: value-fake-16}, 1 node(s) had untolerated taint {key-fake-17: value-fake-17}, 1 node(s) had untolerated taint {key-fake-18: value-fake-18}, 1 node(s) had untolerated taint {key-fake-19: value-fake-19}, 1 node(s) had untolerated taint {key-fake-1: value-fake-1}, 1 node(s) had untolerated taint {key-fake-20: value-fake-20}, 1 node(s) had untolerated taint {key-fake-21: value-fake-21}, 1 node(s) had untolerated taint {key-fake-22: value-fake-22}, 1 node(s) had untolerated taint {key-fake-23: value-fake-23}, 1 nod ...

because all nodes are considered so it is difficult for the user to identify which node is the one which the local PV node affinity matches that was not schedulable as the message here is mixed with all the irrelevant nodes that the pod cannot land on anyways due to PV node affinity mismatch.

I propose we leverage the new PreFilterResult added in #108648 to reduce list of nodes down to only eligible ones based on nodes that satisfy bound claim affinity check for local PVs.

Why is this needed?

This will ensure that scheduling stages beyond PreFilter for pod having bound claims to local PVs would only consider eligible nodes so less number of nodes need to be considered in subsequent scheduling stages.

This will also make the message clear to the user when scheduling fails that the errors are for only eligible nodes rather than user having to figure out which nodes are relevant and which are not.
@k8s-ci-robot k8s-ci-robot added the needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. label May 6, 2022
@k8s-ci-robot
Copy link
Contributor

@yibozhuang: This issue is currently awaiting triage.

If a SIG or subproject determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added the needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. label May 6, 2022
@yibozhuang
Copy link
Contributor Author

/sig scheduling
/sig storage

@k8s-ci-robot k8s-ci-robot added sig/scheduling Categorizes an issue or PR as relevant to SIG Scheduling. sig/storage Categorizes an issue or PR as relevant to SIG Storage. and removed needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels May 6, 2022
@yibozhuang
Copy link
Contributor Author

/assign

@yibozhuang yibozhuang mentioned this issue May 7, 2022
Merged
@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle stale
  • Mark this issue or PR as rotten with /lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Aug 5, 2022
@yibozhuang yibozhuang mentioned this issue Aug 5, 2022
Closed
9 tasks
@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

@k8s-ci-robot k8s-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Sep 4, 2022
@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Reopen this issue with /reopen
  • Mark this issue as fresh with /remove-lifecycle rotten
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

@k8s-ci-robot k8s-ci-robot closed this as not planned Won't fix, can't repro, duplicate, stale Oct 4, 2022
@k8s-ci-robot
Copy link
Contributor

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Reopen this issue with /reopen
  • Mark this issue as fresh with /remove-lifecycle rotten
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@yibozhuang
Copy link
Contributor Author

/reopen
/remove-lifecycle rotten

@k8s-ci-robot
Copy link
Contributor

@yibozhuang: Reopened this issue.

In response to this:

/reopen
/remove-lifecycle rotten

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot reopened this Oct 29, 2022
@k8s-ci-robot k8s-ci-robot removed the lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. label Oct 29, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@yibozhuang yibozhuang

Labels
needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. sig/scheduling Categorizes an issue or PR as relevant to SIG Scheduling. sig/storage Categorizes an issue or PR as relevant to SIG Storage.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

scheduler volumebinding: leverage PreFilterResult for bound local PVs yibozhuang/kubernetes
3 participants
@yibozhuang @k8s-ci-robot @k8s-triage-robot