Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[scheduler] When the hostname and nodename of a node do not match, ensure that pods carrying PVs with nodeAffinity are scheduled correctly. #125398

Merged
merged 2 commits into from
Sep 23, 2024
Merged

[scheduler] When the hostname and nodename of a node do not match, ensure that pods carrying PVs with nodeAffinity are scheduled correctly. #125398

merged 2 commits into from
Sep 23, 2024

Conversation

AxeZhan
Copy link
Member

@AxeZhan AxeZhan commented Jun 8, 2024 •
edited
Loading

What type of PR is this?

/kind bug

What this PR does / why we need it:

Currently, the scheduler assumes that the hostname and nodename of a node are the same when evaluating nodeAffinity for local PVs.

kubernetes/pkg/volume/util/util.go

Line 582 in 6b73ccc

if matchExpr.Key == v1.LabelHostname && matchExpr.Operator == v1.NodeSelectorOpIn {

Node name and hostname are distinct and are definitively known to be different in some clusters.
And this will make the pod scheduled to a wrong node or failed scheduling.

Which issue(s) this PR fixes:

Part of #125336

Special notes for your reviewer:

Does this PR introduce a user-facing change?

Fix a bug when the hostname label of a node does not match the node name, pods bound to a PV with nodeAffinity using the hostname may be scheduled to the wrong node or experience scheduling failures.

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

@k8s-ci-robot k8s-ci-robot added release-note-none Denotes a PR that doesn't merit a release note. kind/bug Categorizes issue or PR as related to a bug. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Jun 8, 2024
@k8s-ci-robot
Copy link
Contributor

This issue is currently awaiting triage.

If a SIG or subproject determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot added needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. sig/scheduling Categorizes an issue or PR as relevant to SIG Scheduling. sig/storage Categorizes an issue or PR as relevant to SIG Storage. and removed do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Jun 8, 2024
@k8s-ci-robot k8s-ci-robot requested review from carlory and gnufied June 8, 2024 09:38
@AxeZhan AxeZhan mentioned this pull request Jun 8, 2024
Open
@carlory
Copy link
Member

carlory commented Jun 11, 2024

@AxeZhan can you add a release note?
/lgtm
/hold
/assign @msau42 @Huang-Wei

@k8s-ci-robot k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jun 11, 2024
@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 11, 2024
@k8s-ci-robot
Copy link
Contributor

LGTM label has been added.

Git tree hash: 27406546fb22f50719ba4ec9418ab5c4fef8855e

@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed release-note-none Denotes a PR that doesn't merit a release note. labels Jun 11, 2024
@msau42
Copy link
Member

msau42 commented Jul 19, 2024

/approve

@carlory
Copy link
Member

carlory commented Jul 20, 2024

/hold cancel
/assign @Huang-Wei

@k8s-ci-robot k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jul 20, 2024
@AxeZhan
Copy link
Member Author

AxeZhan commented Aug 9, 2024

/cc @alculquicondor

@domenicbozzuto domenicbozzuto mentioned this pull request Sep 6, 2024
Merged
alculquicondor
alculquicondor reviewed Sep 23, 2024
View reviewed changes
Copy link
Member

@alculquicondor alculquicondor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/approve

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: alculquicondor, AxeZhan, msau42

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@alculquicondor
Copy link
Member

Please prepare cherry-picks for all supported versions.

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 23, 2024
@gnufied
Copy link
Member

gnufied commented Sep 23, 2024 •
edited
Loading

@AxeZhan let me know if you are interested in making the necessary(or have time) cherry picks. This bug is also important for some of our clusters, so we are trying to get backporting fixes expedited if possible.

@k8s-ci-robot k8s-ci-robot merged commit 211d67a into kubernetes:master Sep 23, 2024
18 checks passed
@k8s-ci-robot k8s-ci-robot added this to the v1.32 milestone Sep 23, 2024
@AxeZhan
Copy link
Member Author

AxeZhan commented Sep 24, 2024

@gnufied Hi gnufied!
Yes, I will prepare the cherry-picks.
Normally, we'll cherry-pick the bugfix to n-3 releases. But it seems this bug is introduced in V1.27, so I will try cherry-pick this pr to release V1.27-V1.31.
May I know what's the version of your cluster? I assume it is included?

This was referenced Sep 24, 2024
Merged
Merged
Merged
Closed
Closed
This was referenced Sep 24, 2024
Merged
Merged
Merged
Merged
k8s-ci-robot added a commit that referenced this pull request Nov 12, 2024
@k8s-ci-robot
Merge pull request #127584 from AxeZhan/automated-cherry-pick-of-#125…
dd898b8 
…398-upstream-release-1.31

Automated cherry pick of #125398: [scheduler] When the hostname and nodename of a node do not match, ensure that pods carrying PVs with nodeAffinity are scheduled correctly.
k8s-ci-robot added a commit that referenced this pull request Nov 12, 2024
@k8s-ci-robot
Merge pull request #127586 from AxeZhan/automated-cherry-pick-of-#125…
4d73f6e 
…398-upstream-release-1.29

Manually cherry pick of #125398: [scheduler] When the hostname and nodename of a node do not match, ensure that pods carrying PVs with nodeAffinity are scheduled correctly.
k8s-ci-robot added a commit that referenced this pull request Nov 12, 2024
@k8s-ci-robot
Merge pull request #127585 from AxeZhan/automated-cherry-pick-of-#125…
627e10a 
…398-upstream-release-1.30

Automated cherry pick of #125398: [scheduler] When the hostname and nodename of a node do not match, ensure that pods carrying PVs with nodeAffinity are scheduled correctly.
This was referenced Nov 24, 2024
Open
Open
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@alculquicondor alculquicondor alculquicondor left review comments

@carlory carlory Awaiting requested review from carlory

@gnufied gnufied Awaiting requested review from gnufied

Assignees

@alculquicondor alculquicondor

@Huang-Wei Huang-Wei

@msau42 msau42

@carlory carlory

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. lgtm "Looks good to me", indicates that a PR is ready to be merged. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/scheduling Categorizes an issue or PR as relevant to SIG Scheduling. sig/storage Categorizes an issue or PR as relevant to SIG Storage. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.
Projects
None yet
Milestone
v1.32
Development

Successfully merging this pull request may close these issues.
7 participants
@AxeZhan @k8s-ci-robot @carlory @msau42 @alculquicondor @gnufied @Huang-Wei