-
Notifications
You must be signed in to change notification settings - Fork 40.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Priority to Kubernetes API #45610
Conversation
ref/ #22212 |
@kubernetes/api-reviewers |
Needs a release note + xref: the proposal. @kubernetes/sig-api-machinery-feature-requests folks will need to sign off on this one. |
da753ec
to
8ecaab9
Compare
pkg/api/v1/types.go
Outdated
@@ -2412,6 +2412,12 @@ type PodSpec struct { | |||
// +patchMergeKey=IP | |||
// +patchStrategy=merge | |||
HostAliases []HostAlias `json:"hostMappings,omitempty" patchStrategy:"merge" patchMergeKey:"IP" protobuf:"bytes,23,rep,name=hostMappings"` | |||
// If specified, indicates the pod's priority. "system" is a special keyword | |||
// which indicates the highest priority. Any other name must be defined in | |||
// Admission Controller config or pod will be rejected. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add a TODO to update this with the name of the admission controller once you write it.
BTW do you think you can do the admission controller for 1.7? It shouldn't be too hard if you mostly copy one of the existing admission controllers. I don't think any of the existing admission controllers are configured by a ConfigMap, though.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let me call it "Priority" admission controller for now. If we decided that it is not a good name, I will change it.
And, yes, I would like to do admission controller for 1.7 as well if we reach a consensus on how priority should be defined and work.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think the admission controller name should be in the API - that has nothing to do with the end user's perception of the field. A custom admission controller can handle this as well as anything else.
pkg/api/v1/types.go
Outdated
@@ -2529,6 +2535,10 @@ type PodStatus struct { | |||
// More info: https://github.com/kubernetes/kubernetes/blob/master/docs/design/resource-qos.md | |||
// +optional | |||
QOSClass PodQOSClass `json:"qosClass,omitempty" protobuf:"bytes,9,rep,name=qosClass"` | |||
// The priority value which is resolved by the Admission Controller from |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same thing here, update with actual name of the admission controller once you write it.
pkg/api/validation/validation.go
Outdated
@@ -2024,6 +2024,14 @@ func ValidateHostAliases(hostAliases []api.HostAlias, fldPath *field.Path) field | |||
return allErrs | |||
} | |||
|
|||
func ValidatePriorityName(priorityName string, fldPath *field.Path) field.ErrorList { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A couple of comments about validation
(1) You should also do validation on Priority (not just PriorityName) even though it is set by an admission controller (user might write a buggy custom admission controller). BTW do we want to allow negative priorities?
(2) I think you need to modify ValidatePodUpdate() to make it possible to update PriorityName (and Priority)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
-
Actually any valid integer is a valid priority, even negative numbers. I am not sure if we can do any particular validation for "Priority".
-
I was not so sure whether we want to allow updating priority or not. I am going to modify ValidatePodUpdate() so that it allows change of priority with the assumption that all updates go through admission checks.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please don't allow priorityName to be updated unless you are absolutely sure you are going to use it in 1.7. If you aren't, leave it create only. We can never tighten validation.
pkg/api/types.go
Outdated
// If specified, indicates the pod's priority. "system" is a special keyword | ||
// which indicates the highest priority. Any other name must be defined in | ||
// Admission Controller config or pod will be rejected. | ||
// If not specified, the pod priority will be zero. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
zero is the lowest?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, priority could potentially be negative. Zero is default.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will admission controller set it to lowest priority? The concern is that: if not specified, there maybe an undefined priority in system, e.g. zero is not a defined priority.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I updated the proposal a few minutes ago and added more details about the default name resolution. We probably go with resolving unspecified PriorityClassName to zero in the initial version, but in a follow up change, we will add the support to annotate one of the PriorityClasses with a "default" annotation. Once such a priority class exists, the system resolves unspecified PriorityClassName to the value of the default class. If there is no such a class, we will still resolve to zero. Zero is not the lowest possible priority in the system given that any valid integer is a valid priority.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We use CamelCase
for all constants, if this is a "opaque constant field". If this is a reference to a priority class API object, the field name should be PriorityClassName
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@bsalamat , "default" annotation makes sense to me :).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@smarterclayton This is a reference to a priority class object. The field is already named PriorityClassName
.
@k82cn that's the plan.
who is allowed to set and update a pod's priority? |
@liggitt I'd expect the user could set a value, but it may get scaled by an admission controller. |
Is this proposing that the same priorityName would resolve to different numerical priorities for different users? e.g. I (as an end-user) submit a pod with "priorityName: system" and get priority of 0, and you (as a superuser) submit the same pod and get a priority of 1000? Or would I be prevented from creating a pod with |
Not yet, but I did have that intention in the long run.
I hope we would lock down the highest priority to admin only. |
Lock down by rejecting the pod, or by translating the |
@liggitt Priority is meant to be global, not per user or namespace. Users can specify PriorityName is their podspec. |
pkg/api/types.go
Outdated
// If specified, indicates the pod's priority. "SYSTEM" is a special keyword | ||
// which indicates the highest priority. Any other name must be defined by | ||
// creating a PriorityClass object with that name. | ||
// If not specified, the pod priority will will be default or zero if there is |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
s/will will/will
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done. Thanks!
pkg/api/types.go
Outdated
// If specified, indicates the pod's priority. "SYSTEM" is a special keyword | ||
// which indicates the highest priority. Any other name must be defined by | ||
// creating a PriorityClass object with that name. | ||
// If not specified, the pod priority will will be default or zero if there is |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
will be default or zero if there is no default
.
Are we going to support default
now? I found that in the design document, it is will
, just want to confirm if the default
will be available in the first implementation? If not, how about add some words such as will
to identify it is future work?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The plan is to have the default in the first version. We will definitely have the ability to define a "global" default priority. In the future versions, we may add per namespace default as well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Got it, then how about update the sentence default or zero if there is no default
a bit?
Such as when using default one and when setting to zero? Using default or zero if there is no default
may make people confused, as they may not clear when using default and when using zero?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall, it's LGTM :); just some comments on cleanup.
pkg/api/v1/types.go
Outdated
@@ -2520,6 +2520,17 @@ type PodSpec struct { | |||
// +patchMergeKey=IP | |||
// +patchStrategy=merge | |||
HostAliases []HostAlias `json:"hostMappings,omitempty" patchStrategy:"merge" patchMergeKey:"IP" protobuf:"bytes,23,rep,name=hostMappings"` | |||
// If specified, indicates the pod's priority. "SYSTEM" is a special keyword | |||
// which indicates the highest priority. Any other name must be defined by a | |||
// PriorityClass object with that name. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
different comments with api/types.go
, should be by creating a PriorityClass
.
@@ -2272,6 +2276,20 @@ func ValidatePodSpec(spec *api.PodSpec, fldPath *field.Path) field.ErrorList { | |||
allErrs = append(allErrs, ValidateHostAliases(spec.HostAliases, fldPath.Child("hostAliases"))...) | |||
} | |||
|
|||
if len(spec.PriorityClassName) > 0 { | |||
if !utilfeature.DefaultFeatureGate.Enabled(features.PodPriority) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
personally, I'd like the following if/else:
if utilfeature.DefaultFeatureGate.Enabled(features.PodPriority) {
...
} else {
...
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
in my version of the code, the feature gate is not checked when priority class is not specified. Also, the condition "if len(spec.PriorityClassName) > 0" should be repeated in both if and else blocks of your version.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes; my version is for sometimes removing this feature gate (just remove the else
block is OK); both are ok to me :).
btw, please run |
@bsalamat how about run |
@gyliu513 I have already. Did you see files which are not updated? |
0ccf575
to
94f9147
Compare
@bsalamat I found that But if you use |
/test pull-kubernetes-unit |
/test pull-kubernetes-unit |
@bsalamat: The following tests failed, say
Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here. |
/lgtm |
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: bsalamat, davidopp Assign the PR to them by writing No associated issue. Update pull-request body to add a reference to an issue, or get approval with The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these OWNERS Files:
You can indicate your approval by writing |
/test pull-kubernetes-unit |
@smarterclayton Could you please take another look? |
Automatic merge from submit-queue (batch tested with PRs 45610, 47628) |
What this PR does / why we need it: This is the first in a series of PRs to add priority to Kubernetes API. Subsequent PRs will add priority name resolution to admission controller.
Release note: