PodDisruptionBudget with an empty selector {}
being honored incorrectly #119731
Closed
Description
What happened?
PodDisruptionBudget
with an empty selector {}
selects all the pods as expected.
But, during the drain
/eviction
, the pdb is not being honored correctly i.e. all the pods are getting evicted.
This issue is only observed with the empty selector and works as expected if a valid label selector is provided that selects all the pods in a namespace.
What did you expect to happen?
When an empty selector {}
is provided in the PodDisruptionBudget
's spec, the created PodDisruptionBudget
needs to be honored by not evicting all the pods during the drain/eviction
api call
How can we reproduce it (as minimally and precisely as possible)?
- Create a simple busybox deployment with 3 replicas on a local cluster (or any cluster)
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: busybox
name: busybox
namespace: default
spec:
replicas: 3
selector:
matchLabels:
app: busybox
strategy:
rollingUpdate:
maxSurge: 25%
maxUnavailable: 25%
type: RollingUpdate
template:
metadata:
labels:
app: busybox
spec:
containers:
- image: busybox
name: busybox
command: ["sleep", "1000"]
dnsPolicy: ClusterFirst
restartPolicy: Always
terminationGracePeriodSeconds: 30
- Create a pdb object with an
empty
selector{}
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: pdb
spec:
minAvailable: 100%
selector: {}
- Check the status of the created pdb object that has the
expectedPods
observed to be 3 (which is expected)
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
creationTimestamp: "2023-08-02T10:33:29Z"
generation: 1
name: pdb
namespace: default
resourceVersion: "495"
uid: 448b956e-4d2f-4d25-b2c4-202d68d3061d
spec:
minAvailable: 100%
selector: {}
status:
conditions:
- lastTransitionTime: "2023-08-02T10:33:33Z"
message: ""
observedGeneration: 1
reason: InsufficientPods
status: "False"
type: DisruptionAllowed
currentHealthy: 3
desiredHealthy: 3
disruptionsAllowed: 0
expectedPods: 3
observedGeneration: 1
- Call the
drain
api to see the pdb not being honored and all the pods being evicted
$kubectl drain 127.0.0.1 --delete-emptydir-data --dry-run=server
node/127.0.0.1 cordoned (server dry run)
evicting pod kube-system/coredns-6994cbdc59-rqs5s (server dry run)
evicting pod default/busybox-6b5dd5dcf8-2hmjf (server dry run)
evicting pod default/busybox-6b5dd5dcf8-x5dn6 (server dry run)
evicting pod default/busybox-6b5dd5dcf8-sgpdz (server dry run)
node/127.0.0.1 drained (server dry run)
- Delete the above pdb object and create a new one with the label selector
app: busybox
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: pdb
spec:
minAvailable: 100%
selector: {}
[svanka@svanka pdb]$ cat pdb.yaml
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: pdb-with-label-selector
spec:
minAvailable: 100%
selector:
matchLabels:
app: busybox
- Observe the expected status of the pdb object same as the old one with the
expectedPods
set to 3
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
creationTimestamp: "2023-08-02T10:38:00Z"
generation: 1
name: pdb-with-label-selector
namespace: default
resourceVersion: "579"
uid: 5788a978-4a5c-40e3-9775-88e855abe38c
spec:
minAvailable: 100%
selector:
matchLabels:
app: busybox
status:
conditions:
- lastTransitionTime: "2023-08-02T10:38:00Z"
message: ""
observedGeneration: 1
reason: InsufficientPods
status: "False"
type: DisruptionAllowed
currentHealthy: 3
desiredHealthy: 3
disruptionsAllowed: 0
expectedPods: 3
observedGeneration: 1
- Call the
drain
api to observe the pdb being honored as expected
$kubectl drain 127.0.0.1 --delete-emptydir-data --dry-run=server
node/127.0.0.1 cordoned (server dry run)
evicting pod kube-system/coredns-6994cbdc59-rqs5s (server dry run)
evicting pod default/busybox-6b5dd5dcf8-sgpdz (server dry run)
evicting pod default/busybox-6b5dd5dcf8-x5dn6 (server dry run)
evicting pod default/busybox-6b5dd5dcf8-2hmjf (server dry run)
error when evicting pods/"busybox-6b5dd5dcf8-2hmjf" -n "default" (will retry after 5s): Cannot evict pod as it would violate the pod's disruption budget.
error when evicting pods/"busybox-6b5dd5dcf8-sgpdz" -n "default" (will retry after 5s): Cannot evict pod as it would violate the pod's disruption budget.
error when evicting pods/"busybox-6b5dd5dcf8-x5dn6" -n "default" (will retry after 5s): Cannot evict pod as it would violate the pod's disruption budget.
evicting pod default/busybox-6b5dd5dcf8-sgpdz (server dry run)
evicting pod default/busybox-6b5dd5dcf8-x5dn6 (server dry run)
evicting pod default/busybox-6b5dd5dcf8-2hmjf (server dry run)
error when evicting pods/"busybox-6b5dd5dcf8-sgpdz" -n "default" (will retry after 5s): Cannot evict pod as it would violate the pod's disruption budget.
error when evicting pods/"busybox-6b5dd5dcf8-x5dn6" -n "default" (will retry after 5s): Cannot evict pod as it would violate the pod's disruption budget.
error when evicting pods/"busybox-6b5dd5dcf8-2hmjf" -n "default" (will retry after 5s): Cannot evict pod as it would violate the pod's disruption budget.
Anything else we need to know?
No response
Kubernetes version
$ kubectl version
WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short. Use --output=yaml|json to get the full version.
Client Version: version.Info{Major:"1", Minor:"26", GitVersion:"v1.26.1", GitCommit:"4e136762ec0ea7216f04665eb983ca30633b1f9f", GitTreeState:"clean", BuildDate:"2023-06-10T04:19:41Z", GoVersion:"go1.20.3", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v4.5.7
Server Version: version.Info{Major:"1", Minor:"28+", GitVersion:"v1.28.0-rc.0.46+3888b341f12c6e", GitCommit:"3888b341f12c6e7241428950394e61d8cc26d5f9", GitTreeState:"clean", BuildDate:"2023-08-02T11:02:24Z", GoVersion:"go1.20.6", Compiler:"gc", Platform:"linux/amd64"}
Cloud provider
local cluster - N/A
OS version
# On Linux:
$ cat /etc/os-release
NAME="Red Hat Enterprise Linux"
VERSION="8.3 (Ootpa)"
ID="rhel"
ID_LIKE="fedora"
VERSION_ID="8.3"
PLATFORM_ID="platform:el8"
PRETTY_NAME="Red Hat Enterprise Linux 8.3 (Ootpa)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:8.3:GA"
HOME_URL="https://www.redhat.com/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
REDHAT_BUGZILLA_PRODUCT_VERSION=8.3
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="8.3"
$ uname -a
Linux svanka.remote.csb 4.18.0-193.28.1.el8_2.x86_64 #1 SMP Fri Oct 16 13:38:49 EDT 2020 x86_64 x86_64 x86_64 GNU/Linux
# On Windows:
C:\> wmic os get Caption, Version, BuildNumber, OSArchitecture
# paste output here
Install tools
Container runtime (CRI) and version (if applicable)
CRI-O