-
Notifications
You must be signed in to change notification settings - Fork 40k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Int overflow in hpa causing incorrect replica count #126892
Comments
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the The Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
/sig autoscaling |
/assign |
The Kubernetes project currently lacks enough contributors to adequately respond to all issues. This bot triages un-triaged issues according to the following rules:
You can:
Please send feedback to sig-contributor-experience at kubernetes/community. /lifecycle stale |
As PR is still open, |
What happened?
The setup:
I am using keda with the prometheus scaler. The query I am using, returns the lag in the message queue i am using, and the threshold is set to
0.1
.What happened:
The lag was increasing for a long time, and the replica count reached the max setting as expected. Everything was running fine for some time. When the lag value reached
214,748,364
hpa decided to reduce the replicas from the max limit to1
.What I think is the problem:
When the lag passes
214,748,364
, the calculation here divides by the threshold0.1
and it passes the max int32 value. causing hpa to scale to the minimum value, 1.It also seems like a lot of other places in this file cast a 64 bit float to a 32 bit int. Should there maybe be a check everywhere this is done?
What did you expect to happen?
I expected the replica count to stay at the max value. Or alternatively, get an error that we have reached the max value for an external metric value
How can we reproduce it (as minimally and precisely as possible)?
Use an external metric, and set it above
214,748,364
with a threshold of0.1
.Anything else we need to know?
No response
Kubernetes version
1.29
Cloud provider
OS version
Install tools
Container runtime (CRI) and version (if applicable)
Related plugins (CNI, CSI, ...) and versions (if applicable)
The text was updated successfully, but these errors were encountered: