/metrics (Prometheus) endpoint always returning a value of 0 #3554
Closed
Description
Report
Running Keda v2.8.0 the /metrics endpoint is always returning a value of zero. I have both AWS SQS Queue and Cron scalers.
My deployment is using the kedacore/keda v2.8.1 helm chart.
If I deploy the helm chart version v2.7.2, things work as expected.
Expected Behavior
Correct values be reported to the /metrics endpoint.
Actual Behavior
Values to the /metrics endpoint are always zero.
Steps to Reproduce the Problem
- Configure a ScaledObject with a Cron Scaler, ensure the time selection is in range, running Keda v2.7.1, and deploy eg:
triggers:
- metadata:
desiredReplicas: "3"
end: 0 23 * * *
start: 0 1 * * *
timezone: Australia/Brisbane
type: cron
- Look at the
/metricsendpoint, eg:
curl keda-operator-metrics-apiserver.keda.svc.cluster.local:9022/metrics
# HELP keda_metrics_adapter_scaler_metrics_value Metric Value used for HPA
# TYPE keda_metrics_adapter_scaler_metrics_value gauge
keda_metrics_adapter_scaler_metrics_value{metric="s0-cron-australia-brisbane-01xxx-023xxx",namespace="coreapi",scaledObject="coreapi-queue",scaler="cronScaler",scalerIndex="0"} 3
- Update Keda to version v2.8.0, and run the
curlagain:
curl keda-operator-metrics-apiserver.keda.svc.cluster.local:9022/metrics
# HELP keda_metrics_adapter_scaler_metrics_value Metric Value used for HPA
# TYPE keda_metrics_adapter_scaler_metrics_value gauge
keda_metrics_adapter_scaler_metrics_value{metric="s0-cron-australia-brisbane-01xxx-023xxx",namespace="coreapi",scaledObject="coreapi-queue",scaler="cronScaler",scalerIndex="0"} 0
Note how the value is now zero at the end of the last line.
Note: These outputs have been trimmed for clarity.
Logs from KEDA operator
No response
KEDA Version
2.8.0
Kubernetes Version
1.22
Platform
Amazon Web Services
Scaler Details
Cron and AWS SQS
Anything else?
Scaling still works as expected.
Of note, if running a query via kubectl the value is correct, eg:
kubectl get --raw "/apis/external.metrics.k8s.io/v1beta1/namespaces/coreapi/s0-cron-australia-brisbane-01xxx-023xxx?labelSelector=scaledobject.keda.sh%2Fname%3Dcoreapi-queue"
{"kind":"ExternalMetricValueList","apiVersion":"external.metrics.k8s.io/v1beta1","metadata":{},"items":[{"metricName":"s0-cron-australia-brisbane-01xxx-023xxx","metricLabels":null,"timestamp":"2022-08-15T05:17:43Z","value":"3"}]}
Metadata
Assignees
Labels
Type
Projects
Status
Done