Cache metric names provided by KEDA Metrics Server #2156
Description
Proposal
Kubernetes API Server is asking for all available metrics that are provided by the KEDA Metrics Server. Internally we are doing a round trip across all ScaledObjects to compile the list of used metrics:
Lines 139 to 140 in a223a0e
This code has been on my radar for quite some time. We should cache the list of metrics in the metrics server and only update it if a ScaledObject has been added/updated/removed. But it is a little bit tricky because metrics server doesn't watch ScaledObjects (Operator does that in the reconcile loop) so it doesn't have the knowledge.
I have been thinking about this for some time and the best solution is probably one of these:
- Add a simple controller to Metrics Server, it will watch ScaledObjects and populate metrics names cache appropriately.
- Operator could maintain a list of all used metrics and update this list as ScaledObjects are being added/removed/updated. Then the Operator dumps this list of metrics into a ConfigMap which will be mounted to metrics server. Metrics Server will populate the metrics cache from this ConfigMap (will check the timestamp of the mounted CM in order to have up to date cache).
I am inclined toward solution 1.
Use-Case
It should improve perfomance, because this will reduce the number of calls to k8s API Server.