Proposal

• KEDA metrics server knows how to get multiple metrics from various sources.
• Currently, the desired size computation works for each scaler individually and the final value is decided according to MAX.
• In real-world - sometimes you need to make decisions with more complex logic.

I think that a "Composed Scaler" that allows to put in a simple formula that references multiple other metrics - can be super useful.

This would make Keda a powerful tool not only for collecting external metrics but also for making smart decisions based on multiple conditions.

(discussed original here)

Use-Case

Here's a real-world problem: I have a backend process that consumes "write" tasks from a queue and persists them to DB. My DB has a single writer node that is CPU-bound. A task may implicit varying load on the CPU (depending on the content of the write).

I have 2 relevant metrics to determine the replica count:

q := queue-length
cpu := DB-writer CPU utilization

Composed metric formula: ceil((1-cpu) * q) can promise that replicas would grow if the queue is high but avoid growing if CPU is loaded. Right now the only option for me to do it is to create my own service that exposes that metric.

Anything else?

In the original discussion, we started talking about how the ScaledObject spec would look like in the custom logic case.

First I suggested this:

triggers:
  - type: composed
    metadata:
      metrics:
        q_length: 
          type: aws-cloudwatch
          metadata:
            namespace: AWS/SQS
            ... # SQS length
        cpu_util:
          type: aws-cloudwatch
          metadata:
            namespace: AWS/Neptune
            ... # neptune writer CPU
      formula: "ceil((1 - $cpu_util) * $q_length)"
      targetMetricValue: "5"
      minMetricValue: "0"

then @zroubalik suggested this (better IMO):

scaledObject.spec:
  ...
  mutlipleTriggersCalculation:    ### we should find a better name for this probably
      formula: "ceil((1 - $cpu_util) * $q_length)"
      targetMetricValue: "5"
      minMetricValue: "0"
  triggers:
    - type: aws-cloudwatch
       name: q_length     ### name added here
       metadata:
            namespace: AWS/SQS
            ... # SQS length
    - type: aws-cloudwatch
       name: cpu_util       ### name added here
       metadata:
            namespace: AWS/Neptune
            ... # neptune writer CPU

Finally, it's valuable to read @tomkerkhove 's comment about it.