/cc @rjnagal

This looks reasonable to me. I feel fairly strongly that it should be a separate resource, not part of the replication controller.

Also this doesn't talk much about how the scaler would be implemented, is there one autoscaler implementation? Or are they pluggable.

It's also a little weird that the threshold interface says "should scale" rather than "ShouldScaleUp" or "ShouldScaleDown"

I didn't understand the AutoScaleThreshold struct. Clearly you are trying to specify a policy for scaling up (and down?) but I couldn't figure out how to interpret the policy you gave in the example.

Yes, I think this was a pretty big miss on my part: boundary definitions. The AutoScaleThreshold definition should defining upper and lower bounds and perhaps the scale increment.

 type AutoScaleThreshold struct {
      //scale by this increment
      Inc int
      //example: RequestsPerSecond StatisticType = "requestPerSecond"
      Type StatisticType
      //after this duration
      Duration time.Duration
      //when this value is passed
      Value float
      //with this comparison (greater than, less than to support falling above or below a value)
      Comp string
  } 

I think this would allow folks to define upper and lower bounds as well as what increment they'd like to scale at and allow the AutoScaleThresholdInterface to stick with a single ShouldScale method. The resize verb would be passed the Inc value which could be positive or negative to scale up or down.

Thoughts, suggestions?

Just to illustrate what I'm thinking a little better, this is what a threshold definition would look like for an app wanting to scale up when rps > 50 and scale down when rps < 25:

 "thresholds": [
    {
        "id": "myapp-rps-up",
        "kind": "AutoScaleThreshold",
        "type": "requestPerSecond", 
        "durationVal": 30,
        "durationInterval": "seconds",
        "value": 50,
        "comp": "greater",
        "inc": 1
    },
    {
        "id": "myapp-rps-down",
        "kind": "AutoScaleThreshold",
        "type": "requestPerSecond", 
        "durationVal": 30,
        "durationInterval": "seconds",
        "value": 25,
        "comp": "less",
        "inc": -1
    },
],

@pweil- How do we support these threshold when resources other than network come into play? If we had to repeat the threshold definition per resource, it might become too unwieldy. From the spec, it sounds like we do plan to extend the same spec for multiple resources.

@pweil- suggest calling out the use-case where the minimum bound is zero and whether scaling to zero is different from other scaling events

@pweil- If scaling to zero is different, how is that expressed in the API, etc

Scaling from zero was briefly touched upon here: #484 (comment)

We have an idling design proposal coming soon for discussion

Would be better not to shorten field names - use increment instead of inc, etc

I agree with @brendandburns that the auto-scaler should be a separate object, not an annotation.

The object should also specify its own selector. We anticipate using separate replication controllers for each deployed software version (e.g., each release track, each new deployment). One could make the auto-scaler resize just one replication controller, but it needs to monitor all pods of the service in order to understand its overall capacity and load.

The deployment strategy as mocked in https://gist.github.com/smarterclayton/0862906d4aea8c8466e6 is different, as it applies to a single replication controller (or 2, if you include the one spun down as well), and can be thought of as notes guiding a temporary process transforming the resource, and is equally likely to be implemented in a client as by a service.

We don't want the auto-scaler to fight whatever deployment mechanism is orchestrating a rolling update, so either the auto-scaler needs to indirect through an object/component aware of the deployment process, or the deployment mechanism needs to manipulate the auto-scaler (e.g., to change the auto-scaler to resize a different replication controller). I currently prefer the latter, since I think the benefit of increased flexibility in deployment orchestration outweighs the cost of manipulating one more object during the process.

@rjnagal When you say "repeat the threshold definition per resource" are you referring to new statistics that come in to play or new scalable resources (like Jobs)?

re: new statistics: I like the idea of not strongly typing the Threshold.Type field to allow them to be added on the fly so long as what is specified in Threshold.Type is translatable to the underlying storage so it can be queried in a uniform way. This is far more flexible than my original suggestion. I suspect that with new statistics come new values to check (and possibly new scale increments) so I'm not sure we can avoid repeating a threshold....maybe a simpler way to define them is in order?

re: new scalable resources: I suppose there are two approaches on the way folks will handle this

1. 1 scaler to rule them all: we can make the auto-scaler be able to handle many selectors with a set of thresholds for each selector so you can have a single auto-scaler in your namespace controlling many other objects in the namespace
2. you can define a new auto-scaler for new resources as you introduce them (leaving the pre-existing scalers running). One of the assumptions in the design was to allow you to make new scalers without affecting old ones.

Not sure that addresses your concerns though.

Updated based on the feedback so far.

TODO: define the statistics selector as mentioned in #2863 (comment) for the use case of how a scaler interacts with in-flight deployment mechanisms by monitoring the entire set of resources.

Agree with @bgrant0607 that the deployment mechanism should update the autoscaler to point at a new replication controller.

When referring to the statistics selectors ( @pmorie @bgrant0607 ) I feel like they belong on the thresholds since each threshold can be a different statistic. Alternatively, we could set the standard that a replication controller deals with a single service only and set the selector at a higher level but I think that is too restrictive. Agree?

Here is how the deployment use case shakes out in my head:

1. replicationController1 = existing deployment
2. replicationController2 = new deployment that is scaling down replicationController1 while it scales itself up
3. autoScaler = targets replicationController1
4. service1 = selector targets pods in both replication controllers

So long as the stats gathering mechanism is writing consolidated stats under the service the scaler will be monitoring both replication controllers (but only targeting replicationController1). At the end of the deployment we then need to update or make a new auto-scaler that targets replicationController2.

We also have the ability to temporarily suspend auto-scaling during deployment via the Enabled field in the Spec.

On Dec 17, 2014, at 9:44 AM, Paul notifications@github.com wrote:

When referring to the statistics selectors ( @pmorie @bgrant0607 ) I feel like they belong on the thresholds since each threshold can be a different statistic. Alternatively, we could set the standard that a replication controller deals with a single service only and set the selector at a higher level but I think that is too restrictive. Agree?

Services will span multiple replication controllers always. Can't use that assumption.
Here is how the deployment use case shakes out in my head:

replicationController1 = existing deployment
replicationController2 = new deployment that is scaling down replicationController1 while it scales itself up
autoScaler = targets replicationController1
service1 = selector targets pods in both replication controllers
So long as the stats gathering mechanism is writing consolidated stats under the service the scaler will be monitoring both replication controllers (but only targeting replicationController1). At the end of the deployment we then need to update or make a new auto-scaler that targets replicationController2.

We also have the ability to temporarily suspend auto-scaling during deployment via the Enabled field in the Spec.

—
Reply to this email directly or view it on GitHub.

Good catch...

Typo, should've read "an auto-scaler deals with a single service" but I still think that that is the wrong approach because it isn't as flexible. I'll update the document to put the statistics selector on the threshold.

Ultimately it's more flexible if the autoscaler accepts it may be controlling a portion of the total traffic (set thresholds higher / lower relative to proportion) or deals with a number of replication controllers proportionally

On Dec 17, 2014, at 12:07 PM, Paul notifications@github.com wrote:

Good catch...

Typo, should've read "an auto-scaler deals with a single service" but I still think that that is the wrong approach because it isn't as flexible. I'll update the document to put the statistics selector on the threshold.

—
Reply to this email directly or view it on GitHub.

Let me know when this is updated with comments.

@smarterclayton sorry for the latency. Will do, should be able to get back to this this week

I agree with @smarterclayton: Services span multiple replication controllers, not all of which will be resizable by the auto-scaler. For instance, users generally don't want to scale sets of canary instances -- they want a precise number.

@pweil- As far as when the auto-scaler is changed to target a new replication controller during a rolling update, I think multiple approaches are possible. One could update it at the end, as you propose, or in the middle of the rolling update, or after confidence the new release is achieved. Or, if it were capable of resizing multiple replication controllers either proportionally or asymmetrically (shrink one, grow the other), at the beginning.

Disabling the auto-scaler entirely during a rolling update can cause unnecessary disruption and risk. Instead, one could disable size decreases only. Or, we could permit full simultaneous auto-scaling. A simple approach would be for the rolling update to simply grow the new replication controller, and rely on the auto-scaler to shrink the original. Depending on the time scales of both processes, it could also make sense for the rolling update to decrease the number of instances of the original controller even while the auto-scaler resizes it, but the rolling updater would then need to update the both the original and new controllers before the auto-scaler would react to the transitionally changed number of instances.

If the auto-scaler changed the number of instances during the rolling update, how would the updater know what the "right number" of instances was? Rather than recording the desired number of instances in the new controller at the beginning, when updating both the original and new controllers, the updater would also record how many instances it had deducted or added in annotations. The updater doesn't want to change the total number of instances desired by the auto-scaler. The annotations would enable the updater to ensure that it had updated both controllers equally, even in the case of failures.

If the auto-scaler had grown the original controller during the update, the updater could simply continue the rolling update until it successfully reduced the count of the original to 0. Retargeting the auto-scaler in the middle of the rolling update (when the size of the new exceeds the size of the old) would facilitate convergence.

If the auto-scaler shrunk the original replication controller to 0, the rolling update could stop and retarget the auto-scaler if it hadn't been retargeted already. If the auto-scaler wanted to reduce the set size more than it was able, we could simply let the auto-scaler shrink the new replication controller after it is retargeted -- it's more important that the auto-scaler is able to grow the number of instances during updates than to shrink the number of instances precisely.

/cc @jlowdermilk re. the interaction of rolling updates and auto-scaling.

Is this PR ready to review? Can you incorporate all of the relevant bits of the long design discussion into the doc itself?

@davidopp I will update the doc shortly to revert the selector and document some of the runtime behavior assumptions based on recent discussions. If folks are feeling ok about the basics of this proposal I can start getting together some small PRs for types, REST, etc and we can close this PR since it is just a document. I just didn't want to jump the gun.

Autoscaling the largest selected replication controller does seem reasonable.

On Mar 2, 2015, at 8:46 AM, Paul notifications@github.com wrote:

@davidopp I will update the doc shortly to revert the selector and document some of the runtime behavior assumptions based on recent discussions. If folks are feeling ok about the basics of this proposal I can start getting together some small PRs for types, REST, etc and we can close this PR since it is just a document. I just didn't want to jump the gun.

—
Reply to this email directly or view it on GitHub.

Updated to change the target back to a selector, re-added the multi-target policy section with a note that the initial default implementation will be to scale the largest target.

Does anyone feel that anything relevant was missed? Please let me know and I'll document it.

I made some comments.

BTW, do you feel reasonably confident that useful real-world auto-scaling policies can be expressed using the policy configuration language you defined here? I totally buy the architecture and basic concepts, but I'm concerned that real-world policies maybe can't just be boiled down to parameters over the set of knobs you defined here, and might instead need general-purpose code.

@davidopp Thanks for the feedback, I've updated the document.

RE: policies. I feel like the most useful piece of policy are the intention based thresholds. Value based thresholds have the potential to create a complex set of interacting rules that would be hard to manage but they do give you the ability to fine tune things if you really want to. However, it seems much more useful to express intent for scaling rather than hard and fast rules.

I also think that value thresholds rely heavily on admin users knowing the landscape of the statistics being gathered which is another down vote from me. I would not be opposed to removing value thresholds altogether and just going with @fbalejko 's suggestion for intention based thresholds.

Is that more along the lines of what you were thinking or were you thinking user-injected general purpose code?

Since this is just a design doc and not an implementation, I don't really think you need to change the proposal (if nothing else, recording the idea for posterity is useful even if we don't end up implementing it). I was just curious because I think autoscaling is one of those things where you can't really know what kinds of policies (or axes of parameterization for a generic autoscaler) will work until you try them. Anyway, this is more philosophical than anything.

LGTM once you address my one remaining comment.

Agree with your thoughts re: policy definition.

Commented on intention of Threshold.Selector and updated the proposal comments as well.

Thanks for your pull request.

It looks like this may be your first contribution to a Google open source project, in which case you'll need to sign a Contributor License Agreement (CLA) at https://cla.developers.google.com/.

If you've already signed a CLA, it's possible we don't have your GitHub username or you're using a different email address. Check the information on your CLA or see this help article on setting the email on your git commits.

Once you've done that, please reply here to let us know. If you signed the CLA as a corporation, please let us know the company's name.

This is LGTM. If no objections from the crowd I'll merge.