Skip to content

Make rc manager more responsive to unexpected adds/deletes #7369

Open
@bprashanth

Description

Currently the rc manager will not respond to unexpected pod creates/deletes till all the creates/deletes it has sent to the apiserver have been observed by the reflector, or a timeout occurs. There are situations where we could land up with:

  • Rc goes to sleep till it sees 1 add
  • One pod gets deleted <- we want the rc to wake up here
  • One pod is created <- rc really wakes up here

The easy way around this is to use the expectations of the rc to compute how many extra pods to create even when the watch hasn't observed the adds/dels. Eg:

observed + (expected add - expected dels) - status.Replicas

The tricky bit here is locking, since currently the expectations are incremented/decremented via atomic operations. This will matter more in hostile netwoks with high lag, where the watch may not transmit pod add/dels for a while.

Metadata

Assignees

No one assigned

    Labels

    area/controller-managerlifecycle/frozenIndicates that an issue or PR should not be auto-closed due to staleness.priority/awaiting-more-evidenceLowest priority. Possibly useful, but not yet enough support to actually get it done.sig/appsCategorizes an issue or PR as relevant to SIG Apps.

    Type

    No type

    Projects

    • Status

      Needs Triage

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions