-
Notifications
You must be signed in to change notification settings - Fork 40k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Suggest a simple rolling update. #6713
Conversation
@thockin too. |
* Do the standard down/up one-by-one until foo has replicas zero and foo-next has replicas N | ||
* delete rc foo | ||
* create rc foo that is identical to foo-next | ||
* delete foo-next |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This will require us changing the validation rules for rc. Currently we block rc creation if the selector is a sub/superset of an existing rc selector. We'd need to change it so that validation allows duplicate rcs with overlapping selectors if they have the same desired size.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should just delete both controllers first. Nothing good can come from having 2 controlling the same set of pods at the same time.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What happens if the server dies somewhere during this multi-stage process? Do we require the client to detect this and finish the remaining operations when the server comes back up? Can we use annotations to help with this, like we're currently using annotations with kubectl rollingupdate?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@bgrant0607
I would really prefer not to leave a user in a situation where their pods are unmanaged, so my preference would be to update the validation rules as @jlowdermilk suggests.
I'd much rather have 2 replication controllers temporarily controlling the same pods, than temporarily have no replication controllers controlling the pods.
Especially if we add validation so that you really can't resize them.
@davidopp
Added some details on annotations.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok. This is an implementation detail. We can always change it later if it causes problems.
### Lightweight rollout | ||
Assume that we have a current replication controller named ```foo``` and it is running image ```image:v1``` | ||
|
||
```kubectl rollingupdate rc foo --image=myimage:v2``` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
BTW, it's rolling-update
now. rollingupdate
is deprecated.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done.
cc @rjnagal |
This is a lightweight design document for simple rolling update in ```kubectl``` | ||
|
||
### Lightweight rollout | ||
Assume that we have a current replication controller named ```foo``` and it is running image ```image:v1``` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This assumes a single container, obviously.
As much as possible, flags should match kubectl run/run-container.
If we were to support multiple containers in run, we would support that here, also.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm ok if the multi-container case requires a config file. I think this will hit the 80% use case pretty squarely.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fair enough.
c8f3eae
to
5333b7a
Compare
Comments addressed. ptal. Thanks |
* ```desired-replicas``` The desired number of replicas for this controller (either N or zero) | ||
* ```update-partner``` A pointer to the resource that is the other half of this update (syntax ```<namespace>:<id>```) | ||
|
||
Recovery is achieved by issuing the 'resume' rolling update command: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why can't they just execute exactly the same command?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agree - anything else is non-intuitive.
On Apr 10, 2015, at 11:48 PM, Brian Grant notifications@github.com wrote:
In docs/design/simple-rolling-update.md:
+### Aborting a rollout
+Abort is assumed to want to reverse a rollout in progress. If the rollout has completed, then rollback is just a special case of rollout.
+
+kubectl rolling-update rc foo [foo-v2] abort
+
+Under the hood this will:
- * look for a replica controller named
foo-next
, or the user-specified name (foo-v2
in the example above)- * Given this controller it will then increment/decrement until
foo
is back to the initial size, andfoo-next
is size zero.- * When
foo-next
is size zero, it is deleted.
+### Recovery
+To facilitate recovery in the case of a crash of the updating process itself, we add the following annotations to each replicaController:
- *
desired-replicas
The desired number of replicas for this controller (either N or zero)- *
update-partner
A pointer to the resource that is the other half of this update (syntax<namespace>:<id>
)
+Recovery is achieved by issuing the 'resume' rolling update command:
Why can't they just execute exactly the same command?—
Reply to this email directly or view it on GitHub.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done.
Updated this to reflect comments and add more detail. @smarterclayton @jlowdermilk @bgrant0607 please take a look. Thanks! |
@smarterclayton @bgrant0607 friendly ping on this, I'd like to start implementing asap. Thanks! |
If the user doesn't specify a name for the 'next' controller, then the 'next' controller is renamed to | ||
the name of the original controller. | ||
|
||
Obviously there is a race here, where if you kill the client between delete foo, and creating the new version of 'foo' you might be surprised about what is there, but I think that's ok. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you need to explain how the rename is implemented, otherwise it's not clear what the race condition is that you're referring to.
|
||
For the purposes of this example, assume that we are rolling from ```foo``` to ```foo-next``` where the only change is an image update from `v1` to `v2` | ||
|
||
#### Initialization |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you detail what happens if the user did not specify foo-next? How do you crash-recover in this case? Where does the update-partner annotation come into play?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done.
964c5a9
to
34d6b44
Compare
@davidopp @thockin @bgrant0607 @smarterclayton Comments addresed. Please re-check. Thanks! |
LGTM |
1 similar comment
LGTM |
Suggest a simple rolling update.
### Aborting a rollout | ||
Abort is assumed to want to reverse a rollout in progress. | ||
|
||
```kubectl rolling-update rc foo [foo-v2] --abort``` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
FWIW I preferred "rollback" or "reverse" to "abort". Abort implies killing an on-going process.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Me, too, but perhaps an argument could be made that it's similar to unstaging a commit in git.
Suggest a simple rolling update.
@bgrant0607 @kelseyhightower @jlowdermilk @davidopp @smarterclayton