Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Suggest a simple rolling update. #6713

Merged
merged 1 commit into from
Apr 23, 2015
Merged

Conversation

brendandburns
Copy link
Contributor

@brendandburns
Copy link
Contributor Author

@thockin too.

* Do the standard down/up one-by-one until foo has replicas zero and foo-next has replicas N
* delete rc foo
* create rc foo that is identical to foo-next
* delete foo-next
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will require us changing the validation rules for rc. Currently we block rc creation if the selector is a sub/superset of an existing rc selector. We'd need to change it so that validation allows duplicate rcs with overlapping selectors if they have the same desired size.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should just delete both controllers first. Nothing good can come from having 2 controlling the same set of pods at the same time.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if the server dies somewhere during this multi-stage process? Do we require the client to detect this and finish the remaining operations when the server comes back up? Can we use annotations to help with this, like we're currently using annotations with kubectl rollingupdate?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bgrant0607
I would really prefer not to leave a user in a situation where their pods are unmanaged, so my preference would be to update the validation rules as @jlowdermilk suggests.

I'd much rather have 2 replication controllers temporarily controlling the same pods, than temporarily have no replication controllers controlling the pods.

Especially if we add validation so that you really can't resize them.

@davidopp
Added some details on annotations.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. This is an implementation detail. We can always change it later if it causes problems.

### Lightweight rollout
Assume that we have a current replication controller named ```foo``` and it is running image ```image:v1```

```kubectl rollingupdate rc foo --image=myimage:v2```
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW, it's rolling-update now. rollingupdate is deprecated.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

@bgrant0607
Copy link
Member

cc @rjnagal

This is a lightweight design document for simple rolling update in ```kubectl```

### Lightweight rollout
Assume that we have a current replication controller named ```foo``` and it is running image ```image:v1```
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This assumes a single container, obviously.

As much as possible, flags should match kubectl run/run-container.

If we were to support multiple containers in run, we would support that here, also.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm ok if the multi-container case requires a config file. I think this will hit the 80% use case pretty squarely.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair enough.

@brendandburns brendandburns force-pushed the docs branch 3 times, most recently from c8f3eae to 5333b7a Compare April 11, 2015 03:28
@brendandburns
Copy link
Contributor Author

Comments addressed. ptal.

Thanks
--brendan

* ```desired-replicas``` The desired number of replicas for this controller (either N or zero)
* ```update-partner``` A pointer to the resource that is the other half of this update (syntax ```<namespace>:<id>```)

Recovery is achieved by issuing the 'resume' rolling update command:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why can't they just execute exactly the same command?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree - anything else is non-intuitive.

On Apr 10, 2015, at 11:48 PM, Brian Grant notifications@github.com wrote:

In docs/design/simple-rolling-update.md:

+### Aborting a rollout
+Abort is assumed to want to reverse a rollout in progress. If the rollout has completed, then rollback is just a special case of rollout.
+
+kubectl rolling-update rc foo [foo-v2] abort
+
+Under the hood this will:

  • * look for a replica controller named foo-next, or the user-specified name (foo-v2 in the example above)
  • * Given this controller it will then increment/decrement until foo is back to the initial size, and foo-next is size zero.
  • * When foo-next is size zero, it is deleted.

+### Recovery
+To facilitate recovery in the case of a crash of the updating process itself, we add the following annotations to each replicaController:

  • * desired-replicas The desired number of replicas for this controller (either N or zero)
  • * update-partner A pointer to the resource that is the other half of this update (syntax <namespace>:<id>)

+Recovery is achieved by issuing the 'resume' rolling update command:
Why can't they just execute exactly the same command?


Reply to this email directly or view it on GitHub.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

@brendandburns
Copy link
Contributor Author

Updated this to reflect comments and add more detail.

@smarterclayton @jlowdermilk @bgrant0607

please take a look.

Thanks!
--brendan

@brendandburns
Copy link
Contributor Author

@smarterclayton @bgrant0607 friendly ping on this, I'd like to start implementing asap.

Thanks!
--brendan

If the user doesn't specify a name for the 'next' controller, then the 'next' controller is renamed to
the name of the original controller.

Obviously there is a race here, where if you kill the client between delete foo, and creating the new version of 'foo' you might be surprised about what is there, but I think that's ok.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you need to explain how the rename is implemented, otherwise it's not clear what the race condition is that you're referring to.


For the purposes of this example, assume that we are rolling from ```foo``` to ```foo-next``` where the only change is an image update from `v1` to `v2`

#### Initialization
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you detail what happens if the user did not specify foo-next? How do you crash-recover in this case? Where does the update-partner annotation come into play?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

@brendandburns brendandburns force-pushed the docs branch 2 times, most recently from 964c5a9 to 34d6b44 Compare April 22, 2015 17:19
@brendandburns
Copy link
Contributor Author

@davidopp @thockin @bgrant0607 @smarterclayton

Comments addresed. Please re-check.

Thanks!
--brendan

@bgrant0607
Copy link
Member

LGTM

1 similar comment
@davidopp
Copy link
Member

LGTM

davidopp added a commit that referenced this pull request Apr 23, 2015
Suggest a simple rolling update.
@davidopp davidopp merged commit 54f2015 into kubernetes:master Apr 23, 2015
### Aborting a rollout
Abort is assumed to want to reverse a rollout in progress.

```kubectl rolling-update rc foo [foo-v2] --abort```
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW I preferred "rollback" or "reverse" to "abort". Abort implies killing an on-going process.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Me, too, but perhaps an argument could be made that it's similar to unstaging a commit in git.

xingzhou pushed a commit to xingzhou/kubernetes that referenced this pull request Dec 15, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants