In general starting a new container guarantees a clean filesystem start state, whereas restart (unless this has changed recently) won't. I think we would want to use clean states as much as possible.

I think you are correct that without checkpoint state we must retain the docker containers. An intermediate solution would be to log the state to disk and then clean the containers.

Using restart has the advantage that any temporary on-disk state would be
preserved across container crash-restarts. What about crashes that are due
to disk quota?
On Jul 25, 2014 10:16 PM, "Clayton Coleman" notifications@github.com
wrote:

In general starting a new container guarantees a clean filesystem start
state, whereas restart (unless this has changed recently) won't. I think we
would want to use clean states as much as possible.

I think you are correct that without checkpoint state we must retain the
docker containers. An intermediate solution would be to log the state to
disk and then clean the containers.

Reply to this email directly or view it on GitHub
#629 (comment)
.

Ideally recording the exit condition and deleting the old container happens before the restart. If your crash is due to quota and you were the one writing it you probably can't restart until the original container is deleted anyway (is debug-ability or being available more important? I think you can come up with reasons why either is preferred, although you may want to get access to the old, crashed container as a separate operation.

We have had a bad experience waiting for disk cleanup on a container
crash/restart. We go to great pain to make it possible to restart
immediately - O(tens to hundreds of millseconds).

Obviously, if you are out of disk space, you are kind of stuck, but
otherwise it should be fast.

On Sat, Jul 26, 2014 at 8:22 AM, Clayton Coleman notifications@github.com
wrote:

Ideally recording the exit condition and deleting the old container
happens before the restart. If your crash is due to quota and you were the
one writing it you probably can't restart until the original container is
deleted anyway. I think you can come up with reasons why either is
preferred, although you may want to get access to the old, crashed
container as a separate operation.

Reply to this email directly or view it on GitHub
#629 (comment)
.

So prioritize at least the possibility that some subset of processes (those that at least do cleanup on start) can come up when restarted? Makes sense

I'm not sure I am arguing one way or the other. Just exploring the idea.

On Sat, Jul 26, 2014 at 8:34 AM, Clayton Coleman notifications@github.com
wrote:

So prioritize at least the possibility that some subset of processes
(those that at least do cleanup on start) can come up when restarted? Makes
sense

Reply to this email directly or view it on GitHub
#629 (comment)
.

Debuggability and availability should be equally important to the user. At different stage of software / service lifecyle, one of them might be preferred. But talking about debuggability, in most of cases, only the initial failure and the last failure are interesting to the users. In this case, we could have an asyn operation to do disk cleanup for all crashed/restarted containers.