I have a similar usecase for this, so I would also like to see this added :)

@kubernetes/goog-ux

See also #19695

cc @vishh

Yeah, very common request. We've ended up adding heuristics "If PodStatus Pending and no containers, display message to user Probably Pulling, but we aren't sure the node is there", etc.

We generate events for this purpose. We currently have a pulling and
pulled
event.

AFAIK, docker does not surface image pull progress. Did that change
recently?

On Sun, Jan 31, 2016 at 1:33 PM, Clayton Coleman notifications@github.com
wrote:

Yeah, very common request. We've ended up adding heuristics "If PodStatus
Pending and no containers, display message to user Probably Pulling, but
we aren't sure the node is there", etc.

—
Reply to this email directly or view it on GitHub
#19077 (comment)
.

Practically speaking, people use pod status to figure out what the pod is
doing. The vast majority of the time between "user creates pod" and "user
sees success" is going to be spent pulling. The fact that we have no
discrete status indicating that appears to be a bug to people. The event
is useful, but if we're updating the status anyway we should know whether
we're pulling immediately or not when we write the pod status.

On Thu, Feb 11, 2016 at 4:33 PM, Vish Kannan notifications@github.com
wrote:

We generate events for this purpose. We currently have a [pulling and
pulled](

https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/container/event.go#L29
)
event.

AFAIK, docker does not surface image pull progress. Did that change
recently?

On Sun, Jan 31, 2016 at 1:33 PM, Clayton Coleman <notifications@github.com

wrote:

Yeah, very common request. We've ended up adding heuristics "If PodStatus
Pending and no containers, display message to user Probably Pulling, but
we aren't sure the node is there", etc.

—
Reply to this email directly or view it on GitHub
<
#19077 (comment)

.

—
Reply to this email directly or view it on GitHub
#19077 (comment)
.

By Status are you referring to the output of kubectl describe or
PodStatus? If users have insight into when the pod was accepted by the
kubelet and when it starts and completes (or fails) to pull an image, would
that suffice?

On Thu, Feb 11, 2016 at 1:46 PM, Clayton Coleman notifications@github.com
wrote:

Practically speaking, people use pod status to figure out what the pod is
doing. The vast majority of the time between "user creates pod" and "user
sees success" is going to be spent pulling. The fact that we have no
discrete status indicating that appears to be a bug to people. The event
is useful, but if we're updating the status anyway we should know whether
we're pulling immediately or not when we write the pod status.

On Thu, Feb 11, 2016 at 4:33 PM, Vish Kannan notifications@github.com
wrote:

We generate events for this purpose. We currently have a [pulling and
pulled](

https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/container/event.go#L29
)
event.

AFAIK, docker does not surface image pull progress. Did that change
recently?

On Sun, Jan 31, 2016 at 1:33 PM, Clayton Coleman <
notifications@github.com

wrote:

Yeah, very common request. We've ended up adding heuristics "If
PodStatus
Pending and no containers, display message to user Probably Pulling,
but
we aren't sure the node is there", etc.

—
Reply to this email directly or view it on GitHub
<

#19077 (comment)

.

—
Reply to this email directly or view it on GitHub
<
#19077 (comment)

.

—
Reply to this email directly or view it on GitHub
#19077 (comment)
.

The requester seems to want a progress measure 30% pulled or x/y

#25032

@smarterclayton

Right now there is no distinction between waiting
for schedule and pull, which is a common state people get into.

The output of kubectl describe pod today includes events that provide the information you seek. Here is an example:

Events:
  FirstSeen LastSeen    Count   From                    SubobjectPath           Type        Reason      Message
  --------- --------    -----   ----                    -------------           --------    ------      -------
  8s        8s      1   {default-scheduler }                            Normal      Scheduled   Successfully assigned busybox-573201948-x4rg4 to kubernetes-minion-31zg
  8s        7s      2   {kubelet kubernetes-minion-31zg}    spec.containers{busybox}    Normal      Pulling     pulling image "busybox"
  7s        7s      1   {kubelet kubernetes-minion-31zg}    spec.containers{busybox}    Normal      Created     Created container with docker id 7ac36eac5dc5
  7s        7s      1   {kubelet kubernetes-minion-31zg}    spec.containers{busybox}    Normal      Started     Started container with docker id 7ac36eac5dc5
  7s        6s      2   {kubelet kubernetes-minion-31zg}    spec.containers{busybox}    Normal      Pulled      Successfully pulled image "busybox"
  6s        6s      1   {kubelet kubernetes-minion-31zg}    spec.containers{busybox}    Normal      Created     Created container with docker id 8a1d39975f1c
  6s        6s      1   {kubelet kubernetes-minion-31zg}    spec.containers{busybox}    Normal      Started     Started container with docker id 8a1d39975f1c

We can possibly add more events that include the progress, if an image pull were to take longer than expected.

@stuartbassett

I don't see why #25032 is needed yet.

Wouldn't UIs include events as well, at-least the critical ones?

@smarterclayton It is hard to update the image pulling progress in pod status for now. Because pod status is updated before each SyncPod, while the whole image pulling process happens during SyncPod.

The plan for now is to periodically (maybe every 5, 10 senconds) send event telling what is the current image pulling progress, which will at least tell user whether the image pulling is stuck there.

Doesn't pulling then effectively block all sync progress for other
containers in the same pod? Or does it merely block parallel startup of
the containers.

On Tue, May 3, 2016 at 6:08 PM, Lantao Liu notifications@github.com wrote:

@smarterclayton https://github.com/smarterclayton It is hard to update
the image pulling progress in pod status for now. Pod status is updated
before each SyncPod, while the whole image pulling process happens during
SyncPod.

The plan for now is to periodically (maybe every 5, 10 senconds) send
event telling what is the current image pulling progress, which will at
least tell user that whether the image pulling is stuck there.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#19077 (comment)

@smarterclayton Assuming that exposing image pull progress is mainly for human consumption, would the following work?

• If an image is being pulled for a container, then update ContainerStateWaiting to contain a reason PullingImage and a message "progress: x%". The progress will be made available on a best-effort basis.
• If a pod is inPending phase because images are being pulled, reflect that in PodStatus.Reason and PodStatus.Message. Message could be an aggregate across all containers.

This would essentially push the burden of generating human friendly pod and container status to Kubelet.

I don't even think progress is required, I'd be happy with a single
reason. But I'll take the message and say that it'll help users even on a
best effort.

On Tue, May 3, 2016 at 6:42 PM, Vish Kannan notifications@github.com
wrote:

@smarterclayton https://github.com/smarterclayton Assuming that
exposing image pull progress is mainly for human consumption, would the
following work?

• If an image is being pulled for a container, then update
  ContainerStateWaiting to contain a reason PullingImage and a message
  "progress: x%". The progress will be made available on a best-effort basis.
• If a pod is inPending phase because images are being pulled, reflect
  that in PodStatus.Reason and PodStatus.Message. Message could be an
  aggregate across all containers.

This would essentially push the burden of generating human friendly pod
and container status to Kubelet.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#19077 (comment)

Basically, yes, I think that would make 90% of clients better.

On Tue, May 3, 2016 at 6:47 PM, Clayton Coleman ccoleman@redhat.com wrote:

I don't even think progress is required, I'd be happy with a single
reason. But I'll take the message and say that it'll help users even on a
best effort.

On Tue, May 3, 2016 at 6:42 PM, Vish Kannan notifications@github.com
wrote:

@smarterclayton https://github.com/smarterclayton Assuming that
exposing image pull progress is mainly for human consumption, would the
following work?

• If an image is being pulled for a container, then update
  ContainerStateWaiting to contain a reason PullingImage and a message
  "progress: x%". The progress will be made available on a best-effort basis.
• If a pod is inPending phase because images are being pulled,
  reflect that in PodStatus.Reason and PodStatus.Message. Message could
  be an aggregate across all containers.

This would essentially push the burden of generating human friendly pod
and container status to Kubelet.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#19077 (comment)

@stuartbassett Will you be able to re-purpose your PR (#25032) to do what's mentioned in #19077 (comment) ? I can provide more specific design details, if you have difficulty parsing #19077 (comment)

@smarterclayton Yeah, for now it will block starting of all other containers, but we definitely do not want that and should make it better in the future, :)

/remove-lifecycle stale

Do we know if there exists any better way of achieving this?

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

/remove-lifecycle stale

We could write up a KEP for this and pitch it in SIG Node. I’d be happy to drive this topic forward, but we should get at least 3 people on board. Who is in? 🙃

@saschagrunert I am interested - what sort of commitment do you need?

@saschagrunert I am interested - what sort of commitment do you need?

I never wrote a KEP, but I’m thrilled to write one. I would just need relevant input, review and maybe implementation support. We could create a small work group in slack if you want. :)

I commented on the cri-o issue, but I think we could separate this into two parts:

1. Progress for end users of kube (the issue this was raised for)
2. Better administrative / operational insight into pulls

The former is definitely Kube since it would have to be exposed via an api. However the second is likely to be fairly specific to container runtime implementation and the storage that backs it. And given the improvements in monitoring since this issue was opened, and that most deployments likely have prometheus or something like it watching their container runtimes, the second might be the best place to start both to make a concrete first step now for admins, while also learning more about how we might expose progress. I do not think the former item was intended to solve the latter, and the latter is probably more broadly applicable since the vast majority of clusters are single team owned.

FWIW my interest comes from working on tools layered on top of Kubernetes, where the rollout is initiated by something other that kubectl (for instance a git commit).
And we'd like that tooling to be able to feed back status and/or issues without guessing.

Status needs to be tied to a specific update, since multiple overlapping updates can be issued.

I can't immediately see how Prometheus solves this requirement.

I commented on the cri-o issue, but I think we could separate this into two parts:

1. Progress for end users of kube (the issue this was raised for)
2. Better administrative / operational insight into pulls

I'd be happy to push both topics forward. From the runtime perspective: If we have the data at hand, then we could expose it to any interface.

I see four major points, whereas the first one could be dropped from here:

1. Getting the data inside the runtime (out of scope from here)
2. Getting the data into the kubelet via the CRI API (KEP)
3. Exposing the data via the API Server (KEP)
4. Exposing the data via the CLI (KEP)

Anything else?

I wrote an email to the SIG Node mailing list regarding the topic and the plan:
https://groups.google.com/forum/#!topic/kubernetes-sig-node/JHEus_TlZzA

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

@fejta-bot: Closing this issue.

Is there a ticket where one could vote for disabling fejta-bot?

It pollutes the long-standing discussions and eventually closes the important issues where, I suspect, people simply got tired of interacting with the bot. I'm certainly annoyed of getting notifications from it both as an author and participant of a few issues.

Is this ever going to be a thing?

@smarterclayton

Is there no progress on this most-wanted feature request?

For those of you interested in this issue, please coordinate your interest into something actionable which in our community is a KEP:

• https://github.com/kubernetes/enhancements#is-my-thing-an-enhancement

Please feel free to use community resources (sig-node mailing list, agenda on sig-node meetings, google docs to seed discussion etc) to figure out what needs to be in the KEP and how the feature progress(es) through community process(es). Some good info can be found in:

• https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/feature-gates.md
• https://github.com/kubernetes/community/tree/master/contributors/devel/sig-architecture
• https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api_changes.md

You should probably read one of the KEP(s) from before in sig-node for example in:

• https://github.com/kubernetes/enhancements/tree/master/keps/sig-node

Looking forward to folks stepping up to help with this! thanks in advance.

Closing the loop, there is a KEP: kubernetes/enhancements#3542