Skip to content

Pod dependencies on services #2385

Closed
Closed
@satnam6502

Description

One thing that is a bad experience at the moment is the bring-up behaviour of one pod that depends on another the services of another pod. For example, in my logging work the Kibana viewer (pod, service) depends on the Elasticsearch (pod, service). When I try and bring them up together from my Makefile I have an intermediate sate like this for quite a while:

NAME                           IMAGE(S)                                                                            HOST                                                           LABELS                      STATUS
influx-grafana                 kubernetes/heapster_influxdb,kubernetes/heapster_grafana,dockerfile/elasticsearch   kubernetes-minion-3.c.kubernetes-elk.internal/146.148.76.82    name=influxdb               Pending
heapster                       kubernetes/heapster                                                                 kubernetes-minion-1.c.kubernetes-elk.internal/130.211.126.68   name=heapster               Running
synthetic-logger-0.25lps-pod   ubuntu:14.04                                                                        kubernetes-minion-1.c.kubernetes-elk.internal/130.211.126.68   name=synth-logging-source   Running
elasticsearch-pod              dockerfile/elasticsearch                                                            kubernetes-minion-2.c.kubernetes-elk.internal/23.236.59.213    app=elasticsearch           Pending
kibana-pod                     kubernetes/kibana:latest                                                            kubernetes-minion-4.c.kubernetes-elk.internal/130.211.121.21   app=kibana-viewer           Failed

i.e. the Kibana viewer fails to start up because Elasticsearch is not ready yet. Eventually things start to look better:

NAME                           IMAGE(S)                                                                            HOST                                                           LABELS                      STATUS
influx-grafana                 kubernetes/heapster_influxdb,kubernetes/heapster_grafana,dockerfile/elasticsearch   kubernetes-minion-3.c.kubernetes-elk.internal/146.148.76.82    name=influxdb               Pending
heapster                       kubernetes/heapster                                                                 kubernetes-minion-1.c.kubernetes-elk.internal/130.211.126.68   name=heapster               Running
synthetic-logger-0.25lps-pod   ubuntu:14.04                                                                        kubernetes-minion-1.c.kubernetes-elk.internal/130.211.126.68   name=synth-logging-source   Running
elasticsearch-pod              dockerfile/elasticsearch                                                            kubernetes-minion-2.c.kubernetes-elk.internal/23.236.59.213    app=elasticsearch           Running
kibana-pod                     kubernetes/kibana:latest                                                            kubernetes-minion-4.c.kubernetes-elk.internal/130.211.121.21   app=kibana-viewer           Running
kubectl.sh get services

but even though the pods are marked as Running they are still not quite ready yet and it takes another five minutes or so before one can make queries to Elasticsearch and see log output in Kibana.

It would be nice to describe in a pod declaration its dependencies on other services so this can be taken into account during scheudling. For example:

apiVersion: v1beta1
kind: Pod
id: kibana-pod
desiredState:
  manifest:
    version: v1beta1
    id: kibana-server
    containers:
      - name: kibana-image
        image: kubernetes/kibana:latest
        ports:
          - name: kibana-port
            containerPort: 80
        dependencies: [elasticsearch]
labels:
  app: kibana-viewer

This would delay the scheduling of this pod until the pod(s) identified by the elasticsearch service are all in the running state.

Metadata

Assignees

No one assigned

    Labels

    kind/designCategorizes issue or PR as related to design.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions