Skip to content
This repository has been archived by the owner on Nov 30, 2021. It is now read-only.

Pods fail to mount secret on k8s 1.3.0 + GKE Β #372

Closed
@mboersma

Description

Builder, database, minio, and registry all mount the "objectstorage-keyfile" secret volume. In k8s 1.3 on GKE, this began to fail (see the final Events listing):

$ kubectl --namespace=deis describe po deis-database-dyqyu 
Name:       deis-database-dyqyu
Namespace:  deis
Node:       gke-mboersma-default-pool-21df92ab-lh7d/10.240.0.16
Start Time: Wed, 13 Jul 2016 15:28:28 -0600
Labels:     app=deis-database
Status:     Pending
IP:     
Controllers:    ReplicationController/deis-database
Containers:
  deis-database:
    Container ID:   
    Image:      quay.io/deisci/postgres:canary
    Image ID:       
    Port:       5432/TCP
    QoS Tier:
      memory:       BestEffort
      cpu:      BestEffort
    State:      Waiting
      Reason:       ContainerCreating
    Ready:      False
    Restart Count:  0
    Readiness:      exec [is_running] delay=30s timeout=1s period=10s #success=1 #failure=3
    Environment Variables:
      DATABASE_STORAGE: minio
Conditions:
  Type      Status
  Initialized   True 
  Ready     False 
  PodScheduled  True 
Volumes:
  database-creds:
    Type:   Secret (a volume populated by a Secret)
    SecretName: database-creds
  objectstore-creds:
    Type:   Secret (a volume populated by a Secret)
    SecretName: objectstorage-keyfile
  deis-database-token-ovk2r:
    Type:   Secret (a volume populated by a Secret)
    SecretName: deis-database-token-ovk2r
Events:
  FirstSeen LastSeen    Count   From                            SubobjectPath   Type        Reason      Message
  --------- --------    -----   ----                            -------------   --------    ------      -------
  4m        4m      1   {default-scheduler }                            Normal      Scheduled   Successfully assigned deis-database-dyqyu to gke-mboersma-default-pool-21df92ab-lh7d
  2m        2m      1   {kubelet gke-mboersma-default-pool-21df92ab-lh7d}           Warning     FailedMount Unable to mount volumes for pod "deis-database-dyqyu_deis(bd127bc0-4940-11e6-af68-42010af001d0)": timeout expired waiting for volumes to attach/mount for pod "deis-database-dyqyu"/"deis". list of unattached/unmounted volumes=[objectstore-creds]
  2m        2m      1   {kubelet gke-mboersma-default-pool-21df92ab-lh7d}           Warning     FailedSync  Error syncing pod, skipping: timeout expired waiting for volumes to attach/mount for pod "deis-database-dyqyu"/"deis". list of unattached/unmounted volumes=[objectstore-creds]

This appears to be related to kubernetes/kubernetes#28750 and maybe kubernetes/kubernetes#28898 and kubernetes/kubernetes#28616.

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions