Skip to content

Docker healthcheck causes high CPU utilization #39102

Open
@oleg-fiksel

Description

Description

Running container with healthchecks causes high CPU utilisation.

Steps to reproduce the issue:

  1. Install OpenFaas https://docs.openfaas.com/deployment/docker-swarm/
  2. Login into the OpenFaas Web UI and install 2-3 functions

Screenshot 2019-04-17 at 17 37 45

  • nodeinfo
  • sentimentanalysis
  1. Do not use the functions for the platform be idle

Describe the results you received:

The load on the Docker Node increases drastically just because of the health checks.

Describe the results you expected:

I would expect the load to be almost the same as not running any healthchecks.

Additional information you deem important (e.g. issue happens only occasionally):

Screenshot 2019-04-17 at 16 38 36

Screenshot 2019-04-17 at 16 38 05

Screenshot 2019-04-17 at 16 38 50

Output of docker version:

# docker version
Client:
 Version:           18.09.3
 API version:       1.39
 Go version:        go1.10.8
 Git commit:        774a1f4
 Built:             Thu Feb 28 06:40:58 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          18.09.5
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.10.8
  Git commit:       e8ff056
  Built:            Thu Apr 11 04:10:53 2019
  OS/Arch:          linux/amd64
  Experimental:     false

Output of docker info:

# docker info
Containers: 10
 Running: 8
 Paused: 0
 Stopped: 2
Images: 9
Server Version: 18.09.5
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: active
 NodeID: ql2lpbvdhdbspzd6n2qsmip4n
 Is Manager: true
 ClusterID: a2f6d1mbqs1mczk8y88axzmpu
 Managers: 1
 Nodes: 1
 Default Address Pool: 10.0.0.0/8
 SubnetSize: 24
 Orchestration:
  Task History Retention Limit: 5
 Raft:
  Snapshot Interval: 10000
  Number of Old Snapshots to Retain: 0
  Heartbeat Tick: 1
  Election Tick: 10
 Dispatcher:
  Heartbeat Period: 5 seconds
 CA Configuration:
  Expiry Duration: 3 months
  Force Rotate: 0
 Autolock Managers: false
 Root Rotation In Progress: false
 Node Address: 172.16.11.137
 Manager Addresses:
  172.16.11.137:2377
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: e6b3f5632f50dbc4e9cb6288d911bf4f5e95b18e
runc version: 6635b4f0c6af3810594d2770f662f34ddc15b40d
init version: fec3683
Security Options:
 apparmor
 seccomp
  Profile: default
Kernel Version: 4.4.0-143-generic
Operating System: Ubuntu 16.04.6 LTS
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 1.936GiB
Name: openfaas
ID: DWEP:3RFR:F6EL:3DEP:SVRG:MMZ5:2KBT:5ZZQ:IWHZ:VAIJ:UMHA:4QIL
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false
Product License: Community Engine

WARNING: No swap limit support

Additional environment details (AWS, VirtualBox, physical, etc.):

VM on VMware Fusion on Mac. Same results with a VM on KVM on Linux.

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions