Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

daemon option (--storage-opt dm.basesize) for increasing the base device size on daemon restart #19123

Merged
merged 1 commit into from
Jan 13, 2016
Merged

daemon option (--storage-opt dm.basesize) for increasing the base device size on daemon restart #19123

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
83 changes: 79 additions & 4 deletions daemon/graphdriver/devmapper/deviceset.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ import (
var (
defaultDataLoopbackSize int64 = 100 * 1024 * 1024 * 1024
defaultMetaDataLoopbackSize int64 = 2 * 1024 * 1024 * 1024
defaultBaseFsSize uint64 = 100 * 1024 * 1024 * 1024
defaultBaseFsSize uint64 = 10 * 1024 * 1024 * 1024
defaultThinpBlockSize uint32 = 128 // 64K = 128 512b sectors
defaultUdevSyncOverride = false
maxDeviceID = 0xffffff // 24 bit, pool limit
Expand All @@ -47,6 +47,7 @@ var (
driverDeferredRemovalSupport = false
enableDeferredRemoval = false
enableDeferredDeletion = false
userBaseSize = false
)

const deviceSetMetaFile string = "deviceset-metadata"
Expand Down Expand Up @@ -1056,6 +1057,80 @@ func (devices *DeviceSet) setupVerifyBaseImageUUIDFS(baseInfo *devInfo) error {
return nil
}

func (devices *DeviceSet) checkGrowBaseDeviceFS(info *devInfo) error {

if !userBaseSize {
return nil
}

if devices.baseFsSize < devices.getBaseDeviceSize() {
return fmt.Errorf("devmapper: Base device size cannot be smaller than %s", units.HumanSize(float64(devices.getBaseDeviceSize())))
}

if devices.baseFsSize == devices.getBaseDeviceSize() {
return nil
}

info.lock.Lock()
defer info.lock.Unlock()

devices.Lock()
defer devices.Unlock()

info.Size = devices.baseFsSize

if err := devices.saveMetadata(info); err != nil {
// Try to remove unused device
delete(devices.Devices, info.Hash)
return err
}

return devices.growFS(info)
}

func (devices *DeviceSet) growFS(info *devInfo) error {
if err := devices.activateDeviceIfNeeded(info, false); err != nil {
return fmt.Errorf("Error activating devmapper device: %s", err)
}

defer devices.deactivateDevice(info)

fsMountPoint := "/run/docker/mnt"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this path always valid? can users override it somehow? we should take that into account.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should allow users to change this. This directory is only needed since you have to actually mount the device in order to expand it. So we need to temporarily mount the device.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agreed that it should not be user configurable. I am wondering why not create a directory say tmpmnt in devmapper graph driver directory itself. Say /var/lib/docker/devicemapper/mnt/tmpmnt

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rhatdan what do you think of mounting at /var/lib/docker/devicemapper/mnt/tmpmnt (what @rhvgoyal suggested) ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't care, I am just not sure why it matters other then the mount point is not available to non privileged users.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought it is simpler. We don't have to create another directory inside /run. Also I am not sure if all the systems where docker runs have /run or not. And also not sure why there is a need for unprivileged users to be able to see this temporary mount point.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is definitely no reason for non privileged users to see this. /run is a standard, that all distributions use, but I don't really care where we put it as long as it is not /tmp/

if _, err := os.Stat(fsMountPoint); os.IsNotExist(err) {
if err := os.MkdirAll(fsMountPoint, 0700); err != nil {
return err
}
defer os.RemoveAll(fsMountPoint)
}

options := ""
if devices.BaseDeviceFilesystem == "xfs" {
// XFS needs nouuid or it can't mount filesystems with the same fs
options = joinMountOptions(options, "nouuid")
}
options = joinMountOptions(options, devices.mountOptions)

if err := mount.Mount(info.DevName(), fsMountPoint, devices.BaseDeviceFilesystem, options); err != nil {
return fmt.Errorf("Error mounting '%s' on '%s': %s", info.DevName(), fsMountPoint, err)
}

defer syscall.Unmount(fsMountPoint, syscall.MNT_DETACH)

switch devices.BaseDeviceFilesystem {
case "ext4":
if out, err := exec.Command("resize2fs", info.DevName()).CombinedOutput(); err != nil {
return fmt.Errorf("Failed to grow rootfs:%v:%s", err, string(out))
}
case "xfs":
if out, err := exec.Command("xfs_growfs", info.DevName()).CombinedOutput(); err != nil {
return fmt.Errorf("Failed to grow rootfs:%v:%s", err, string(out))
}
default:
return fmt.Errorf("Unsupported filesystem type %s", devices.BaseDeviceFilesystem)
}
return nil
}

func (devices *DeviceSet) setupBaseImage() error {
oldInfo, _ := devices.lookupDeviceWithLock("")

Expand All @@ -1069,9 +1144,8 @@ func (devices *DeviceSet) setupBaseImage() error {
return err
}

if devices.baseFsSize != defaultBaseFsSize && devices.baseFsSize != devices.getBaseDeviceSize() {
logrus.Warnf("devmapper: Base device is already initialized to size %s, new value of base device size %s will not take effect",
units.HumanSize(float64(devices.getBaseDeviceSize())), units.HumanSize(float64(devices.baseFsSize)))
if err := devices.checkGrowBaseDeviceFS(oldInfo); err != nil {
return err
}

return nil
Expand Down Expand Up @@ -2379,6 +2453,7 @@ func NewDeviceSet(root string, doInit bool, options []string, uidMaps, gidMaps [
if err != nil {
return nil, err
}
userBaseSize = true
devices.baseFsSize = uint64(size)
case "dm.loopdatasize":
size, err := units.RAMInBytes(val)
Expand Down
18 changes: 15 additions & 3 deletions docs/reference/commandline/daemon.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -213,11 +213,23 @@ options for `zfs` start with `zfs`.
* `dm.basesize`

Specifies the size to use when creating the base device, which limits the
size of images and containers. The default value is 100G. Note, thin devices
are inherently "sparse", so a 100G device which is mostly empty doesn't use
100 GB of space on the pool. However, the filesystem will use more space for
size of images and containers. The default value is 10G. Note, thin devices
are inherently "sparse", so a 10G device which is mostly empty doesn't use
10 GB of space on the pool. However, the filesystem will use more space for
the empty case the larger the device is.

The base device size can be increased at daemon restart which will allow
all future images and containers (based on those new images) to be of the
new base device size.

Example use:

$ docker daemon --storage-opt dm.basesize=50G

This will increase the base device size to 50G. The Docker daemon will throw an
error if existing base device size is larger than 50G. A user can use
this option to expand the base device size however shrinking is not permitted.

This value affects the system-wide "base" empty filesystem
that may already be initialized and inherited by pulled images. Typically,
a change to this value requires additional steps to take effect:
Expand Down
6 changes: 3 additions & 3 deletions docs/userguide/storagedriver/device-mapper-driver.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -249,11 +249,11 @@ You can use the `lsblk` command to see the device files created above and the `p
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
xvda 202:0 0 8G 0 disk
└─xvda1 202:1 0 8G 0 part /
xvdf 202:80 0 100G 0 disk
xvdf 202:80 0 10G 0 disk
├─vg--docker-data 253:0 0 90G 0 lvm
│ └─docker-202:1-1032-pool 253:2 0 100G 0 dm
│ └─docker-202:1-1032-pool 253:2 0 10G 0 dm
└─vg--docker-metadata 253:1 0 4G 0 lvm
└─docker-202:1-1032-pool 253:2 0 100G 0 dm
└─docker-202:1-1032-pool 253:2 0 10G 0 dm

The diagram below shows the image from prior examples updated with the detail from the `lsblk` command above.

Expand Down
16 changes: 13 additions & 3 deletions man/docker-daemon.8.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -271,12 +271,22 @@ Example use: `docker daemon --storage-opt dm.thinpooldev=/dev/mapper/thin-pool`
#### dm.basesize

Specifies the size to use when creating the base device, which limits
the size of images and containers. The default value is 100G. Note,
thin devices are inherently "sparse", so a 100G device which is mostly
empty doesn't use 100 GB of space on the pool. However, the filesystem
the size of images and containers. The default value is 10G. Note,
thin devices are inherently "sparse", so a 10G device which is mostly
empty doesn't use 10 GB of space on the pool. However, the filesystem
will use more space for base images the larger the device
is.

The base device size can be increased at daemon restart which will allow
all future images and containers (based on those new images) to be of the
new base device size.

Example use: `docker daemon --storage-opt dm.basesize=50G`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As above

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@shishir-a412ed looks like you forgot to address this one? Can you change this to:

Example use:

    docker daemon --storage-opt dm.basesize=50G

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ping @shishir-a412ed can you fix this one, then we are ready to go

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@thaJeztah This is in sync with other examples below.
Let me know if you still want me to change it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@shishir-a412ed fair, alright, lets keep it like this then 👍


This will increase the base device size to 50G. The Docker daemon will throw an
error if existing base device size is larger than 50G. A user can use
this option to expand the base device size however shrinking is not permitted.

This value affects the system-wide "base" empty filesystem that may already
be initialized and inherited by pulled images. Typically, a change to this
value requires additional steps to take effect:
Expand Down