Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

libnetwork vendoring #18775

Merged
merged 1 commit into from
Dec 26, 2015
Merged

libnetwork vendoring #18775

merged 1 commit into from
Dec 26, 2015

Conversation

mrjana
Copy link
Contributor

@mrjana mrjana commented Dec 18, 2015

Vendoring libnetwork @ 9f0563ea8f430d8828553aac97161cbff4056436

Brings in:
* Support for overlay network driver in 3.10+ kernels
* Freebsd compilation fixes
* Remove .dockerinit dependency
* IPAM driver capability support
* Network internal mode support
* Misc. fixes

Fixes #18657

Signed-off-by: Jana Radhakrishnan mrjana@docker.com

@cpuguy83
Copy link
Member

Does it really close #18355? Seems like there's still stuff to do in engine after this is merged.

@mrjana
Copy link
Contributor Author

mrjana commented Dec 18, 2015

@cpuguy83 I thought libnetwork was the only place referencing it. If not I will remove it.

@LK4D4
Copy link
Contributor

LK4D4 commented Dec 18, 2015

Yeah, it doesn't close #18355, docker referencing .dockerinit in many places.
Apart from misspelling in the commit message (capabAlity) LGTM for me.

@mavenugo
Copy link
Contributor

LGTM

It will also be good to update the doc userguide/networking/get-started-overlay.md by removing the pre-req

A host with a 3.16 kernel version or higher.

@mrjana
Copy link
Contributor Author

mrjana commented Dec 18, 2015

Ok fixed and updated.

@clinta
Copy link

clinta commented Dec 19, 2015

This says "Freebsd compilation fixes", should this close #18249?

Because I still get netlink errors compiling this fork on FreeBSD:

$ git remote -v
origin  https://github.com/mrjana/docker.git (fetch)
origin  https://github.com/mrjana/docker.git (push)
$ git status
On branch mh
Your branch is up-to-date with 'origin/mh'.
nothing to commit, working directory clean
$ git log -1
ESC[33mcommit 5b450e97387e9f87550b5cb58b3d4efa85394494ESC[m
Author: Jana Radhakrishnan <mrjana@docker.com>
Date:   Fri Dec 18 11:03:57 2015 -0800

    libnetwork vendoring

    Vendoring libnetwork @ df3f8a10796e2173732aeafa41af45f970ced9f6

    Brings in:
        * Support for overlay network driver in 3.10+ kernels
        * Freebsd compilation fixes
        * Remove .dockerinit dependency
        * IPAM driver capability support

    Signed-off-by: Jana Radhakrishnan <mrjana@docker.com>
$ export AUTO_GOPATH=1 && ./hack/make.sh binary
# WARNING! I don't seem to be running in the Docker container.
# The result of this command might be an incorrect build, and will not be
#   officially supported.
#
# Try this instead: make all
#

date: illegal option -- -
usage: date [-jnRu] [-d dst] [-r seconds] [-t west] [-v[+|-]val[ymwdHMS]] ... 
            [-f fmt date | [[[[[cc]yy]mm]dd]HH]MM[.ss]] [+format]
bundles/1.10.0-dev already exists. Removing.

---> Making bundle: binary (in bundles/1.10.0-dev/binary)
Building: bundles/1.10.0-dev/binary/docker-1.10.0-dev
vendor/src/github.com/vishvananda/netlink/filter.go:6:2: no buildable Go source files in /home/clint/docker/vendor/src/github.com/vishvananda/netlink/nl

@cpuguy83
Copy link
Member

Having all kinds of issues with this.

root@test2:~# docker run -it --rm --net=abcd busybox
WARN[0008] failed to cleanup ipc mounts:
failed to umount /var/lib/docker/containers/007c412e965c726a57d2d28b5cbf86512147b99e6f5ff4022a3212c5de3d1d59/shm: no such file or directory
failed to umount /var/lib/docker/containers/007c412e965c726a57d2d28b5cbf86512147b99e6f5ff4022a3212c5de3d1d59/mqueue: no such file or directory
ERRO[0008] Handler for POST /v1.21/containers/007c412e965c726a57d2d28b5cbf86512147b99e6f5ff4022a3212c5de3d1d59/start returned error: subnet sandbox join failed for "10.0.0.0/24": error creating vxlan interface: file exists
Error response from daemon: subnet sandbox join failed for "10.0.0.0/24": error creating vxlan interface: file exists

I've reset the k/v store, the docker storge dir, /var/run/docker/netns... Keep getthing this error.
In addition, /var/run/docker/netns is _not_cleaned up properly when this failure occurs.

@cpuguy83
Copy link
Member

root@test2:~# docker info
Containers: 0
Images: 1
Server Version: 1.10.0-dev
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 2
 Dirperm1 Supported: false
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 3.13.0-71-generic
Operating System: Ubuntu 14.04.3 LTS
CPUs: 1
Total Memory: 490 MiB
Name: test2
ID: 7T2U:JLMY:Z26F:WGWD:E3NK:WL24:R5FU:MZVB:HAFU:WSLA:BQFV:UQDA
WARNING: No swap limit support
Cluster store: etcd://<redacted>:4001
Cluster advertise: <redacted>:2375

@mavenugo
Copy link
Contributor

@cpuguy83 i dont think it is related to this change. as we have discussed in the past, if the user decides to cleanup the internal states (/var/lib/docker/netns or the kv-store states), then the user must take care of cleaning up all the relevant internal states and unmount the devices (in this case it is the vxlan).
Since it seems there has been a partial irrecoverable cleanup, could you restart the machine and try it out again ?

@cpuguy83
Copy link
Member

@mavenugo I'll play some more, but I did it twice with fresh VM's on DO.

@mavenugo
Copy link
Contributor

@cpuguy83 thanks for the confirmation. If its a fresh VM, then its a problem. I will try it as well.

@mavenugo
Copy link
Contributor

@cpuguy83 @mrjana I could not repro the error creating vxlan interface: file exists issue on a fresh installation (vm with kernel < 3.16). But, I did reproduce it using the following steps

  1. run docker 1.9.0 (or master without this fix) & create an overlay network
  2. run a container in this network, which fails due to the kernel unsupported issue as expected
  3. stop the docker daemon, upgrade the daemon with this PR & start the upgraded daemon
  4. run a container on the same network, this fails with error creating vxlan interface: file exists

Restarting the vm fixed the issue though.

I guess we have to fix it by cleaning up the vxlan device on daemon start ?
@mrjana WDYT ?

@mavenugo
Copy link
Contributor

@cpuguy83 @mrjana seen again here : #18770 (comment)

@mrjana
Copy link
Contributor Author

mrjana commented Dec 22, 2015

@cpuguy83 @mavenugo There was name change in the vxlan interface before this change and after this change. We moved to a predictable name change. Because of this name change stale vxlan interfaces which were created by the previous versions were not being deleted properly. I've added a better cleanup logic which cleans up based on VNI and works better. PTAL

@aboch
Copy link
Contributor

aboch commented Dec 22, 2015

This vendoring also fixes #18657

@mavenugo
Copy link
Contributor

@mrjana it solved the issue I reproduced. But am facing another issue with iptables (I will confirm if this is related to this change or not).

Vendoring libnetwork @ 9f0563ea8f430d8828553aac97161cbff4056436

Brings in:
    * Support for overlay network driver in 3.10+ kernels
    * Freebsd compilation fixes
    * Remove .dockerinit dependency
    * IPAM driver capability support
    * Network internal mode support
    * Misc. fixes

Signed-off-by: Jana Radhakrishnan <mrjana@docker.com>
@mrjana
Copy link
Contributor Author

mrjana commented Dec 22, 2015

@clinta This vendoring in PR just brings in some freebsd compilation fixes but we don't know if it fixes all freebsd compilation errors. We haven't instituted freebsd cross compilation checks in libnetwork CI yet.

@mavenugo
Copy link
Contributor

@mrjana I tried the latest vendored in and things work just fine.
@cpuguy83 would you like to give it a go as well ?

LGTM.

@icecrime
Copy link
Contributor

AFAICT this is LGTM from me assuming CI is green.

One thing I'd like to point out: as we split out the Engine and create more repos, we will soon be enforcing that vendoring of our docker's dependencies will go through tags (cc @anusha-ragunathan).

@@ -16,7 +16,6 @@ network. Docker Engine supports multi-host networking out-of-the-box through the
`overlay` network driver. Unlike `bridge` networks, overlay networks require
some pre-existing conditions before you can create one. These conditions are:

* A host with a 3.16 kernel version or higher.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't we continue mentioning a minimal kernel version nevertheless?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is going to work on all Docker Engine supported kernel versions. There are no specific minimal kernel version needs for overlay driver. Should we still specify a minimal kernel version?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but there is the kernel version in the commit msg: Support for overlay network driver in 3.10+ kernels

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wenchma that's because 3.10 is the minimum supported kernel version for docker engine. Pls refer to https://docs.docker.com/engine/installation/ubuntulinux/ and other distributions for the pre-req documentation. I guess this covers all the docker features including overlay networking.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome! It's perfect then thanks :-)

@icecrime
Copy link
Contributor

Related?

----------------------------------------------------------------------
FAIL: docker_cli_network_unix_test.go:837: DockerNetworkSuite.TestDockerNetworkHostModeUngracefulDaemonRestart

[d26442272] waiting for daemon to start
[d26442272] daemon started
[d26442272] exiting daemon
[d26442272] waiting for daemon to start
[d26442272] daemon started
docker_cli_network_unix_test.go:858:
    c.Assert(err, checker.IsNil)
... value *exec.ExitError = &exec.ExitError{ProcessState:(*os.ProcessState)(0xc82049f360)} ("exit status 1")

[d26442272] exiting daemon

@mrjana
Copy link
Contributor Author

mrjana commented Dec 22, 2015

@icecrime I don't think it's related. As a matter of fact the same test passed in experimental. I am going to restart the tests.

@thaJeztah
Copy link
Member

Oh, it's all green!

docs LGTM

thaJeztah added a commit that referenced this pull request Dec 26, 2015
@thaJeztah thaJeztah merged commit d4270c1 into moby:master Dec 26, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

IPv6 gateway is not configured when using custom networks
10 participants