always create network with ipv6 #1526

BenTheElder · 2020-04-29T09:06:09Z

/shrug
turns out "enabling ipv6" is only for the default bridge ...?
so we can just always create the kind network with ipv6 enabled.

k8s-ci-robot · 2020-04-29T09:06:16Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: BenTheElder

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [BenTheElder]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

BenTheElder · 2020-04-29T09:07:33Z

FYI @amwat lol 😂

BenTheElder · 2020-04-29T09:10:10Z

/hold
checking on some other systems

aojea · 2020-04-29T09:15:10Z

pkg/cluster/internal/providers/docker/network.go

-		return exec.Command("docker", "network", "create", "-d=bridge", "--ipv6", "--subnet=fc00:db8:2::/64", name).Run()
-	}
-	return exec.Command("docker", "network", "create", "-d=bridge", name).Run()
+	return exec.Command("docker", "network", "create", "-d=bridge", "--ipv6", "--subnet=fc00:db8:2::/64", name).Run()


we can't create the same range in all networks, we should add some bytes hashing the name or something

That is true, however this is not new to this PR, and currently there is a single caller with a fixed value.
There's an existing TODO here.
I'm limiting this PR to just resolving the aspect of "ipv4 clusters before ipv6 clusters => bad state"

ups, I forgot that was hardcoded now XD

what happens if the host does not have ipv6?
the command will fail?

#1526 (comment)

bad wording, not that IPv6, the host IPv6
net.ipv6.conf.all.disable_ipv6 = 1

it still works but i'm not sure that setting is sufficient to test e.g. lack of modules

$ sudo sysctl net.ipv6.conf.all.disable_ipv6 net.ipv6.conf.all.disable_ipv6 = 1 $ docker exec kind-control-plane ip addr 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: veth94b73da2@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 82:b9:38:07:5d:5b brd ff:ff:ff:ff:ff:ff link-netns cni-7e2721d4-185b-b646-b43f-41f797ddfa65 inet 10.244.0.1/32 brd 10.244.0.1 scope global veth94b73da2 valid_lft forever preferred_lft forever 3: vetheb4ad147@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 4e:c3:91:c2:4a:c0 brd ff:ff:ff:ff:ff:ff link-netns cni-ad9efa8f-e278-bed2-a72d-f634c116ede3 inet 10.244.0.1/32 brd 10.244.0.1 scope global vetheb4ad147 valid_lft forever preferred_lft forever 4: veth39517578@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 02:c6:7a:67:8c:b0 brd ff:ff:ff:ff:ff:ff link-netns cni-fbf3ca43-0688-1ff0-09ba-c90621b6407e inet 10.244.0.1/32 brd 10.244.0.1 scope global veth39517578 valid_lft forever preferred_lft forever 17: eth0@if18: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 02:42:ac:12:00:02 brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 172.18.0.2/16 brd 172.18.255.255 scope global eth0 valid_lft forever preferred_lft forever inet6 fc00:db8:2::2/64 scope global nodad valid_lft forever preferred_lft forever inet6 fe80::42:acff:fe12:2/64 scope link valid_lft forever preferred_lft forever

that's fair enough, I think that we can move forward with this :)

i'm definitely nervous about that, maybe I can fire up a COS GKE node and see if it still works

BenTheElder · 2020-04-29T09:15:30Z

sanity checked that this PR works fine on docker desktop (macOS), ubuntu 20.04 / docker-ce, neither with ipv6 """enabled""".

BenTheElder · 2020-04-29T09:17:05Z

/hold cancel

BenTheElder · 2020-04-29T09:18:45Z

we'll want to update the docs after the release, you don't need to muck with daemon config to do ipv6 anymore 🎉

we can also consider removing that hack from KRTE

BenTheElder · 2020-04-29T16:21:01Z

/override pull-kind-conformance-parallel-1-12
/override pull-kind-conformance-parallel-1-13
/override pull-kind-conformance-parallel-1-14
will fix CI in a bit.

k8s-ci-robot · 2020-04-29T16:21:06Z

@BenTheElder: Overrode contexts on behalf of BenTheElder: pull-kind-conformance-parallel-1-12, pull-kind-conformance-parallel-1-13, pull-kind-conformance-parallel-1-14

In response to this:

/override pull-kind-conformance-parallel-1-12
/override pull-kind-conformance-parallel-1-13
/override pull-kind-conformance-parallel-1-14
will fix CI in a bit.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

BenTheElder · 2020-04-29T16:21:43Z

/test pull-kind-e2e-kubernetes

BenTheElder · 2020-04-29T16:53:10Z

/retest

amwat · 2020-04-29T21:49:58Z

hmm are we sure?
atleast their docs say you still need to enable it on the daemon before creating any networks with ipv6
https://docs.docker.com/network/bridge/#use-ipv6

BenTheElder · 2020-04-29T21:54:56Z

$ ps aux | grep docker
root       12194  0.0  0.2 1824648 96092 ?       Ssl  11:32   0:07 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
root       13538  0.0  0.0 548940  3572 ?        Sl   11:33   0:00 /usr/bin/docker-proxy -proto tcp -host-ip 127.0.0.1 -host-port 35601 -container-ip 172.18.0.2 -container-port 6443
root       13545  0.0  0.0 109104  6564 ?        Sl   11:33   0:00 containerd-shim -namespace moby -workdir /var/lib/containerd/io.containerd.runtime.v1.linux/moby/0705010047ec8100ec5ac99f6a5d0a24e389234cd4c5af275961986aaca92bf5 -address /run/containerd/containerd.sock -containerd-binary /usr/bin/containerd -runtime-root /var/run/docker/runtime-runc
benthee+   51134  0.0  0.0   9032  2604 pts/1    S+   14:48   0:00 grep --color=auto docker


$ sudo cat /etc/docker/daemon.json
[sudo] password for bentheelder: 
cat: /etc/docker/daemon.json: No such file or directory

BenTheElder · 2020-04-29T22:00:06Z

i'm having difficulty proving it, but AFAICT docker ipv6 daemon options are just for the default bridge. (fixed-cidr definitely is)

aojea · 2020-04-29T22:11:06Z

'm having difficulty proving it, but AFAICT docker ipv6 daemon options are just for the default bridge. (fixed-cidr definitely is)

yeah, I think that they've moved to libnetwork the new networking and those are legacy options, but is just my impression

BenTheElder · 2020-04-29T23:29:08Z

I brought up a multi-node ipv6 cluster on a machine with default docker (no config file).

BenTheElder · 2020-04-30T00:34:11Z

it worked on a COS node (1.14.10-gke.27), though it appears the ip6tables modules are loaded on my clean cluster 🤷

BenTheElder · 2020-04-30T00:36:28Z

/hold cancel

BenTheElder · 2020-04-30T00:36:32Z

/retest

amwat

/lgtm

marcosnils · 2020-05-01T21:05:10Z

Hey there,

This just broke my kind deployment since I have ipv6 disable from ipv6.disable=1 in grub config and now I can't create clusters anymore :(. And it doesn't seem to be a way that I can avoid kind try to create an ipv6 docker network :(

BenTheElder · 2020-05-01T21:06:43Z

I tested this with ipv6 disable=1 (see discussion above), can you share more details about your environment?

BenTheElder · 2020-05-01T21:07:21Z

As an immediate workaround you can:

docker network rm kind && docker network create kind (skip the other flags)

BenTheElder · 2020-05-01T21:07:43Z

kind will only create the network if it does not exist.

aojea · 2020-05-01T21:08:57Z

I tested this with ipv6 disable=1 (see discussion above), can you share more details about your environment?

Is because it has disable it in grub .... I'm at mobile now but I think we should to probe ipv6 or try to create the network in ipv6 and fall back to ipv4

BenTheElder · 2020-05-01T21:15:27Z

i'll send a patch and we can cut an 0.8.1 ...

marcosnils · 2020-05-01T21:26:21Z

Is because it has disable it in grub .... I'm at mobile now but I think we should to probe ipv6 or try to create the network in ipv6 and fall back to ipv4

^ this. ipv6 is disabled at kernel level and not sysctl. I liked the previous way where you could configure your ipFamily to ipv4 in config.yaml and kind wouldn't try to create an ipv6 network. Can we bring that back?.

Btw, thanks for the almost immediate reply and for the amazing project :)

marcosnils · 2020-05-01T21:30:20Z

Created #1544 to keep track.

BenTheElder · 2020-05-01T21:35:22Z

^ this. ipv6 is disabled at kernel level and not sysctl. I liked the previous way where you could configure your ipFamily to ipv4 in config.yaml and kind wouldn't try to create an ipv6 network. Can we bring that back?.

🤦

thanks, I think we can come up with something equivilant. the tricky part is that the dockerd may not be running on the same machine as kind...

BenTheElder · 2020-05-01T21:35:37Z

Btw, thanks for the almost immediate reply and for the amazing project :)

🙃 thanks

BenTheElder · 2020-05-01T22:55:16Z

#1545

always create network with ipv6

6f55f4b

k8s-ci-robot added ¯\_(ツ)_/¯ ¯\\\_(ツ)_/¯ cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Apr 29, 2020

k8s-ci-robot requested review from krzyzacy and neolit123 April 29, 2020 09:06

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Apr 29, 2020

BenTheElder added this to the v0.8.0 milestone Apr 29, 2020

k8s-ci-robot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Apr 29, 2020

BenTheElder mentioned this pull request Apr 29, 2020

Release v0.8.0 #1475

Closed

BenTheElder requested review from munnerz and aojea and removed request for krzyzacy April 29, 2020 09:07

k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Apr 29, 2020

aojea reviewed Apr 29, 2020

View reviewed changes

k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Apr 29, 2020

aojea mentioned this pull request Apr 29, 2020

Don't configure ipv6 docker daemon kubernetes/test-infra#17434

Merged

aojea mentioned this pull request Apr 29, 2020

generate ipv6 subnet based on the network name #1527

Closed

amwat reviewed Apr 30, 2020

View reviewed changes

k8s-ci-robot assigned amwat Apr 30, 2020

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Apr 30, 2020

k8s-ci-robot merged commit 2cbfa0a into kubernetes-sigs:master Apr 30, 2020

BenTheElder deleted the network-ipv6-always branch April 30, 2020 01:10

marcosnils mentioned this pull request May 1, 2020

Can't create ipv4 clusters if ipv6 is disabled at kernel level #1544

Closed

always create network with ipv6 #1526

always create network with ipv6 #1526

Conversation

BenTheElder commented Apr 29, 2020

k8s-ci-robot commented Apr 29, 2020

BenTheElder commented Apr 29, 2020

BenTheElder commented Apr 29, 2020

Choose a reason for hiding this comment

BenTheElder Apr 29, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

BenTheElder commented Apr 29, 2020

BenTheElder commented Apr 29, 2020

BenTheElder commented Apr 29, 2020

BenTheElder commented Apr 29, 2020

k8s-ci-robot commented Apr 29, 2020

BenTheElder commented Apr 29, 2020

BenTheElder commented Apr 29, 2020

amwat commented Apr 29, 2020

BenTheElder commented Apr 29, 2020

BenTheElder commented Apr 29, 2020

aojea commented Apr 29, 2020 • edited Loading

BenTheElder commented Apr 29, 2020

BenTheElder commented Apr 30, 2020

BenTheElder commented Apr 30, 2020

BenTheElder commented Apr 30, 2020

amwat left a comment

Choose a reason for hiding this comment

marcosnils commented May 1, 2020 • edited Loading

BenTheElder commented May 1, 2020

BenTheElder commented May 1, 2020

BenTheElder commented May 1, 2020

aojea commented May 1, 2020

BenTheElder commented May 1, 2020

marcosnils commented May 1, 2020 • edited Loading

marcosnils commented May 1, 2020

BenTheElder commented May 1, 2020

BenTheElder commented May 1, 2020

BenTheElder commented May 1, 2020

BenTheElder Apr 29, 2020 •

edited

Loading

aojea commented Apr 29, 2020 •

edited

Loading

marcosnils commented May 1, 2020 •

edited

Loading

marcosnils commented May 1, 2020 •

edited

Loading