-
Notifications
You must be signed in to change notification settings - Fork 18.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add IP_NF_MANGLE to check-config.sh #46667
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not 100% sure this is the correct place. There are three places that we create mangle rules (and the last one in my list is the one you got bit by):
- SCTP packet egress:
moby/libnetwork/iptables/iptables.go
Lines 363 to 380 in b85185e
if proto == "sctp" { // Linux kernel v4.9 and below enables NETIF_F_SCTP_CRC for veth by // the following commit. // This introduces a problem when conbined with a physical NIC without // NETIF_F_SCTP_CRC. As for a workaround, here we add an iptables entry // to fill the checksum. // // https://github.com/torvalds/linux/commit/c80fafbbb59ef9924962f83aac85531039395b18 args = []string{ "-p", proto, "--sport", strconv.Itoa(destPort), "-j", "CHECKSUM", "--checksum-fill", } if err := iptable.ProgramRule(Mangle, "POSTROUTING", action, args); err != nil { return err } } - Encrypted overlay networks:
moby/libnetwork/drivers/overlay/encryption.go
Lines 228 to 250 in b85185e
func programMangle(vni uint32, add bool) error { var ( m = strconv.FormatUint(mark, 10) chain = "OUTPUT" rule = append(matchVXLAN(overlayutils.VXLANUDPPort(), vni), "-j", "MARK", "--set-mark", m) a = iptables.Append action = "install" ) // TODO IPv6 support iptable := iptables.GetIptable(iptables.IPv4) if !add { a = iptables.Delete action = "remove" } if err := iptable.ProgramRule(iptables.Mangle, chain, a, rule); err != nil { return fmt.Errorf("could not %s mangle rule: %w", action, err) } return nil } - IPVS load-balancing:
moby/libnetwork/service_linux.go
Lines 547 to 554 in b85185e
for _, iPort := range ingressPorts { var ( protocol = strings.ToLower(PortConfig_Protocol_name[int32(iPort.Protocol)]) publishedPort = strconv.FormatUint(uint64(iPort.PublishedPort), 10) ) rule := []string{"-t", "mangle", addDelOpt, "PREROUTING", "-p", protocol, "--dport", publishedPort, "-j", "MARK", "--set-mark", fwMarkStr} rules = append(rules, rule) }
The latter two are only used with the overlay
network driver, so I'm somewhat inclined to say that this should be scoped to the driver. On the other hand, we do rely on this for SCTP in certain situations, and other netfilter/xtables features are listed here.
I'll defer to @akerouanton and @corhere on whether this should be moved (or checked twice, once for overlay and one for SCTP, etc.) or not.
Oh, also, forgot: Thank you for contributing! It appears your commit message is missing a DCO sign-off, We require all commit messages to have a
There is no need to open a new pull request, but to fix this (and make CI pass), Unfortunately, it's not possible to do so through GitHub's web UI, so this needs You can find some instructions in the output of the DCO check (which can be found Steps to do so "roughly" come down to:
Sorry for the hassle (I wish GitHub would make this a bit easier to do), and let me know if you need help or more detailed instructions! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
SCTP is the transport protocol used in WebRTC so not much of an edge case these days. Just about anyone running a multiplayer browser game, realtime chat or other similar server in a container is going to run into this. I think it makes sense to consider SCTP packet egress a baseline feature, so unless we can work around the kernel issue without it, IP_NF_MANGLE would have to be generally required.
Add IP_NF_MANGLE to "Generally Required" kernel features, since it appears to be necessary for Docker Swarm to work. Closes moby#46636 Signed-off-by: Stephan Henningsen <stephan-henningsen@users.noreply.github.com>
b5ffcc9
to
cf90733
Compare
Yes, I had a feeling I would fail here. I found out about the sign-off too late, panicked, and added it in the PR comment ;) I believe it's properly signed off now. |
Add IP_NF_MANGLE to "Generally Required" kernel features, since it appears to be necessary for Docker Swarm to work.
Closes #46636
- What I did
Used the
check-config.sh
script for my custom Linux to be compatible with Docker. And it mostly works, except connecting to local services in swarm mode!I tested on a single-node swarm cluster and started a simple web service:
The last command will fail to connect to the published port:
curl: (7) Failed to connect to 127.0.0.1 port 8088 after 0 ms: Error
- How I did it
I isolated the cause for the connection refused to a single kernel module, using lots of
lsmod
anddiff
between a working system and my broken system, and moving various kernel modules out of the way and back again. It took some time.- How to verify it
First establish a failing test case:
sudo mv /lib/modules/6.1.51/kernel/net/ipv4/netfilter/iptable_mangle.ko{,.disabled}
(or equivalent) to move kernel modules out of the way.docker swarm init --default-addr-pool 10.22.0.0/16
docker service create --publish published=8088,target=80 --name=www nginx:1.25.2-alpine
curl 127.0.0.1:8088
Expected: Connected, HTML is shown .
Actual: Connection failed.
Run
lsmod | grep iptable_mangle
and confirm the module isn't loaded.Cleanup:
docker service rm www
docker swarm leave --force
Now confirm that the module is indeed required by swarm:
sudo mv /lib/modules/6.1.51/kernel/net/ipv4/netfilter/iptable_mangle.ko{.disabled,}
to reinstall kernel moduledocker swarm init --default-addr-pool 10.22.0.0/16
lsmod | grep iptable_mangle
and confirm the module is loaded but unused.docker service create --publish published=8088,target=80 --name=www nginx:1.25.2-alpine
lsmod | grep iptable_mangle
and confirm the module is in fact loaded and in use.curl 127.0.0.1:8088
Expected/Actual:
iptable_mangle
is listed, HTML is shown.- Description for the changelog
Add IP_NF_MANGLE to the list of "Generally Required" because it is required by Swarm.
- A picture of a cute animal (not mandatory but encouraged)
Signed-off-by: Stephan Henningsen stephan+github@asklandd.dk