libnetwork: support encrypted overlay networks on systems without xt_u32 kernel module #45118
Conversation
The overlay-network encryption code is woefully under-documented, which is especially problematic as it operates on under-documented kernel interfaces. Document what I have puzzled out of the implementation for the benefit of the next poor soul to touch this code. Signed-off-by: Cory Snider <csnider@mirantis.com>
The iptables rule clause used to match on the VNI of VXLAN datagrams looks like line noise to the uninitiated. It doesn't help that the expression is repeated twice and neither copy has any commentary. DRY out the rule builder to a common function, and document what the rule does and how it works. Signed-off-by: Cory Snider <csnider@mirantis.com>
corhere
added
kind/enhancement
|
cc: @akerouanton |
Some newer distros such as RHEL 9 have stopped making the xt_u32 kernel module available with the kernels they ship. They do ship the xt_bpf kernel module, which can do everything xt_u32 can and more. Add an alternative implementation of the iptables match rule which uses xt_bpf to implement exactly the same logic as the u32 filter using a BPF program. Allow the implementation to be selected at compile time using a build tag. Signed-off-by: Cory Snider <csnider@mirantis.com>
corhere
force-pushed
the
libnet/overlay-bpf
branch
from
695b0fd to
307afda
Compare
|
I thought we discussed switching on this at runtime? |
|
This is an MVP to keep the review focus on the iptables rules themselves without getting sidetracked by runtime-switching concerns. |
|
@corhere / @neersighted , |
|
No, this PR was not merged. I'm not sure why you would want to backport the fixes to 20.10.16 rather than just upgrading to 20.10.24+ which have the vulnerabilities fixed. The fixes were backported to moby/libnetwork in moby/libnetwork@0d7f33e, which was vendored into the 20.10 branch in bbec670. |
@corhere Thanks for the details, Today, I have tried one more thing to just upgrade moby/libnetwork SRCREV_libnetwork = "0d7f33e190f5c52414e18d7d76310554b704ba9c" which has the fixes for all these CVE-2023-28840, CVE-2023-28841 & CVE-2023-28842. It has been built successfully without any error/issue and I could see all the changes till here moby/libnetwork@0d7f33e in moby/libnetwork source code. Is it fine doing SRCREV_libnetwork changes to fix all these CVEs mentioned as per my current "go" version 1.17.13 condition ? |
|
"Fine" is not a binary state -- you are able to compile, which appears to meet your needs, with a different version of libnetwork which contains the patches. That's good. As the Go standard library is shipped with the compiler, and only N-1 versions are supported, you really should consider updating your Go compiler to get CVE fixes for the standard library, if scanner results are your primary concern. The Go compatibility promise will (ideally) make upgrading the compiler transparent to any other Go code you are building, though in practice it sometimes is moderately more complex than that. Still, you have effectively forked moby/moby, and since we can only provide support in this repository for the code as it exists in this repository, you are in uncharted territory. It's probably "fine," but no one can reason about the end result but you. |
@neersighted , Thanks for your comments. |
The
xt_u32kernel module provides the xtablesu32match, which allows firewall rules to be written which match on the contents of the raw packet. The libnetwork overlay network driver creates firewall rules usingu32matches to match the VXLAN ID (VNI) of VXLAN datagrams so that the rules are selectively applied only to packets associated with VXLANs for the overlay networks managed by the driver. RHEL/CentOS 8 no longer ship thext_u32kernel module by default, and RHEL/CentOS 9 do not build it at all.- What I did
I added an alternative implementation of the iptables rules to the overlay driver which does not depend on the
xt_u32kernel module. It can be selected at compile-time using thelibnetwork_overlay_bpfbuild tag.- How I did it
I wrote a BPF filter which is a drop-in replacement for the
u32expression and used that filter with thext_bpfkernel module to match using the filter in the iptables rules.- How to verify it
Spin up a Swarm cluster with at least two nodes, at least one of which is v23.0.1 (to test back-compatibility). Create an encrypted overlay network, start a container on each node attached to the network, and verify that processes on one node's container can connect to processes listening in the other node's container.
- Description for the changelog
- A picture of a cute animal (not mandatory but encouraged)