Open
Description
Description:
We ran into this issue that seems definitely unusual. I have considered if this is caused due to underlying host not supporting IPv6 or working in AWS bridge mode but not able to reproducible locally.
A NACK thrown in Envoy version v1.27.
failureReason:Error adding/updating listener(s) egress: malformed IP address: 2600:f0f0:0:0:0:0:0:1
As per NACK Envoy couldn't validate that this is a proper IPv6 address which was provided in filter chain match like
"filter_chain_match": {
"prefix_ranges": [
{
"address_prefix": "127.255.0.1",
"prefix_len": 32
},
{
"address_prefix": "2600:f0f0:0:0:0:0:0:1",
"prefix_len": 128
}
],
I believe the IPv6 address validation is done in
envoy/source/common/network/utility.cc
Line 117 in 83e604a
Repro steps:
Not able to reproduce as this was noted on customer's environment.
Config:
Same as the one mentioned above.
{
"name": "egress",
"active_state": {
"version_info": "1",
"listener": {
"@type": "type.googleapis.com/envoy.config.listener.v3.Listener",
"name": "egress",
"address": {
"socket_address": {
"address": "127.0.0.1",
"port_value": 59632
}
},
"filter_chains": [
{
"filter_chain_match": {
"prefix_ranges": [
{
"address_prefix": "127.255.0.1",
"prefix_len": 32
},
{
"address_prefix": "2600:f0f0:0:0:0:0:0:1",
"prefix_len": 128
}
],
"destination_port": 8156
},
"filters": [
....Logs:
Logs on Control Plane
26 09 2023 20:20:50,731 [WARN] envoy.common.EnvoyNackHandler: Received NACK(failureReason:Error adding/updating listener(s) egress: malformed IP address: 2600:f0f0:0:0:0:0:0:1) for resource(type:MetaType(identity=envoy.listener), version:, nonce:488d0f4a-425a-467c-99ac-1ddd0a0e9543, names:[]) from Envoy(version:v1.27.0.0)
Activity
zuercher commentedon Oct 2, 2023
I think we need more information to determine whether this anything that we'd want to address. Do you know what OS, version, etc was in use?
suniltheta commentedon Oct 3, 2023
OS: AL2023 (based on Fedora Linux distribution), kernel 6.1.52-71.125.amzn2023.x86_64
It was using AWS ECS networking in Bridge mode - I am not sure if this network mode would influence filter_chain_match IPs.
zuercher commentedon Oct 4, 2023
Looking at the code it seems this must be getaddrinfo returning an error. That's about as far as I can get with the time I have.
suniltheta commentedon Oct 4, 2023
Thanks for taking a look.
If this getaddrinfo functionality is provided by glibc, want to understand when Envoy does
Api::OsSysCallsSingleton::get().getaddrinfo(call, would it be handled with some library already include with Envoy static binary or would Envoy make call to underlying OS and the glibc functionality has to be supported in the OS itself?In the above custom Envoy image that is used the binary & the image is built on AmazonLinux2 (CentOS flavor). But the container is run on AmazonLinux2023 host which is Fedora based. So I am wonder if there is some incompatibility that would cause this problem (rarely).
zuercher commentedon Oct 6, 2023
From my experience, glibc is the one library that Envoy links dynamically. If the OS version is older than the compile version it'll fail to start.
ondrejmo commentedon Dec 18, 2024
I have a similar issue with Cilium 1.16.5, where the
cilium-envoy-...pods crash claiming::1is a malformed ip. All affected systems have IPv6 support disabled via a grub flag.cilium/cilium@c4b1379
sayboras commentedon Dec 19, 2024
👋
For the information, Cilium 1.16.5 is using envoy 1.30.8/Distribution/RELEASE/BoringSSL.
envoy: Configure internal address config based on IP family
23 remaining items