Skip to content

Gateway XDS Skipped for Tagged Gateways #54458

Open
@therealmitchconnors

Description

@therealmitchconnors

Edit: this bug originally covered only the status of a waypoint not being updated, but on further investigation, it was discovered that this impacts all gateways, and causes them to be improperly programmed, not just to lack status.

Is this the right place to submit this?

  • This is not a security vulnerability or a crashing bug
    This is not a question about how to use Istio

Bug Description

When using a gateway with an istio.io/rev label pointing to a tag rather than a revision, the gateway is ignored during the construction of the pushContext, causing incorrect XDS to be delivered to it, as well as preventing the status field from being set (if present). This issue affects both Istio Gateways and K8s Gateways across all methods of orchestration (helm, injection, managed, unmanaged).

Version

This issue was discovered on 1.24.1, but appears to have been present since tags were added to the codebase, sometime before 1.16.

client version: 1.24.1
control plane version: 1.24.1
data plane version: 1.24.1 (7 proxies)

Additional Information

to reproduce:

istioctl install --set profile=ambient --revision 1-24-1
istioctl tag set rapid --revision 1-24-1
istioctl waypoint apply --revision rapid --for all

Notice that status is not updated on the waypoint gateway resource. Additionally, run istioctl pc -o json against the gateway pod, and store this as tagged.json. Finally, run kubectl label deployment waypoint istio.io/rev=1-24-1 --overwrite to use the explicit revision rather than the tag. Status should now be set, and the above pc command should have several thousand lines of significant difference frmo tagged.json.

Activity

therealmitchconnors

therealmitchconnors commented on Dec 23, 2024

@therealmitchconnors
ContributorAuthor

This appears to be because the gateway status controller is listing gateways from the gateway controller, which is limited to gateways in this revision, while it is intended to write status for all gateways... that being said, I can't see how this ever worked, and I strongly suspect it worked in the past.

pilot/pkg/config/kube/crdclient/client.go:L71

keithmattix

keithmattix commented on Dec 23, 2024

@keithmattix
Contributor

while it is intended to write status for all gateways

Even gateways that don't belong to this istiod's revision? That doesn't seem expected to me?

added this to the 1.25 milestone on Jan 17, 2025
changed the title Waypoint Status empty if using tags Gateway XDS Skipped for Tagged Gateways on Jan 17, 2025
therealmitchconnors

therealmitchconnors commented on Jan 17, 2025

@therealmitchconnors
ContributorAuthor

Even gateways that don't belong to this istiod's revision?

No, you are correct. Only gateways that belong to this revision of Istiod, based on their use of revision or tags in labels on the gateway or namespace.

therealmitchconnors

therealmitchconnors commented on Jan 17, 2025

@therealmitchconnors
ContributorAuthor

In response to the question of how we could have not noticed this for so long, it appears most documentation about tag usage has historically pointed to using tags on namespaces, which still works. This bug is only triggered when a gateway object is explicitly labeled with a tag.

JanaSabuj

JanaSabuj commented on Feb 13, 2025

@JanaSabuj

I am facing a similar issue I think. I have the istio.io/rev=1-23 label on the gtw object and the waypoint that spawns up is not configured properly. I see that clusters and endpoints are missing (for the service I intended the waypoint to be for) in istioctl pc clusters/endpoints <waypoint>.

Slack thread for the issue: https://istio.slack.com/archives/C041EQL1XMY/p1738912239987509

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions

    Gateway XDS Skipped for Tagged Gateways · Issue #54458 · istio/istio