I believe this needs to process k8s gateway resources since Waypoints currently have a network in their key.

Should this be related to the render logic of waypoint template? What I've noticed is that when we set values.global.network and process the waypoint template, only the ISTIO_META_NETWORK environment variable is set; the topology.istio.io/network label is not. This results in the network being evaluated differently for waypoints compared to sidecars.

If the waypoint will have the same behavior as the sidecar injection, then its network will not be affected by this PR. Otherwise, this PR will change the network of the waypoint, even if the ISTIO_META_NETWORK environment variable is set ( from topology.istio.io/network) - which is incorrect.

Good point. Right now it's really broken. We inject that network label on the Pod, but the waypoint logic looks at the k8s Gateway which likely won't have the network label.

I have observed at least once a case where a new Istiod processes the Waypint/Gateway resource first, then the namespace. This results in the ambientindex storing /waypointIP when it should be network/waypointIP.

This results in the ambientindex storing /waypointIP when it should be network/waypointIP.

This makes sense based on my tests with the waypoint network. If we can inject the network label similarly like sidecar injection, then regardless of whether the gateway resource is processed first, the network key of the waypoint will not be affected. I'll take a look to see if we can handle this in the template or deploymentcontroller.

So would we edit the deployment on network change causing a rollout? Network change should be very rare/close to 1-time so I would be okay with this.

This wouldn't. Originally when the network was changed, we had to rollout the deployment to push new network config change, but now we can dynamically update it to ztunnel without a rollout.

Have to be careful that we don't trigger the deployment when istiod hasn't determined the network properly on startup.

I'll verify this.

Any update on "verify" this @hanxiaop ?

@linsun This PR looks good. It works only when the istio-system network label changes, but not during startup. The traffic behavior aligns with the description provided.
However I do see some issues for the ztunnel network setup during the test.

There is one issue with this PR where the ztunnel network may be inaccurately represented in the ztunnel config dump: since it processes all workloads after the network label changes, the ztunnel pod network might also be incorrectly altered - even if its network has already been set. Originally, without this PR, the ztunnel network might also be inaccurate if the pod was updated(like annotation updates), as they both share the same syncPod logic. BTW this won't affect the actual traffic behavior of ztunnel since the network is configured as environmental variables during installation and not changed, only the workload configdump shows the incorrect network value.

With some research, I think this is not directly related to this PR. However, we need to address these issues:

1. We currently lack a uniformed method for setting the ztunnel network during installation, so the network of ztunnel's workload info is altered incorrectly.
2. The behavior of the ztunnel network for network="" is different from that in proxies. In proxies, network="" implies that the network is reachable. In contrast, for ztunnel, it simply means the network is an empty string without any value. If it does not exactly match the network, the traffic will be unrecognized.

May need @stevenctl to confirm if these are problems. The first one is related to this PR.

For ztunnel, now it can only get the network from ENV NETWORK