Is this a BUG REPORT or FEATURE REQUEST?:

/kind feature
/sig azure

What happened:

Mixed protocols LoadBalacer services are not supported by apiserver. It will decline those services with error "cannot create an external load balancer with non-TCP/UDP ports":

As stated in #64471 ,

It's not that "not all implementations support it", it's that almost all implementations (as far as I know) do NOT support it. I would like to be able to allow this, but it would be just another source of support issues when people try to bridge environments.

This could be an implementation-specific annotation, which could build support for an argument that it IS portable, but to the best of my knowledge it is not.

We will add a new annotation service.beta.kubernetes.io/azure-load-balancer-mixed-protocols=true to support this feature. When annotated, two load balancer rules with different protocols (TCP and UDP) will be created for the service.

e.g.

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/azure-load-balancer-mixed-protocols: "true"
  name: web
  namespace: default
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: web
  sessionAffinity: None
  type: LoadBalancer

What you expected to happen:

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?:

Refer #23880 and #64471.

Environment:

• Kubernetes version (use kubectl version):
• Cloud provider or hardware configuration:
• OS (e.g. from /etc/os-release):
• Kernel (e.g. uname -a):
• Install tools:
• Others: