Skip to content

AWS: Proposed set of annotations for ELB "advanced features"Β #24978

Closed
@justinsb

Description

There are a few PRs in flight to use annotations to add extra features to the ELBs that k8s sets up (DNS, SSL, Proxy Protocol), and I've written up here what I think a consistent set of those annotations would be. I think the PRs that we have are pretty close to what we have here.

Proxy protocol support:

  • service.beta.kubernetes.io/aws-load-balancer-proxy-protocol=*

This is required to get the client IP, but the service must be aware of the extra header bytes that are sent. Likely our ingress controllers will turn this on and then set an HTTP X-Forwarded-For header, and also some advanced services might set it also to indicate they "speak proxy protocol".

cc @williamsandrew and cf #24569

SSL support at the ELB:

  • service.beta.kubernetes.io/aws-load-balancer-ssl-cert=arn:aws:acm:us-east-1:123456789012:certificate/12345678-1234-1234-1234-123456789012
  • service.beta.kubernetes.io/aws-load-balancer-ssl-ports=* (or e.g. https)

If the user wants the ELB to do SSL decryption, they should set these annotations. ssl-cert specifies the certificate to use; ssl-ports specifies on which ports we should enable SSL. ssl-ports can be optional and default to "*". We need not implement ssl-ports right away, because the primary use for SSL and non-SSL is likely http and https for webservers, and they will likely use ingress, which will likely use SNI instead. (But I do think some people will want direct-to-the-service http & https, so I bet we'll want it eventually..)

cc @therc and cf #23495

DNS support at the service level:

  • service.beta.kubernetes.io/hostname=myservice.myzone.com

If set, Kubernetes will configure a DNS alias for the ELB. Because this isn't specific to AWS, I don't think it gets an "aws-" prefix.

cc @chbatey and @quinton-hoole and cf #21397

Sharing / reusing an ELB

  • service.beta.kubernetes.io/load-balancer-secret=some-sort-of-secret-value

This allows an ELB to be (safely?) shared between two k8s clusters. If set, k8s will look for an ELB with a tag derived from the secret. If it finds one, it will use that ELB, otherwise it will create one (and tag it). In this way, two k8s clusters with services with the same tag will share an ELB. If the secret is kept reasonably secret then a user won't be able to "steal" someone else's ELB (this will be more realistic when we have RBAC etc).

cf #23483

Metadata

Assignees

No one assigned

    Labels

    area/apiIndicates an issue on api area.area/federationlifecycle/rottenDenotes an issue or PR that has aged beyond stale and will be auto-closed.priority/backlogHigher priority than priority/awaiting-more-evidence.sig/multiclusterCategorizes an issue or PR as relevant to SIG Multicluster.sig/networkCategorizes an issue or PR as relevant to SIG Network.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions