Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add label to delegate endpoints control to custom controller #87488

Closed

Conversation

s1061123
Copy link

What type of PR is this?
/kind feature

What this PR does / why we need it:
This PR introduces new label, "service.kubernetes.io/endpoint-controller-name", to delegate endpoints control to custom controller.

Which issue(s) this PR fixes:
Fixes #87412

Special notes for your reviewer:

Does this PR introduce a user-facing change?:


Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:


…dpoints control

Signed-off-by: Tomofumi Hayashi <tohayash@redhat.com>
@k8s-ci-robot
Copy link
Contributor

@s1061123: Adding the "do-not-merge/release-note-label-needed" label because no release-note block was detected, please follow our release note process to remove it.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added kind/feature Categorizes issue or PR as related to a new feature. do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Jan 23, 2020
@k8s-ci-robot
Copy link
Contributor

Welcome @s1061123!

It looks like this is your first PR to kubernetes/kubernetes 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes/kubernetes has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

@k8s-ci-robot k8s-ci-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Jan 23, 2020
@k8s-ci-robot
Copy link
Contributor

Hi @s1061123. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added area/test sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. sig/apps Categorizes an issue or PR as relevant to SIG Apps. sig/network Categorizes an issue or PR as relevant to SIG Network. sig/testing Categorizes an issue or PR as relevant to SIG Testing. and removed needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Jan 23, 2020
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: s1061123
To complete the pull request process, please assign deads2k
You can assign the PR to them by writing /assign @deads2k in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@s1061123
Copy link
Author

/assign @deads2k

@fedebongio
Copy link
Contributor

/sig network
/cc @cheftako

@k8s-ci-robot k8s-ci-robot requested a review from cheftako January 23, 2020 21:11
@danwinship
Copy link
Contributor

/assign @robscott

@robscott
Copy link
Member

Hey @s1061123, just trying to make sure I understand the goal here. The corresponding issue compares this to what exists for kube-proxy to tell it to ignore Services with the service.kubernetes.io/service-proxy-name label set. This seems to be more focused on instructing the Endpoints controller to ignore Endpoints with the service.kubernetes.io/endpoint-controller-name label set?

If I'm understanding this correctly, we already have a label that covers this capability for EndpointSlices (endpointslice.kubernetes.io/managed-by). Do we need a similar label for Endpoints? I'm hesitant to make any changes to a stable core API or corresponding controller at this point, especially if the functionality in EndpointSlices is already sufficient.

@s1061123
Copy link
Author

Hi @robscott,

Thank you for your comments!

I've looked the code and I suppose there are a little gaps. First, let me clarify the label, endpointslice.kubernetes.io/managed-by.

Looking the code (master branch), this label is for endpointslice, added by k8s endpointslice controller and once it is changed to another value, then k8s endpointslice controller ignores only to watch update.

When the Service/Pod are created with service selector, k8s endpointslice controller always creates endpointslice whatever the label is. If we change endpointslice label value, endpointslice.kubernetes.io/managed-by once it created, then k8s endpointslice controller ignores it and k8s endpointslice controller creates another endpointslice after service change.

I tested it in K8s v1.17.2 and here's the logs. I added nginx service, changed the label and scale the service, then k8s endpointslice controller creates another endpointslice.

https://gist.github.com/s1061123/bfa450cb28a923c8624360ae17e5e522

With custom controller case, I imagined that the following scenario might be reasonable:

  1. Pod/Service are created with selector and Service has label for custom controller
  2. K8s endpointslice controller ignores the CREATE event and no endpointslice is created
    (For now, k8s endpointslice controller always creates it, actually)
  3. Custom controller gets the CREATE events then create endpointslice for that, with endpointslice.kubernetes.io/managed-by=<custom controller>
  4. When the service is changed (e.g. scale), k8s endpointslice controller ignores the UPDATE event due to different label value, endpointslice.kubernetes.io/managed-by, and no endpointslice is created
    (For now, K8s endpointslice controller creates another endpointslice due to label)

I understood that Endpoint will be replaced with EndpointSlice, hence I am okey to use EndpointSlice, however, current endpointslice.kubernetes.io/managed-by is not sufficient from custom controller point of view. The label for Service, that k8s endpointslice controller ignores, is required to interworking with custom controller service/endpoints/endpointslice.

I am going to update diffs to support endpointslices.

@robscott
Copy link
Member

Hey @s1061123, thanks for the additional detail! I'm still not quite sure I'm understanding this proposal correctly though. It sounds like you're hoping to have some kind of label on a Service that would prevent the EndpointSlice controller from creating or managing EndpointSlices for the Service. Right now the only way to accomplish that would be to have an empty selector on the Service itself.

Currently we have two scenarios covered that seem related to this proposal:

  1. Allow other controllers to create/manage EndpointSlices without having the EndpointSlice controller interfere (use endpointslice.kubernetes.io/managed-by annotation).
  2. Tell the EndpointSlice controller not to create EndpointSlices for a Service (use an empty selector on the Service).

Are you able to accomplish what you want with 1 or 2? If not, it would be helpful to have some more examples of how you'd like this to work and why.

@s1061123
Copy link
Author

s1061123 commented Jan 27, 2020

Thank you for your quick reply, @robscott,

  1. Allow other controllers to create/manage EndpointSlices without having the EndpointSlice controller interfere (use endpointslice.kubernetes.io/managed-by annotation).
  2. Tell the EndpointSlice controller not to create EndpointSlices for a Service (use an empty selector on the Service).

I would like to use service selector for the service/pod of my custom controller because service selector is the good way to making association between service and pods and it is common in Kubernetes community hence I would like to use this mechanisms.

@robscott
Copy link
Member

Hey @s1061123, thanks for clarifying. This kind of addition sounds significant enough that it would likely need an associated KEP. In that KEP you could explain why the existing options discussed above are insufficient, and what use cases this proposal would be helpful for.

@s1061123
Copy link
Author

Thank you for your advice! I am going to write KEP for that.

By the way, should I close this PR, or keep it as WIP?

@robscott
Copy link
Member

👍 Sounds good. I think it would be better to close this at least temporarily since our processes tend to bug us about PRs that have gone stale. This could always be reopened after the KEP process.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/test cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. kind/feature Categorizes issue or PR as related to a new feature. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. sig/apps Categorizes an issue or PR as relevant to SIG Apps. sig/network Categorizes an issue or PR as relevant to SIG Network. sig/testing Categorizes an issue or PR as relevant to SIG Testing. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add well-known label for custom endpoint controller as well as "service.kubernetes.io/service-proxy-name"
6 participants