Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support cross resource group load balancer. #81054

Merged
merged 2 commits into from
Aug 9, 2019
Merged

Support cross resource group load balancer. #81054

merged 2 commits into from
Aug 9, 2019

Conversation

nilo19
Copy link
Member

@nilo19 nilo19 commented Aug 7, 2019 •
edited
Loading

/kind feature

What this PR does / why we need it:

Users want to reuse their existing LoadBalancer that are created in another resource group before the Kubernetes cluster is created.

Which issue(s) this PR fixes:

Fixes #78843

Does this PR introduce a user-facing change?:

Add a azure cloud configuration `LoadBalancerName` and `LoadBalancerResourceGroup` to allow the corresponding customizations of azure load balancer.

@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. kind/feature Categorizes issue or PR as related to a new feature. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Aug 7, 2019
@k8s-ci-robot
Copy link
Contributor

Hi @nilo19. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@nilo19
Copy link
Member Author

nilo19 commented Aug 7, 2019

/sig azure

@k8s-ci-robot k8s-ci-robot added area/cloudprovider sig/cloud-provider Categorizes an issue or PR as relevant to SIG Cloud Provider. and removed needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Aug 7, 2019
@feiskyer
Copy link
Member

feiskyer commented Aug 7, 2019

/ok-to-test
/area provider/azure
/priority important-soon

@k8s-ci-robot k8s-ci-robot added area/provider/azure Issues or PRs related to azure provider priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Aug 7, 2019
feiskyer
feiskyer reviewed Aug 7, 2019
View reviewed changes
staging/src/k8s.io/legacy-cloud-providers/azure/azure_loadbalancer.go Outdated
return cloudprovider.DefaultLoadBalancerName(service)
}

func (az *Cloud) getLoadBalancerResourceGroup(service *v1.Service) string {
if rgName, found := service.Annotations[ServiceAnnotationLoadBalancerResourceGroup]; found && strings.TrimSpace(rgName) != "" {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's add two new configuration options in Config:

LoadBalancerName string `json:"loadBalancerName,omitempty" yaml:"loadBalancerName,omitempty"`
LoadBalancerResourceGroup string `json:"loadBalancerResourceGroup,omitempty" yaml:"loadBalancerResourceGroup,omitempty"`

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, getLoadBalancerResourceGroup could be

if az.LoadBalancerResourceGroup != "" {
  return az.LoadBalancerResourceGroup
}

return az.ResourceGroup

Copy link
Member

@feiskyer feiskyer Aug 7, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same for loadBalancerName. But notice az.LoadBalancerName is only applied for defaultLBName (replace clusterName when generating the LB name).

@k8s-ci-robot k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Aug 8, 2019
@feiskyer
Copy link
Member

feiskyer commented Aug 8, 2019

/retest
/test pull-kubernetes-e2e-aks-engine-azure

@nilo19
Copy link
Member Author

nilo19 commented Aug 8, 2019

/test pull-kubernetes-integration

feiskyer
feiskyer reviewed Aug 8, 2019
View reviewed changes
staging/src/k8s.io/legacy-cloud-providers/azure/azure_backoff.go
@@ -304,7 +307,8 @@ func (az *Cloud) listLBWithRetry(service *v1.Service) ([]network.LoadBalancer, e
ctx, cancel := getContextWithCancel()
defer cancel()

allLBs, retryErr = az.LoadBalancerClient.List(ctx, az.ResourceGroup)
rgName := az.getLoadBalancerResourceGroup(service)
allLBs, retryErr = az.LoadBalancerClient.List(ctx, rgName)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you also fix this on L457 and L478:

az.LoadBalancerClient.Delete(ctx, az.ResourceGroup, lbName)

feiskyer
feiskyer reviewed Aug 8, 2019
View reviewed changes
staging/src/k8s.io/legacy-cloud-providers/azure/azure_backoff.go
@@ -282,7 +284,8 @@ func (az *Cloud) ListLB(service *v1.Service) ([]network.LoadBalancer, error) {
ctx, cancel := getContextWithCancel()
defer cancel()

allLBs, err := az.LoadBalancerClient.List(ctx, az.ResourceGroup)
rgName := az.getLoadBalancerResourceGroup(service)
allLBs, err := az.LoadBalancerClient.List(ctx, rgName)
if err != nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please also use rgName in logs below.

feiskyer
feiskyer reviewed Aug 8, 2019
View reviewed changes
staging/src/k8s.io/legacy-cloud-providers/azure/azure_loadbalancer.go Outdated
return cloudprovider.DefaultLoadBalancerName(service)
}

func (az *Cloud) getLoadBalancerResourceGroup(service *v1.Service) string {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

param service is not used here, could we remove it?

feiskyer
feiskyer reviewed Aug 8, 2019
View reviewed changes
staging/src/k8s.io/legacy-cloud-providers/azure/azure_loadbalancer.go Outdated
// TODO: replace DefaultLoadBalancerName to generate more meaningful loadbalancer names.
if az.LoadBalancerName != "" {
return az.LoadBalancerName
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please rollback the changes in L222-L224. The name here is used for frontendipconfigurations, service UID should be used so that different services configurations won't be conflicted.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for reviewing. The corresponding codes have been fixed.

@feiskyer
Copy link
Member

feiskyer commented Aug 8, 2019

The resource group name on

kubernetes/staging/src/k8s.io/legacy-cloud-providers/azure/azure_wrap.go

Line 236 in ef88694

lb, err := az.LoadBalancerClient.Get(ctx, az.ResourceGroup, key, "")
should also be changed to az.getLoadBalancerResourceGroup()

@nilo19 nilo19 force-pushed the t-qini-cross_rg_lb branch from d282513 to 0500dfd Compare August 8, 2019 13:34
@feiskyer
Copy link
Member

feiskyer commented Aug 9, 2019

/test pull-kubernetes-e2e-aks-engine-azure

@feiskyer
Copy link
Member

feiskyer commented Aug 9, 2019

https://github.com/kubernetes/kubernetes/pull/81054/files#diff-4c5af42498de1d56820fcdbe9f860147R479 Please fix the resource group name here

@nilo19 nilo19 force-pushed the t-qini-cross_rg_lb branch from 0500dfd to 659e857 Compare August 9, 2019 06:16
feiskyer
feiskyer approved these changes Aug 9, 2019
View reviewed changes
Copy link
Member

@feiskyer feiskyer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Aug 9, 2019
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: feiskyer, nilo19

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 9, 2019
@k8s-ci-robot k8s-ci-robot merged commit 8527c74 into kubernetes:master Aug 9, 2019
@k8s-ci-robot k8s-ci-robot added this to the v1.16 milestone Aug 9, 2019
@feiskyer
Copy link
Member

feiskyer commented Aug 9, 2019

Thanks @nilo19. Please open a PR to update docs in cloud-provider-azure repo.

@nilo19 nilo19 mentioned this pull request Aug 19, 2019
Closed
6 tasks
@nilo19 nilo19 deleted the t-qini-cross_rg_lb branch August 20, 2020 14:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@feiskyer feiskyer feiskyer approved these changes

@karataliu karataliu Awaiting requested review from karataliu

@khenidak khenidak Awaiting requested review from khenidak

Assignees

@feiskyer feiskyer

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/cloudprovider area/provider/azure Issues or PRs related to azure provider cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/feature Categorizes issue or PR as related to a new feature. lgtm "Looks good to me", indicates that a PR is ready to be merged. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/cloud-provider Categorizes an issue or PR as relevant to SIG Cloud Provider. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Milestone
v1.16
Development

Successfully merging this pull request may close these issues.

Cross resource group LoadBalancers
3 participants
@nilo19 @k8s-ci-robot @feiskyer