Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixup openstack cloud provider loadbalancer deletion error #68188

Merged
merged 1 commit into from
Sep 4, 2018

Conversation

esevan
Copy link
Contributor

@esevan esevan commented Sep 3, 2018

This change enables getLoadBalancer to return the loadbalancer even if no floating ip is associated to the VIP port of the loadbalancer.

Signed-off-by: Eunsoo Park esevan.park@gmail.com

What this PR does / why we need it:
This PR fixes the bug like below.

Warning  CreatingLoadBalancerFailed  17m (x3445 over 12d)  service-controller  Error creating load balancer (will retry): error getting LB for service default/influxdb: error getting floating ip for port 81253cae-acd6-4bed-8006-814c8729be8c: failed to find object

Special notes for your reviewer:
Refer to following bug description for better understanding of this PR.
When k8s service type is changed from LoadBalancer to others, there's a deleting loadbalancer process in kubernetes cloud provider as below.

https://github.com/kubernetes/kubernetes/blob/master/pkg/controller/service/service_controller.go#L284

if !wantsLoadBalancer(service) {
	_, exists, err := s.balancer.GetLoadBalancer(context.TODO(), s.clusterName, service)
	if err != nil {
		return fmt.Errorf("error getting LB for service %s: %v", key, err)
	}
	if exists {
		glog.Infof("Deleting existing load balancer for service %s that no longer needs a loadbalancer.", key)
		s.eventRecorder.Event(service, v1.EventTypeNormal, "DeletingLoadBalancer", "Deleting loadbalancer")
		if err := s.balancer.EnsureLoadBalancerDeleted(context.TODO(), s.clusterName, service); err != nil {
			return err
		}
		s.eventRecorder.Event(service, v1.EventTypeNormal, "DeletedLoadBalancer", "Deleted loadbalancer")
	}

Openstack cloud provider returns error even though LB exists since there's no associated floating IP with VIP port.
https://github.com/kubernetes/kubernetes/blob/master/pkg/cloudprovider/providers/openstack/openstack_loadbalancer.go#L476

floatIP, err := getFloatingIPByPortID(lbaas.network, portID)
if err != nil {
	return nil, false, fmt.Errorf("error getting floating ip for port %s: %v", portID, err)
}

This caused GetLoadBalancer() failed and retrying over and over in processServiceUpdate of k8sServiceController.

Release note:

NONE

This change enables ```getLoadBalancer``` to return the loadbalancer even if no floating ip is associated to the VIP port of the loadbalancer.

Signed-off-by: Eunsoo Park <esevan.park@gmail.com>
@k8s-ci-robot k8s-ci-robot added size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Sep 3, 2018
@k8s-ci-robot k8s-ci-robot added needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. sig/cloud-provider Categorizes an issue or PR as relevant to SIG Cloud Provider. labels Sep 3, 2018
@esevan
Copy link
Contributor Author

esevan commented Sep 3, 2018

/sig cloud-provider
/sig openstack

@k8s-ci-robot k8s-ci-robot added area/provider/openstack Issues or PRs related to openstack provider release-note-none Denotes a PR that doesn't merit a release note. and removed do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Sep 3, 2018
Copy link
Contributor

@FengyunPan2 FengyunPan2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch, thank you!

@FengyunPan2
Copy link
Contributor

/ok-to-test

@k8s-ci-robot k8s-ci-robot removed the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Sep 3, 2018
Copy link
Member

@yastij yastij left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@dims can you approve ?

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Sep 3, 2018
@FengyunPan2
Copy link
Contributor

FengyunPan2 commented Sep 3, 2018

@yastij Just wait to pass the e2e tests.

@yastij
Copy link
Member

yastij commented Sep 3, 2018

@FengyunPan2 - played a bit with this, as we needed it. Works as intended.

@FengyunPan2
Copy link
Contributor

/lgtm

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: esevan, FengyunPan2, yastij

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 3, 2018
@FengyunPan2
Copy link
Contributor

/test pull-kubernetes-e2e-gce

@dixudx
Copy link
Member

dixudx commented Sep 3, 2018

@FengyunPan2 Do we still need to support this in-tree openstack cloud provider? This has already been marked as deprecated and will be removed in future releases. We should use https://github.com/kubernetes/cloud-provider-openstack instead.

/cc @dims

/hold

@k8s-ci-robot k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Sep 3, 2018
@dims
Copy link
Member

dims commented Sep 3, 2018

@dixudx we should keep both tree(s) in sync until v1.13 opens up in k/k. at that point, i'll submit a PR to entirely remove the tree.

/hold cancel

@k8s-ci-robot k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Sep 3, 2018
@yastij
Copy link
Member

yastij commented Sep 3, 2018

/retest

@dims
Copy link
Member

dims commented Sep 4, 2018

/milestone v1.12

@k8s-ci-robot k8s-ci-robot added this to the v1.12 milestone Sep 4, 2018
@k8s-github-robot
Copy link

/test all [submit-queue is verifying that this PR is safe to merge]

@k8s-github-robot
Copy link

Automatic merge from submit-queue (batch tested with PRs 68135, 68188). If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.

@k8s-github-robot k8s-github-robot merged commit e5b30e0 into kubernetes:master Sep 4, 2018
@esevan esevan deleted the fix_bug_openstack_lb branch September 4, 2018 01:54
@k8s-ci-robot
Copy link
Contributor

@esevan: The following test failed, say /retest to rerun them all:

Test name Commit Details Rerun command
pull-kubernetes-e2e-kops-aws da5618e link /test pull-kubernetes-e2e-kops-aws

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/provider/openstack Issues or PRs related to openstack provider cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. release-note-none Denotes a PR that doesn't merit a release note. sig/cloud-provider Categorizes an issue or PR as relevant to SIG Cloud Provider. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants