Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[IMPROVEMENT] Clean up backup target in IM-R pod if the backup target setting is unset #5741

Closed
ChanYiLin opened this issue Apr 12, 2023 · 2 comments
Assignees
Labels
area/orphan Longhorn orphaned resource related like replica, backup backport/1.4.5 backport/1.5.4 component/longhorn-instance-manager Longhorn instance manager (interface between control and data plane) component/longhorn-manager Longhorn manager (control plane) kind/improvement Request for improvement of existing function priority/0 Must be implement or fixed in this release (managed by PO)
Milestone

Comments

@ChanYiLin
Copy link
Contributor

Is your improvement request related to a feature? Please describe (? if you like this request)

When the user creates a volume and makes a backup, there is a mount point in IM-R for the replica to do the backup.
If the user unsets the backup target, longhorn won't unmount the mount point in IM-R and might introduce system instability such as high CPU loading when the backup server is gone.

We have tickets related
#5391: every time when LH tries to mount the mount point, it will unmount other mount points first and then mount the current setting Target.
#5655: backup-target-controller will unmount mount points in longhorn-manager pods when the URL is unset.

We would like to handle IM-R pod mount points left issue later when we implement multiple backup target support feature in #5411

Describe the solution you'd like

When the user unset the backup target, the mount point in IM-R pod should also be unmounted.

Describe alternatives you've considered

A clear and concise description of any alternative solutions or features you've considered.

Additional context

We might need to consider adding a new grpc service in IM-R for IM-R to do the cleanup in the pod.
Currently, there are only ProxyEngineService and ProcessManagerService these two services which are not appropriate for this general task. (see #5655)

@ChanYiLin ChanYiLin added the kind/improvement Request for improvement of existing function label Apr 12, 2023
@innobead innobead added this to the v1.6.0 milestone Apr 12, 2023
@innobead innobead added component/longhorn-manager Longhorn manager (control plane) component/longhorn-instance-manager Longhorn instance manager (interface between control and data plane) area/orphan Longhorn orphaned resource related like replica, backup labels Apr 12, 2023
@innobead innobead added the priority/0 Must be implement or fixed in this release (managed by PO) label Sep 14, 2023
@longhorn-io-github-bot
Copy link

longhorn-io-github-bot commented Nov 2, 2023

Pre Ready-For-Testing Checklist

@chriscchien
Copy link
Contributor

Verified pass on longhorn master (longhorn-manager 16c09f, longhorn-instance-manager 29b7a7) with test steps

  1. When a nfs backup store set in backup target, can see nfs mount in instance-manager pod
instance-manager-eea46f9cb64945896dda10fccdcc7a4f:/ # mount | grep "nfs"
longhorn-test-nfs-svc.default:/opt/backupstore on /var/lib/longhorn-backupstore-mounts/longhorn-test-nfs-svc_default/opt/backupstore type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,acregmin=1,acregmax=1,acdirmin=1,acdirmax=1,soft,proto=tcp,timeo=300,retrans=2,sec=sys,clientaddr=10.42.0.11,local_lock=none,addr=10.42.1.8)
  1. Unset the backup target, the nfs mount disappeared
instance-manager-eea46f9cb64945896dda10fccdcc7a4f:/ # mount | grep "nfs"
instance-manager-eea46f9cb64945896dda10fccdcc7a4f:/ #
  1. Set the nfs backup store as backup target again, the backup worked well

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/orphan Longhorn orphaned resource related like replica, backup backport/1.4.5 backport/1.5.4 component/longhorn-instance-manager Longhorn instance manager (interface between control and data plane) component/longhorn-manager Longhorn manager (control plane) kind/improvement Request for improvement of existing function priority/0 Must be implement or fixed in this release (managed by PO)
Projects
Status: Closed
Development

No branches or pull requests

4 participants