Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[IMPROVEMENT] Improve the replica rebuilding using different data transmission protocol #5002

Open
derekbit opened this issue Dec 6, 2022 · 3 comments
Open

[IMPROVEMENT] Improve the replica rebuilding using different data transmission protocol #5002

derekbit opened this issue Dec 6, 2022 · 3 comments
Assignees
@PhanLe1010
Labels
area/performance System, volume performance area/replica Volume replica where data is placed kind/improvement Request for improvement of existing function priority/1 Highly recommended to implement or fix in this release (managed by PO)
Milestone
v1.9.0

Comments

@derekbit
Copy link
Member

derekbit commented Dec 6, 2022 •
edited
Loading

Is your improvement request related to a feature? Please describe (👍 if you like this request)

The data block or requets transmission in replica rebuilding is using HTTP protocol. To improve the transmission efficiency, the HTTP can be replaced with a more efficient protocol.

Describe the solution you'd like

A clear and concise description of what you want to happen.

Describe alternatives you've considered

A clear and concise description of any alternative solutions or features you've considered.

Additional context

Add any other context or screenshots about the feature request here.
@derekbit derekbit added the kind/improvement Request for improvement of existing function label Dec 6, 2022
@innobead innobead added area/performance System, volume performance area/replica Volume replica where data is placed priority/0 Must be implement or fixed in this release (managed by PO) labels Dec 6, 2022
@innobead innobead added this to the v1.5.0 milestone Dec 6, 2022
@derekbit
Copy link
Member Author

We've improved the rebuilding performance by multiple threads, large block size and skipping unnecessary block checksum calculation. Then, the main bottleneck is disk IO. Do we need to change the transmission protocol? cc @innobead

@innobead innobead modified the milestones: v1.5.0, v1.6.0 Apr 13, 2023
@innobead
Copy link
Member

Let's include this in #6600, so we can review it later via performance testing before making a decision.

@innobead innobead modified the milestones: v1.6.0, v1.7.0 Sep 14, 2023
@innobead innobead mentioned this issue Sep 14, 2023
Open
@innobead innobead modified the milestones: v1.7.0, v1.8.0 Feb 26, 2024
@Toutou98 Toutou98 mentioned this issue May 10, 2024
Open
@derekbit derekbit modified the milestones: v1.8.0, v1.9.0 Jul 28, 2024
@PhanLe1010
Copy link
Contributor

PhanLe1010 commented Jul 29, 2024 •
edited
Loading

I think this one is a part of the Epic #6600 together with the new front end #5159 . Should we move it back to 1.8.0 and assign it to me?

cc @derekbit @innobead

@PhanLe1010 PhanLe1010 assigned PhanLe1010 and unassigned derekbit Jul 30, 2024
@PhanLe1010 PhanLe1010 modified the milestones: v1.9.0, v1.8.0 Jul 30, 2024
@derekbit derekbit moved this to Analysis and Design in Longhorn Sprint Aug 3, 2024
@innobead innobead added priority/1 Highly recommended to implement or fix in this release (managed by PO) and removed priority/0 Must be implement or fixed in this release (managed by PO) labels Aug 5, 2024
@derekbit derekbit modified the milestones: v1.8.0, v1.9.0 Sep 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@PhanLe1010 PhanLe1010

Labels
area/performance System, volume performance area/replica Volume replica where data is placed kind/improvement Request for improvement of existing function priority/1 Highly recommended to implement or fix in this release (managed by PO)
Projects
Longhorn Sprint
Status: Analysis and Design
Milestone
v1.9.0
Development

No branches or pull requests
3 participants
@innobead @derekbit @PhanLe1010