Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DLedger should support TransientStorePool #166

Open
humkum opened this issue Jul 1, 2022 · 2 comments
Open

DLedger should support TransientStorePool #166

humkum opened this issue Jul 1, 2022 · 2 comments

Comments

@humkum
Copy link

humkum commented Jul 1, 2022

FEATURE REQUEST

We found that there would be frequent "pagecache busy" with the growth of tps. And RocketMQ create a transientStorePool by using DirectByteBuffer to insulate write and read in Master-Slave mode. 
So I guess it would work better if dledger support transientStorePool to reduce jitter caused by pagecache busy.
There are some of my opinions:
1. Leader node uses directByteBuffer to write and follower nodes still use mappedByteBuffer to aviod data loss in some cases.
2. Initialize transientStorePool when the broker role changes to "leader", and release these memory space while the broker role change to "follower" from "leader".
3. If the broker role changed to "leader" from "follower", it could still use mappedBytebuffer to write the uncompleted file, and use directByteBuffer to write next file.
I'm looking forward to your suggestion. 
@akkw
Copy link
Contributor

akkw commented Oct 17, 2022

我理解在leader节点不会直接将数据落盘,接受到来自客户端的请求之后写入directByteBuffer,然后异步的向follower同步日志,以及异步的将directByteBuffer中的数据写入到mmapfile。 我的理解对吗?

@humkum
Copy link
Author

humkum commented Jan 12, 2024

我理解在leader节点不会直接将数据落盘,接受到来自客户端的请求之后写入directByteBuffer,然后异步的向follower同步日志,以及异步的将directByteBuffer中的数据写入到mmapfile。 我的理解对吗?

需要成功向 1/2 以上节点同步完成后才可以返回成功

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants