kafka.(*readBuffer) #325
Closed
kafka.(*readBuffer) #325
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 7 commits
July 24, 2019 12:16
|
@stevevls if you want to take a second look at this, I think it should be ready to go. Besides the main change which is the introduction of the |
added 7 commits
October 16, 2019 15:43
achille-roussel
force-pushed
the
read-buffer
branch
from
7a87340 to
68ef225
Compare
|
Closing in favor of #438 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR refactors the kafka-go read paths to use a new
readBuffertype (similar to what we introduced in #318).This new model reduces the amount of error handling and byte counting that was proliferating all over the place, making the code more robust and easier to read. It also pave the road towards removing the intermediary buffer used when decompressing record batches, and brings us one step closer to being able to offer a zero-copy API for consumers.
I also took some time to cleanup a lot of tests, remove code duplication, and hopefully fix a couple of bugs on the way by reducing the amount of special assumptions that are made between different layers of the codebase.