Performance degradation with MaxMessageBytes #328
Comments
|
@sagarkrkv thanks for reporting! I'll look into this shortly. Other changes have landed between 0.2.4 and 0.2.5, and since we've also released 0.3. Having some simple example to highlight the problem would be very helpful to track down which commit introduced the regression. |
|
@achille-roussel I believe this was caused by #208. I created a PR to fix this issue, PTAL. |
|
I think #318 has reduced this problem to a large extent. I created #329 based on what I've observed in 0.2.5, there's still some slight performance benefit with my new PR. I'm totally willing to abandon the PR if you think a small performance benefit is not enough to justify the new function. Cheers. |
|
Improvements are always welcome! |
|
imo you should always be setting some sort of
I think this is the best route and looking at #329 it should help. |
|
Fixed by #329 |
Describe the bug
Our application produces a lot of messages in Async mode. Since we are in control of the creation of these messages, we don't need the ability to set MaxMessageBytes. But we are paying the performance penalty for this feature.
v0.2.4
I noticed we don't need to create a channel when producing messages in async mode, so I created a PR to fix that #290.
So I expected v0.2.5 to perform better than the previous version, as this unnecessary allocation was removed. But I was surprised to see a new expensive function running, especially since we didn't change any config to enable this feature.
v0.2.5
Kafka Version
To Reproduce
This was found during benchmark testing of our application. If needed I can probably write a simple benchmark to show this.
Expected behavior
Users who don't set
MaxMessageBytesshouldn't pay the cost for it.Additional context
Maybe we can look into reducing the cost of calculating the message size.
cc @ShaneSaww @achille-roussel @stevevls
The text was updated successfully, but these errors were encountered: