Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[docs] Clarify StoreOffset overload behaviour difference #1830

Closed
wants to merge 1 commit into from
Closed

[docs] Clarify StoreOffset overload behaviour difference #1830

wants to merge 1 commit into from

Conversation

sazarubin
Copy link

@sazarubin sazarubin commented Jun 8, 2022

Currently difference of behaviour between StoreOffset overloads is unclear.

I extended existing doc comments with clarifications.

Hope this helps!

@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@sazarubin
Copy link
Author

sazarubin commented Jun 8, 2022

Uhmm, I'm not willing to leave my address (which is required in the CLA agreement signing form) just for doc clarification 🤦

I'll be glad if the change gets into mainline by merging this PR or by opening a new one from existing contributors

@mhowlett
Copy link
Contributor

thanks, agree some better docs here would be good (though I'd choose different wording).

i'll address this when I make changes for KIP-320, which confuses things even further by committing the leader epoch associated with the message before the committed offset...

@mhowlett mhowlett closed this Jun 15, 2022
@sazarubin sazarubin deleted the clarify-store-offset-difference branch August 10, 2022 14:13
@sazarubin
Copy link
Author

Hi, it's me again.

So, as you supposed, KIP-320 changes really messed things up. Code that used StoreOffset, just silently broke when we upgraded Confluent.Kafka from 2.0.2 to 2.1.1.

The reason is that the leaderEpoch parameter isn't required by the TopicPartitionOffset ctor but at the same time without it StoreOffset doesn't do anything without throwing any exception or logging any errors.

I was able to figure it out when switched debug logs on and saw a bunch of [thrd:main]: Topic <topic_name> [<partition_number>]: stored offset INVALID (leader epoch -1), committed offset 873041 (leader epoch -1): not including in commit messages.

I think it should be at least documented and/or marked as breaking in release notes, though I'm not sure what the real solution should be.

@sazarubin
Copy link
Author

Should I post a separate issue?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants