Add rd_kafka_query_watermark_offsets call to rd_kafka_topic_leader_query #694
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Query watermark offsets uses rd_kafka_toppar_get2 to create a topic which in turn calls rd_kafka_topic_new0, which unlike rd_kafka_topic_new does not call rd_kafka_topic_leader_query. This had the effect that rd_kafka_toppar_leader would return NULL until rd_kafka_topic_leader_query was called by rd_kafka_topic_scan_tmr_cb, which is run every 1000ms by a timer. Because of this query watermark offsets would occasionally take 1000ms longer to finish, or return WAIT_COORD error if timeout was shorter than that.
| @@ -1911,6 +1911,9 @@ rd_kafka_query_watermark_offsets (rd_kafka_t *rk, const char *topic, | |||
| return RD_KAFKA_RESP_ERR__UNKNOWN_PARTITION; | |||
| rktp = rd_kafka_toppar_s2i(s_rktp); | |||
|
|
|||
| /* Query for the topic leader (async) */ | |||
There was a problem hiding this comment.
We only need to do this if there is no leader found, so maybe move this down a couple of lines below the failing rd_kafka_toppar_leader() check?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Query watermark offsets uses rd_kafka_toppar_get2 to create a topic which
in turn calls rd_kafka_topic_new0, which unlike rd_kafka_topic_new does
not call rd_kafka_topic_leader_query.
This had the effect that rd_kafka_toppar_leader would return NULL until
rd_kafka_topic_leader_query was called by rd_kafka_topic_scan_tmr_cb,
which is run every 1000ms by a timer. Because of this query watermark
offsets would occasionally take 1000ms longer to finish, or return
WAIT_COORD error if timeout was shorter than that.