Description
Dear @TsuyoshiUshio and @fbeltrao,
firstly, I would like to apologize for bumping into the review process at #186 (comment) and #186 (comment). Let's better continue a more thorough discussion here.
As a followup to #185, I would like to dedicate this topic a separate issue regarding
I would like to emphasize that it is probably also crucial to apply the recommended configuration settings for Kafka on Azure as outlined within confluentinc/librdkafka#3109 (comment).
As we found out recently, Microsoft published some recommended configuration properties for librdkafka at [1]. It would be nice if the Azure function bindings for Kafka would already use these recommendations out of the box. I didn't check the code base thoroughly yet, so I am humbly asking for your support on that for the upcoming release when the bump to librdkafka 1.5.2 will take place.
As outlined within [1], the configuration properties socket.keepalive.enable
and metadata.max.age.ms
are crucial to get right within Azure environments in order to save users from running into issues like outlined within confluentinc/librdkafka#3109. I strongly recommend to apply these default values out of the box as most regular users will not be aware of them and might experience nasty things.
Property | Recommended Values | Permitted Range | Notes |
---|---|---|---|
socket.keepalive.enable | true | Necessary if connection is expected to idle. Azure will close inbound TCP idle > 240,000 ms. | |
metadata.max.age.ms | ~ 180000 | < 240000 | Can be lowered to pick up metadata changes sooner. |
While these settings [1] are primarily dedicated to Event Hubs, they will also apply to all communications with vanilla Kafka server components, as the underlying networking infrastructure problem outlined within confluentinc/librdkafka#3109 will be the same.
Thanks for listening and with kind regards,
Andreas.