Skip to content

Review default values for configuration settings #187

Closed
@amotl

Description

Dear @TsuyoshiUshio and @fbeltrao,

firstly, I would like to apologize for bumping into the review process at #186 (comment) and #186 (comment). Let's better continue a more thorough discussion here.

As a followup to #185, I would like to dedicate this topic a separate issue regarding

I would like to emphasize that it is probably also crucial to apply the recommended configuration settings for Kafka on Azure as outlined within confluentinc/librdkafka#3109 (comment).

As we found out recently, Microsoft published some recommended configuration properties for librdkafka at [1]. It would be nice if the Azure function bindings for Kafka would already use these recommendations out of the box. I didn't check the code base thoroughly yet, so I am humbly asking for your support on that for the upcoming release when the bump to librdkafka 1.5.2 will take place.

As outlined within [1], the configuration properties socket.keepalive.enable and metadata.max.age.ms are crucial to get right within Azure environments in order to save users from running into issues like outlined within confluentinc/librdkafka#3109. I strongly recommend to apply these default values out of the box as most regular users will not be aware of them and might experience nasty things.

Property Recommended Values Permitted Range Notes
socket.keepalive.enable true   Necessary if connection is expected to idle. Azure will close inbound TCP idle > 240,000 ms.
metadata.max.age.ms ~ 180000 < 240000 Can be lowered to pick up metadata changes sooner.

While these settings [1] are primarily dedicated to Event Hubs, they will also apply to all communications with vanilla Kafka server components, as the underlying networking infrastructure problem outlined within confluentinc/librdkafka#3109 will be the same.

Thanks for listening and with kind regards,
Andreas.

[1] https://docs.microsoft.com/en-us/azure/event-hubs/apache-kafka-configurations#librdkafka-configuration-properties

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions