Closed
Description
Hi,
Updating Apache Kafka to 2.0.0 causes problems with the current version of the sarama library.
2018-08-23T11:32:57Z E! Error in plugin [inputs.kafka_consumer]: Consumer Error: kafka: error while consuming telegraftopic/22: kafka: error decodin:wqg packet: CRC didn't match expected 0x0 got 0xe38a6876
They have already addressed this:
IBM/sarama#1149
Maybe an update in the sarama library version is in order?
I already compiled telegraf (545b59f) with the latest sarama (647feef69a1a7e0e23cccceeb786176bc4fcf5ea). I can see that telegraf works properly now.
I have only tried it against kafka 2.0.0.
System info:
Telegraf version: Telegraf v1.7.3 (git: release-1.7 4dc29d0)
Kafka version: 2.0.0
Scala Version (for Kafka) : 2.12
Steps to reproduce:
- Try to consume from a kafka 2.0.0 topic
- Fail with:
2018-08-23T11:32:57Z E! Error in plugin [inputs.kafka_consumer]: Consumer Error: kafka: error while consuming telegraftopic/22: kafka: error decodin:wqg packet: CRC didn't match expected 0x0 got 0xe38a6876
Activity
danielnelson commentedon Aug 23, 2018
@HristoMohamed Would you be able to try Telegraf master and set the new version option to 1.1.0? I think this may allow using Kafka 2.0 with the current version, and I would feel more comfortable backporting the version option to Telegraf 1.7.x.
HristoMohamed commentedon Aug 29, 2018
Hey,
Yes I can test by later this week.
dvenza commentedon Aug 31, 2018
I can confirm that the latest master, with the version option works with Kafka 2.0
danielnelson commentedon Aug 31, 2018
Good news, thanks for checking this out @dvenza.
I think this will need to wait for 1.8 because of the changes to the sarama library version.
Bump sarama to version 1.18.0 to support Kafka 2.0
Bump sarama to version 1.18.0 to support Kafka 2.0