I am using Spark 2.4 and using createDstream to read from kafka topic. The
topic has messaged written from a transactional producer.
I am getting the following error
"requirement failed: Got wrong record for
spark-executor-FtsTopicConsumerGrp7 test11-1 even after seeking to offset 85
got offset 86 instead. If this is a compacted topic, consider enabling
When i enable spark.streaming.kafka.allowNonConsecutiveOffsets, I am
getting the following error
java.lang.IllegalArgumentException: requirement failed: Failed to get
records for compacted spark-executor-FtsTopicConsumerGrpTESTING_5
fts.analytics-0 after polling for 10000
Also I set kafka.isolation.level="read_committed".