Any impact on Driver out memory with long batch queue.
Has anyone observed that having long streaming queues impact Driver memory. Our Spark Driver is getting killed due Java heap out memory (4GB). We are not doing any computation / aggregation in Driver just consuming from Kafka topics. But there around 100-500 queued batches.