I have also tried using collect_list() in the aggregate expression of
groupByKey, but that is taking more time to process the datasets.
Also, since we are aggregating - we could only use either 'Complete' or
'Update' in output modes, but 'Append' mode looks more suitable for our use
I have also looked at the groupByKey(Num_Partitions) and reduceByKey()
functions available in Direct Dstream which gives results like in the form
of -> (String, Itreable[String,Int]) without doing any aggregates.
Is there something available similar to that in structured streaming ?