I'm having a little performance issue, with different Spark versions.
I've a spark embedded application written in scala, Initially I've use Spark 2.0.2, and works fine, with good speed response, but when I updated to 2.3.2 , with no any code changes It becomes slower.
Mainly what the application do is to gather information from a source, apply transformations with filters and performs aggregation over it. It's source is mainly parquet and no write is done just a serialization from the result.
Maybe I'm using deprecated api functions or the order of the operations are not generating a good plan...
Can someone give me some idea of any change on the versions that could generate this behavior?