A code example of Catalyst optimization

classic Classic list List threaded Threaded
1 message Options
jgp
Reply | Threaded
Open this post in threaded view
|

A code example of Catalyst optimization

jgp
Hi there,

I am looking for an example of optimization through Catalyst, that you can demonstrate via code. Typically, you load some data in a dataframe, you do something, you do the opposite operation, and, when you collect, it’s super fast because nothing really happened to the data. Hopefully, my request is clear enough - I’d like to use that in teaching when explaining the laziness of Spark.

Does anyone has that in his labs?

tia,

jg


---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]