How thriftserver load data

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

How thriftserver load data

Soheil Pourbafrani
Hi, I want to write an application that load data from HDFS into tables and create a ThriftServer and submit it to the YARN cluster.

The question is how Spark actually load data. Does Spark load data in the memory since the application started or it waits for query and just loads data according to the query needs?