Client APIs for Accessing Spark Data Frames Directly
Is there anyway to access the Data Frames content
directly/interactively via some client access APIs? Some
We have a Java client application that uses spark launcher to
submit a spark job to a spark master.
The default spark launcher API has only a handle API that
provides job status information
It is possible to use the Hive JDBC to access data via the
built-in thrifty server, but the data have to be saved into hive
tables, not sure if these tables have to be persisted on disk
and require huge overhead for data I/O. Since DF is in memory,
it would be more convenient and faster to access the data
directly in memory if such APIs are available.