keep getting empty table while using saveAsTable() to save DataFrame as table

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

keep getting empty table while using saveAsTable() to save DataFrame as table

nicholasl
Hi,
    I am using Spark SQL in a cluster, and try to use the CBO supported in
Spark SQL. The dataset that I am using is TPC-DS. In order to collect the
statistic of these data, I first load the data from HDFS to create
dataframes. Then use saveAsTable() to save dataframes as tables. All the
process seems to be fine, and I didn't get any exceptions. However, when I
was trying to check those created tables, all of them is empty.

Question: Did anyone have the same problem when you are using the CBO
provided in Spark SQL? Please help me with this issue. Thank you very much.



--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]