Hive with apache spark

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Hive with apache spark

Hafiz Mujadid
This post has NOT been accepted by the mailing list yet.
Hi

how can we read data from external hive server. Hive server is running and I want to read data remotely using spark. is there any example ?


thanks
Reply | Threaded
Open this post in threaded view
|

Re: Hive with apache spark

abhimadav
This post has NOT been accepted by the mailing list yet.
Hi,

My guess is that you would only need the hive.metastore.uris to be specified in your spark-defaults.conf on runtime properties used by the spark application. It would use remote-metastore setup for reading/writing  hive tables.

Early search results got me this - http://www.cloudera.com/documentation/archive/cdh/4-x/4-2-0/CDH4-Installation-Guide/cdh4ig_topic_18_4.html

Have you tried something similar? Also, you would have to enableHiveSupport in your spark session if your spark version is 2.0+ (here)