JDBC to hive batch use case in spark

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

JDBC to hive batch use case in spark

hokam chauhan
Hi,
I have an use case in which I wants to read data from a jdbc source(Oracle) table and write it to hive table on periodic basis. I tried this using the SQL context to read from Oracle and Hive context to write the data in hive. The data read parts works fine but when I ran the save call on hive context to write data, it throws the exception and it says the table or view does not exists even though the table is precreated in hive.

Please help if anyone tried such scenario.

Thanks
Reply | Threaded
Open this post in threaded view
|

Re: JDBC to hive batch use case in spark

KevinZwx
If you don't mind, I think it will help if you post your code

Hokam Singh Chauhan <[hidden email]>于2017年12月9日周六 下午8:02写道:
Hi,
I have an use case in which I wants to read data from a jdbc source(Oracle) table and write it to hive table on periodic basis. I tried this using the SQL context to read from Oracle and Hive context to write the data in hive. The data read parts works fine but when I ran the save call on hive context to write data, it throws the exception and it says the table or view does not exists even though the table is precreated in hive.

Please help if anyone tried such scenario.

Thanks
Reply | Threaded
Open this post in threaded view
|

Re: JDBC to hive batch use case in spark

ayan guha
Please enable hive support onnspark session,  if using spark2. If you are on spark1, use hiveContext instead of sqlContext.

On Sun, 10 Dec 2017 at 12:20 am, 张万新 <[hidden email]> wrote:
If you don't mind, I think it will help if you post your code

Hokam Singh Chauhan <[hidden email]>于2017年12月9日周六 下午8:02写道:
Hi,
I have an use case in which I wants to read data from a jdbc source(Oracle) table and write it to hive table on periodic basis. I tried this using the SQL context to read from Oracle and Hive context to write the data in hive. The data read parts works fine but when I ran the save call on hive context to write data, it throws the exception and it says the table or view does not exists even though the table is precreated in hive.

Please help if anyone tried such scenario.

Thanks
--
Best Regards,
Ayan Guha