Spark 2.4.4 with Hive 2.3.6

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

Spark 2.4.4 with Hive 2.3.6

Vinod Kancharana

I am trying to build Spark 2.4.4 from the source code. I see the default Hive version as 1.2.1.

Is there a way I can compile spark with Hive 2.3.6?

I tried running ./dev/ --pip --tgz -Phadoop-2.7 -Dhadoop.version=2.8.5 -Psparkr -Phive -Phive-thriftserver -Dhive.version=2.3.6 -Dhive.version.short=2.3.6 -Pmesos -Pyarn -Pkubernetes but running into errors while compiling hive.

Any suggestions?