Spark udf from external jar without enabling Hive

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Spark udf from external jar without enabling Hive

Swapnil Chougule
Hi Team,

I am creating udf as follow from external jar

val spark = SparkSession.builder.appName("UdfUser")
      .master("local")
      .enableHiveSupport()
      .getOrCreate()

spark.sql("CREATE FUNCTION uppercase AS 'path.package.udf.UpperCase' " +
      "USING JAR '/home/swapnil/udf-jars/spark-udf-1.0.jar'")


Can I do same without enabling hive support (i.e. memory catalog)? I tried same but didn't get solution. Any help is really appreciated.

Thanks,
Swapnil