Enable hql on the JDBC thrift server

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Enable hql on the JDBC thrift server

Arnaud Wolf
Hi everyone.

I try to build a middleware that executes injection-safe dynamic SQL
queries using Spark, with Cassandra as storage engine. Results of
executed queries need to be stored into a newly created Cassandra table.

Spark SQL does not provide any feature for safe parameter binding, so I
thought about using the JDBC thrift server and the JDBC interface.

Inserting data into an external table from hive is performed by running
CREATE EXTERNAL TABLE <table_name>... STORED BY...

However, when trying to execute this statement through the thrift
server, I always get the following error:

java.sql.SQLException: org.apache.spark.sql.catalyst.parser.ParseException:
Operation not allowed: STORED BY(line 1, pos 31)

After some web digging, I could find that the reason was that the thrift
server was using Spark SQL and that Hive support is disabled by default
on Spark SQL. Then the common recommendation is to manually create a
SparkSession and use enableHiveSupport().

The problem is that I don't want to do that as I can't do safe query
binding with Spark SQL. I want to use only the JDBC interface. Do you
know if there is a way to enable Hive support at thrift server startup?

If there is no way, can somebody recommend a safe way for parameter binding?

Thank you for your consideration, and sorry for the long question.


---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Enable hql on the JDBC thrift server

Arnaud Wolf
I forgot to precise that I'm using Spark 2.1.0.


On 2017/11/10 16:08, Arnaud Wolf wrote:

> Hi everyone.
>
> I try to build a middleware that executes injection-safe dynamic SQL
> queries using Spark, with Cassandra as storage engine. Results of
> executed queries need to be stored into a newly created Cassandra table.
>
> Spark SQL does not provide any feature for safe parameter binding, so
> I thought about using the JDBC thrift server and the JDBC interface.
>
> Inserting data into an external table from hive is performed by
> running CREATE EXTERNAL TABLE <table_name>... STORED BY...
>
> However, when trying to execute this statement through the thrift
> server, I always get the following error:
>
> java.sql.SQLException:
> org.apache.spark.sql.catalyst.parser.ParseException:
> Operation not allowed: STORED BY(line 1, pos 31)
>
> After some web digging, I could find that the reason was that the
> thrift server was using Spark SQL and that Hive support is disabled by
> default on Spark SQL. Then the common recommendation is to manually
> create a SparkSession and use enableHiveSupport().
>
> The problem is that I don't want to do that as I can't do safe query
> binding with Spark SQL. I want to use only the JDBC interface. Do you
> know if there is a way to enable Hive support at thrift server startup?
>
> If there is no way, can somebody recommend a safe way for parameter
> binding?
>
> Thank you for your consideration, and sorry for the long question.
>


---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]