Can not start thrift-server on spark2.4

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Can not start thrift-server on spark2.4

Moein Hosseini
I like to start spark thrift server on cluster of 3 machine with HDFS and standalone HA spark (v2.4).
So I started it with following command under user spark24 but get runtime exception about hdfs permissions.
Command:
./start-thriftserver.sh --master spark://master:7077
 exception:
Caused by: java.lang.RuntimeException: java.lang.RuntimeException: The root scratch dir: /tmp/hive on HDFS should be writable. Current permissions are: rwxr-xr-x                                                  

But I think the hdfs permission is fine because I chmod everything under /tmp/hive to 777 and set rwx access to spark24 and hive users and all groups.

$ hdfs dfs -ls -d /tmp/hive
drwxrwxrwx+  - hive hdfs          0 2019-02-03 14:13 /tmp/hive

$ hdfs dfs -getfacl /tmp/hive
# file: /tmp/hive
# owner: hive
# group: hdfs
user::rwx
user:hive:rwx
user:spark24:rwx
group::rwx
group:hive:rwx
group:spark24:rwx
mask::rwx
other::rwx

What is wrong with in my case?
--

Moein Hosseini
Data Engineer
mobile: <a href="tel:+98+912+468+1859" style="background-color:transparent;color:rgb(0,0,1)" target="_blank">+98 912 468 1859
site: www.moein.xyz
email: [hidden email]
linkedin
twitter