Start a standalone server as root and use it with user accounts
I'd like to have a single standalone
server, running as root on my machine, on which jobs can be run from
multiple user accounts on the same machine.
However, when I do this, writing files gives me error similar to the one in this Stackoverflow question. The first answer
to this question explains why: the server (running as root) is creating
a temporary file, and the job (running as the user) is trying to move
it, but it doesn't have access.
Is there a way
around this? I feel like it should be possible to run a Spark server as
root and run a job on it from a user account.
The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you should delete this message. You are hereby notified that any disclosure, copying, distribution or taking action in relation to the contents of this information is strictly prohibited and may be unlawful.
Re: Start a standalone server as root and use it with user accounts
I hate to be "that guy", but I'd like to know myself.
I tried to setup something similar, except I created a "service" account
which starts the Spark service, but like you, I kept on getting file
permission errors when submitting jobs under my own login. My current
workaround was to su to the service account and run under that login.
As ultimately what I am trying to achieve is an unattended, scheduled series
of processes, to have a service account run everything is OK in my example.
But at some stage, this behavior will always raise eyebrows from the
security team where I work.
I am sure a lot of this comes down to folder ownership and permissions and
possibly ACL's. I haven't had the chance to deep dive myself, but hopefully
bubbling this up might bring someone out of the woodwork with a solution.