Writing dataframe to vertica

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Writing dataframe to vertica

Nikhil Goyal
Hi guys,

I am trying to write dataframe to vertica using spark. It seems like spark is creating a temp table under public schema. I don't have access to public schema hence the job is failing. Is there a way to specify another schema?

Error
ERROR s2v.S2VUtils: createJobStatusTable: FAILED to create job status table public.S2V_JOB_STATUS_USER_NGOYAL
java.lang.Exception: S2V: FATAL ERROR for job S2V_job8087339107009511230. Unable to create status table for tracking this job:public.S2V_JOB_STATUS_USER_NGOYAL

Thanks
Nikhil
Reply | Threaded
Open this post in threaded view
|

Re: [External Sender] Writing dataframe to vertica

Anthony, Olufemi
How are you trying to write to Vertica ? Can you provide some snippets of code ?


Femi

On Tue, Oct 16, 2018 at 7:24 PM Nikhil Goyal <[hidden email]> wrote:
Hi guys,

I am trying to write dataframe to vertica using spark. It seems like spark is creating a temp table under public schema. I don't have access to public schema hence the job is failing. Is there a way to specify another schema?

Error
ERROR s2v.S2VUtils: createJobStatusTable: FAILED to create job status table public.S2V_JOB_STATUS_USER_NGOYAL
java.lang.Exception: S2V: FATAL ERROR for job S2V_job8087339107009511230. Unable to create status table for tracking this job:public.S2V_JOB_STATUS_USER_NGOYAL

Thanks
Nikhil


The information contained in this e-mail is confidential and/or proprietary to Capital One and/or its affiliates and may only be used solely in performance of work or services for Capital One. The information transmitted herewith is intended only for use by the individual or entity to which it is addressed. If the reader of this message is not the intended recipient, you are hereby notified that any review, retransmission, dissemination, distribution, copying or other use of, or taking of any action in reliance upon this information is strictly prohibited. If you have received this communication in error, please contact the sender and delete the material from your computer.

Reply | Threaded
Open this post in threaded view
|

Re: [External Sender] Writing dataframe to vertica

Nikhil Goyal
Got the issue fixed. Had to set 'dbschema' param in connection options. By default it is set to public

Thanks

On Tue, Oct 16, 2018 at 9:49 PM Femi Anthony <[hidden email]> wrote:
How are you trying to write to Vertica ? Can you provide some snippets of code ?


Femi

On Tue, Oct 16, 2018 at 7:24 PM Nikhil Goyal <[hidden email]> wrote:
Hi guys,

I am trying to write dataframe to vertica using spark. It seems like spark is creating a temp table under public schema. I don't have access to public schema hence the job is failing. Is there a way to specify another schema?

Error
ERROR s2v.S2VUtils: createJobStatusTable: FAILED to create job status table public.S2V_JOB_STATUS_USER_NGOYAL
java.lang.Exception: S2V: FATAL ERROR for job S2V_job8087339107009511230. Unable to create status table for tracking this job:public.S2V_JOB_STATUS_USER_NGOYAL

Thanks
Nikhil


The information contained in this e-mail is confidential and/or proprietary to Capital One and/or its affiliates and may only be used solely in performance of work or services for Capital One. The information transmitted herewith is intended only for use by the individual or entity to which it is addressed. If the reader of this message is not the intended recipient, you are hereby notified that any review, retransmission, dissemination, distribution, copying or other use of, or taking of any action in reliance upon this information is strictly prohibited. If you have received this communication in error, please contact the sender and delete the material from your computer.