How to submit a job via REST API?

[Zhou Yang]

Dear experts,

I found a convenient way to submit job via Rest API at https://gist.github.com/arturmkrtchyan/5d8559b2911ac951d34a#file-submit_job-sh.

But I did not know whether can I append `—conf` parameter like what I did in spark-submit. Can someone can help me with this issue?

Regards, Yang

[Dennis Suhari]

Hi Yang,

I am using Livy Server for submitting jobs.

Br,

Dennis

Von meinem iPhone gesendet

Am 24.11.2020 um 03:34 schrieb Zhou Yang <[hidden email]>:



Dear experts,

I found a convenient way to submit job via Rest API at https://gist.github.com/arturmkrtchyan/5d8559b2911ac951d34a#file-submit_job-sh.

But I did not know whether can I append `—conf` parameter like what I did in spark-submit. Can someone can help me with this issue?

Regards, Yang

[Sonal Goyal]

You should be able to supply the --conf and its values as part of appArgs argument

On Tue, Nov 24, 2020 at 11:31 AM Dennis Suhari <[hidden email]> wrote:

Hi Yang,

I am using Livy Server for submitting jobs.

Br,

Dennis

Von meinem iPhone gesendet

Am 24.11.2020 um 03:34 schrieb Zhou Yang <[hidden email]>:



Dear experts,

I found a convenient way to submit job via Rest API at https://gist.github.com/arturmkrtchyan/5d8559b2911ac951d34a#file-submit_job-sh.

But I did not know whether can I append `—conf` parameter like what I did in spark-submit. Can someone can help me with this issue?

Regards, Yang

[vaquar khan]

Hi Yang,

Please find following link 

https://stackoverflow.com/questions/63677736/spark-application-as-a-rest-service/63678337#63678337

Regards,

Vaquar khan

On Wed, Nov 25, 2020 at 12:40 AM Sonal Goyal <[hidden email]> wrote:

You should be able to supply the --conf and its values as part of appArgs argument

Cheers,
Sonal
Nube Technologies 

Join me at

Data Con LA Oct 23 | Big Data Conference Europe. Nov 24 | GIDS AI/ML Dec 3

On Tue, Nov 24, 2020 at 11:31 AM Dennis Suhari <[hidden email]> wrote:

Hi Yang,

I am using Livy Server for submitting jobs.

Br,

Dennis

Von meinem iPhone gesendet

Am 24.11.2020 um 03:34 schrieb Zhou Yang <[hidden email]>:



Dear experts,

I found a convenient way to submit job via Rest API at https://gist.github.com/arturmkrtchyan/5d8559b2911ac951d34a#file-submit_job-sh.

But I did not know whether can I append `—conf` parameter like what I did in spark-submit. Can someone can help me with this issue?

Regards, Yang

--

[Zhou Yang]

Hi all,

I found the solution through the source code. Appending the —conf k-v into `sparkProperties` work.

For example:

./spark-submit \

—conf foo=bar \

xxx

equals to

{

“xxx” : “yyy”,

“sparkProperties” : {

“foo": "bar"

}

}

Thanks for your reply.

2020年11月25日 下午3:55，vaquar khan <[hidden email]> 写道：

Hi Yang,

Please find following link 

https://stackoverflow.com/questions/63677736/spark-application-as-a-rest-service/63678337#63678337

Regards,

Vaquar khan

On Wed, Nov 25, 2020 at 12:40 AM Sonal Goyal <[hidden email]> wrote:

You should be able to supply the --conf and its values as part of appArgs argument

Cheers,
Sonal
Nube Technologies 

Join me at

Data Con LA Oct 23 | Big Data Conference Europe. Nov 24 | GIDS AI/ML Dec 3

On Tue, Nov 24, 2020 at 11:31 AM Dennis Suhari <[hidden email]> wrote:

Hi Yang,

I am using Livy Server for submitting jobs.

Br,

Dennis

Von meinem iPhone gesendet

Am 24.11.2020 um 03:34 schrieb Zhou Yang <[hidden email]>:



Dear experts,

I found a convenient way to submit job via Rest API at https://gist.github.com/arturmkrtchyan/5d8559b2911ac951d34a#file-submit_job-sh.

But I did not know whether can I append `—conf` parameter like what I did in spark-submit. Can someone can help me with this issue?

Regards, Yang

--

[Daniel de Oliveira Mantovani]

In my opinion this should be part of the official documentation. Amazing work Zhou Yang.

On Wed, Nov 25, 2020 at 5:45 AM Zhou Yang <[hidden email]> wrote:

Hi all,

I found the solution through the source code. Appending the —conf k-v into `sparkProperties` work.

For example:

./spark-submit \

—conf foo=bar \

xxx

equals to

{

“xxx” : “yyy”,

“sparkProperties” : {

“foo": "bar"

}

}

Thanks for your reply.

2020年11月25日 下午3:55，vaquar khan <[hidden email]> 写道：

Hi Yang,

Please find following link 

https://stackoverflow.com/questions/63677736/spark-application-as-a-rest-service/63678337#63678337

Regards,

Vaquar khan

On Wed, Nov 25, 2020 at 12:40 AM Sonal Goyal <[hidden email]> wrote:

You should be able to supply the --conf and its values as part of appArgs argument

Cheers,
Sonal
Nube Technologies 

Join me at

Data Con LA Oct 23 | Big Data Conference Europe. Nov 24 | GIDS AI/ML Dec 3

On Tue, Nov 24, 2020 at 11:31 AM Dennis Suhari <[hidden email]> wrote:

Hi Yang,

I am using Livy Server for submitting jobs.

Br,

Dennis

Von meinem iPhone gesendet

Am 24.11.2020 um 03:34 schrieb Zhou Yang <[hidden email]>:



Dear experts,

I found a convenient way to submit job via Rest API at https://gist.github.com/arturmkrtchyan/5d8559b2911ac951d34a#file-submit_job-sh.

But I did not know whether can I append `—conf` parameter like what I did in spark-submit. Can someone can help me with this issue?

Regards, Yang

--

Regards,

Vaquar Khan

[phone]

Greater Chicago

--