Spark batch job chaining

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Spark batch job chaining

Amit Sharma-2
Hi, I want to write a batch job which would call another batch job based on condition. Can I call one batch job through another in scala or I can do it just by python script. Example would be really helpful.


Thanks
Amit
Reply | Threaded
Open this post in threaded view
|

Re: Spark batch job chaining

Amit Sharma-2
Any help is appreciated. I have spark batch job based on condition I would like to start another batch job by invoking .sh file. Just want to know can we achieve that?

Thanks
Amit

On Fri, Aug 7, 2020 at 3:58 PM Amit Sharma <[hidden email]> wrote:
Hi, I want to write a batch job which would call another batch job based on condition. Can I call one batch job through another in scala or I can do it just by python script. Example would be really helpful.


Thanks
Amit


Reply | Threaded
Open this post in threaded view
|

Re: Spark batch job chaining

Jun Zhu-2
Hi
I am using Airflow in such scenario