[Beginner] How to control each spark worker node to do some special job?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[Beginner] How to control each spark worker node to do some special job?

ybdesire
This post has NOT been accepted by the mailing list yet.
I have some experience for mult-process programming by python multiprocessing, but limited spark programming experience.

I want to control my spark cluster as multiprocessing programming logic, and my question has been published to stackoverflow at below link.

http://stackoverflow.com/questions/42829629/how-to-control-each-spark-worker-node-to-do-some-special-job

Would you help to provide some spark like programming suggestion for this question?

I will be grateful for any help you can provide.
Loading...