Async API to save RDDs?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Async API to save RDDs?

Antonin Delpeuch (lists)
Hi,

The RDD API provides async variants of a few RDD methods, which let the
user execute the corresponding jobs asynchronously. This makes it
possible to cancel the jobs for instance:
https://spark.apache.org/docs/latest/api/java/org/apache/spark/rdd/AsyncRDDActions.html

There does not seem to be async versions of the save methods such as
`saveAsTextFile`:
https://spark.apache.org/docs/latest/api/java/org/apache/spark/rdd/RDD.html#saveAsTextFile-java.lang.String-

Is there another way to start such jobs and get a handle on them (such
as the job id)? Specifically, I would like to be able to stop save jobs
on user request.

Thank you,
Antonin

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]