Shared access to RDD

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Shared access to RDD

David Thomas
Is it possible for multiple users to access the same RDD, say, from their respective spark-shells?
Reply | Threaded
Open this post in threaded view
|

Re: Shared access to RDD

Koert Kuipers
it is possible to run multiple queries using a shared SparkContext (which holds the shared RDD). however this is not easily available in spark-shell i believe.

alternatively tachyon can be used to share (serialized) RDDs


On Mon, Feb 17, 2014 at 11:41 AM, David Thomas <[hidden email]> wrote:
Is it possible for multiple users to access the same RDD, say, from their respective spark-shells?

Reply | Threaded
Open this post in threaded view
|

Re: Shared access to RDD

Mayur Rustagi
In reply to this post by David Thomas
No not right now...
1. You can look at jobserver opensourced by Ooyala which allows multiple codes access to same rdd through rest interface
2. You can ofcourse go with shark server and load the data as sql table for access, but that may or may not work for you as the RDD structure is predefined by shark.
Its a popular request I believe..
Regards
Mayur



On Mon, Feb 17, 2014 at 8:41 AM, David Thomas <[hidden email]> wrote:
Is it possible for multiple users to access the same RDD, say, from their respective spark-shells?