RESTful API for Spark

classic Classic list List threaded Threaded
5 messages Options
od
Reply | Threaded
Open this post in threaded view
|

RESTful API for Spark

od
Hello,

I am constructing a data pipeline which relies on Spark. As part of this
exercise, I would like to provide my users a web based interface to
submit queries that would translate into Spark jobs. So, something like
"user enters dates and other criteria for e.g. retention analysis" =>
this ends up being some kind of a RESTful API call with appropriate json
sent to server => this ends up being some kind of a spark
.filter.map.reduceByKey thing => this ends up sending a json formatted
reply back to web interface.

Is there anything available now to take care of the RESTful step? If
not, how are people going about setting up this "mini" workflow?

Thanks!
Ognen
Reply | Threaded
Open this post in threaded view
|

Re: RESTful API for Spark

Mayur Rustagi
Check out jobserver on Spark. Haven't used it myself but would love to know your feedback around it. 

On Monday, February 17, 2014, Ognen Duzlevski <[hidden email]> wrote:
Hello,

I am constructing a data pipeline which relies on Spark. As part of this exercise, I would like to provide my users a web based interface to submit queries that would translate into Spark jobs. So, something like "user enters dates and other criteria for e.g. retention analysis" => this ends up being some kind of a RESTful API call with appropriate json sent to server => this ends up being some kind of a spark .filter.map.reduceByKey thing => this ends up sending a json formatted reply back to web interface.

Is there anything available now to take care of the RESTful step? If not, how are people going about setting up this "mini" workflow?

Thanks!
Ognen


--
Sent from Gmail Mobile
od
Reply | Threaded
Open this post in threaded view
|

Re: RESTful API for Spark

od
Mayur,

On 2/17/14, 7:37 PM, Mayur Rustagi wrote:
> Check out jobserver on Spark. Haven't used it myself but would love to
> know your feedback around it.

Thanks. I am not sure what you are referring to? This?
https://github.com/ooyala/incubator-spark
Ognen
Reply | Threaded
Open this post in threaded view
|

Re: RESTful API for Spark

Ognen Duzlevski-2
Sorry, I meant the jobserver-preview branch.
Ognen

On 2/17/14, 10:31 PM, Ognen Duzlevski wrote:
> Mayur,
>
> On 2/17/14, 7:37 PM, Mayur Rustagi wrote:
>> Check out jobserver on Spark. Haven't used it myself but would love
>> to know your feedback around it.
>
> Thanks. I am not sure what you are referring to? This?
> https://github.com/ooyala/incubator-spark
> Ognen

--
Some people, when confronted with a problem, think "I know, I'll use regular expressions." Now they have two problems.
-- Jamie Zawinski

Reply | Threaded
Open this post in threaded view
|

Re: RESTful API for Spark

Mayur Rustagi



On Mon, Feb 17, 2014 at 8:35 PM, Ognen Duzlevski <[hidden email]> wrote:
Sorry, I meant the jobserver-preview branch.
Ognen


On 2/17/14, 10:31 PM, Ognen Duzlevski wrote:
Mayur,

On 2/17/14, 7:37 PM, Mayur Rustagi wrote:
Check out jobserver on Spark. Haven't used it myself but would love to know your feedback around it.

Thanks. I am not sure what you are referring to? This? https://github.com/ooyala/incubator-spark
Ognen

--
Some people, when confronted with a problem, think "I know, I'll use regular expressions." Now they have two problems.
-- Jamie Zawinski