What's the best way to have Spark a service?

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

What's the best way to have Spark a service?

David Espinosa
Hi all,

I'm quite new to Spark, and I would like to ask whats the best way to have Spark as a service, and for that I mean being able to include the response of a scala app/job running in a Spark into a RESTful common request.

Up now I have read about Apache Livy (which I tried and found incompatibility problems with my scala app), Spark Jobsserver, Finch and I read that I could use also Spark Streams.

Thanks in advance,
David
Reply | Threaded
Open this post in threaded view
|

Re: What's the best way to have Spark a service?

Liana Napalkova

Hi David,

Which type of incompatibility problems do you have with Apache Livy?

BR,

Liana



From: David Espinosa <[hidden email]>
Sent: 15 March 2018 12:06:20
To: [hidden email]
Subject: What's the best way to have Spark a service?
 
Hi all,

I'm quite new to Spark, and I would like to ask whats the best way to have Spark as a service, and for that I mean being able to include the response of a scala app/job running in a Spark into a RESTful common request.

Up now I have read about Apache Livy (which I tried and found incompatibility problems with my scala app), Spark Jobsserver, Finch and I read that I could use also Spark Streams.

Thanks in advance,
David

DISCLAIMER: Aquest missatge pot contenir informació confidencial. Si vostè no n'és el destinatari, si us plau, esborri'l i faci'ns-ho saber immediatament a la següent adreça: [hidden email] Si el destinatari d'aquest missatge no consent la utilització del correu electrònic via Internet i la gravació de missatges, li preguem que ens ho comuniqui immediatament.

DISCLAIMER: Este mensaje puede contener información confidencial. Si usted no es el destinatario del mensaje, por favor bórrelo y notifíquenoslo inmediatamente a la siguiente dirección: [hidden email] Si el destinatario de este mensaje no consintiera la utilización del correo electrónico vía Internet y la grabación de los mensajes, rogamos lo ponga en nuestro conocimiento de forma inmediata.

DISCLAIMER: Privileged/Confidential Information may be contained in this message. If you are not the addressee indicated in this message you should destroy this message, and notify us immediately to the following address: [hidden email]. If the addressee of this message does not consent to the use of Internet e-mail and message recording, please notify us immediately.


 

jgp
Reply | Threaded
Open this post in threaded view
|

Re: What's the best way to have Spark a service?

jgp
In reply to this post by David Espinosa
Hi David,

I ended building up my own. Livy sounded great on paper, but heavy to manipulate. I found out about Jobserver too late. We did not find too complicated to build ours, with a small Spring boot app that was holding the session (we did not need more than one session).

jg


> On Mar 15, 2018, at 07:06, David Espinosa <[hidden email]> wrote:
>
> Hi all,
>
> I'm quite new to Spark, and I would like to ask whats the best way to have Spark as a service, and for that I mean being able to include the response of a scala app/job running in a Spark into a RESTful common request.
>
> Up now I have read about Apache Livy (which I tried and found incompatibility problems with my scala app), Spark Jobsserver, Finch and I read that I could use also Spark Streams.
>
> Thanks in advance,
> David


---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]