Mesos + Spark users going forward?

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Mesos + Spark users going forward?

Sean Owen
I noted that Apache Mesos is moving to the attic, so won't be actively developed soon:

That doesn't mean people will stop using it as a Spark resource manager soon. But it suggests the SparkĀ + Mesos integration is a candidate for deprecation and eventual removal in Spark at some point.

This is mostly an informal poll: are there Mesos users out there planning to continue using it for a while? moving off? just seeing if it's reasonable to even start deprecation in 3.2.0.

Sean

Reply | Threaded
Open this post in threaded view
|

Re: Mesos + Spark users going forward?

Mridul Muralidharan

Unfortunate about Mesos, +1 on deprecation of mesos integration.

Regards,
Mridul


On Wed, Apr 7, 2021 at 7:12 AM Sean Owen <[hidden email]> wrote:
I noted that Apache Mesos is moving to the attic, so won't be actively developed soon:

That doesn't mean people will stop using it as a Spark resource manager soon. But it suggests the SparkĀ + Mesos integration is a candidate for deprecation and eventual removal in Spark at some point.

This is mostly an informal poll: are there Mesos users out there planning to continue using it for a while? moving off? just seeing if it's reasonable to even start deprecation in 3.2.0.

Sean

Reply | Threaded
Open this post in threaded view
|

Re: Mesos + Spark users going forward?

dmcwhorter
In reply to this post by Sean Owen
We are using the mesos integration at Premier (https://www.premierinc.com/).
Obviously with the move to the attic we will likely move away from Mesos in
the future.  I think deprecating the mesos integration makes sense.  We
would probably continue to utilize the spark mesos components for another
release or two if its possible to include them for a little bit longer
before they're removed.



--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]