GraphX performance feedback

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

GraphX performance feedback

mahzad kalantari
Hi all

My question is about GraphX, I 'm looking for user feedbacks on the performance.

I read this paper written by Facebook team that says Graphx has very poor performance.

Has anyone already encountered performance problems with Graphx, and is it a good choice if I want to do large scale graph modelling?


Thanks!

Mahzad 
Reply | Threaded
Open this post in threaded view
|

Re: GraphX performance feedback

Jörn Franke
I think it depends what you want do. Interactive big data graph analytics are probably better of in Janusgraph or similar. 
Batch processing (once-off) can be still fine in graphx - you have though to carefully design the process. 

Am 25.11.2019 um 20:04 schrieb mahzad kalantari <[hidden email]>:


Hi all

My question is about GraphX, I 'm looking for user feedbacks on the performance.

I read this paper written by Facebook team that says Graphx has very poor performance.

Has anyone already encountered performance problems with Graphx, and is it a good choice if I want to do large scale graph modelling?


Thanks!

Mahzad 
Reply | Threaded
Open this post in threaded view
|

Re: GraphX performance feedback

mahzad kalantari
Thanks for your answer, my use case is friend recommandation for 200 million profils.

Le lun. 25 nov. 2019 à 14:10, Jörn Franke <[hidden email]> a écrit :
I think it depends what you want do. Interactive big data graph analytics are probably better of in Janusgraph or similar. 
Batch processing (once-off) can be still fine in graphx - you have though to carefully design the process. 

Am 25.11.2019 um 20:04 schrieb mahzad kalantari <[hidden email]>:


Hi all

My question is about GraphX, I 'm looking for user feedbacks on the performance.

I read this paper written by Facebook team that says Graphx has very poor performance.

Has anyone already encountered performance problems with Graphx, and is it a good choice if I want to do large scale graph modelling?


Thanks!

Mahzad 
Reply | Threaded
Open this post in threaded view
|

Re: GraphX performance feedback

Phillip Henry
I saw a large improvement in my GraphX processing by:

- using fewer partitions
- using fewer executors but with much more memory.

YMMV.

Phillip

On Mon, 25 Nov 2019, 19:14 mahzad kalantari, <[hidden email]> wrote:
Thanks for your answer, my use case is friend recommandation for 200 million profils.

Le lun. 25 nov. 2019 à 14:10, Jörn Franke <[hidden email]> a écrit :
I think it depends what you want do. Interactive big data graph analytics are probably better of in Janusgraph or similar. 
Batch processing (once-off) can be still fine in graphx - you have though to carefully design the process. 

Am 25.11.2019 um 20:04 schrieb mahzad kalantari <[hidden email]>:


Hi all

My question is about GraphX, I 'm looking for user feedbacks on the performance.

I read this paper written by Facebook team that says Graphx has very poor performance.

Has anyone already encountered performance problems with Graphx, and is it a good choice if I want to do large scale graph modelling?


Thanks!

Mahzad 
Reply | Threaded
Open this post in threaded view
|

Re: GraphX performance feedback

mahzad kalantari
Ok thanks!

Le jeu. 28 nov. 2019 à 11:27, Phillip Henry <[hidden email]> a écrit :
I saw a large improvement in my GraphX processing by:

- using fewer partitions
- using fewer executors but with much more memory.

YMMV.

Phillip

On Mon, 25 Nov 2019, 19:14 mahzad kalantari, <[hidden email]> wrote:
Thanks for your answer, my use case is friend recommandation for 200 million profils.

Le lun. 25 nov. 2019 à 14:10, Jörn Franke <[hidden email]> a écrit :
I think it depends what you want do. Interactive big data graph analytics are probably better of in Janusgraph or similar. 
Batch processing (once-off) can be still fine in graphx - you have though to carefully design the process. 

Am 25.11.2019 um 20:04 schrieb mahzad kalantari <[hidden email]>:


Hi all

My question is about GraphX, I 'm looking for user feedbacks on the performance.

I read this paper written by Facebook team that says Graphx has very poor performance.

Has anyone already encountered performance problems with Graphx, and is it a good choice if I want to do large scale graph modelling?


Thanks!

Mahzad