GraphX: Get edges for a vertex

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

GraphX: Get edges for a vertex

Daniil Osipov
Hello,

I'm attempting to implement a clustering algorithm on top of Pregel implementation in GraphX, however I'm hitting a wall. Ideally, I'd like to be able to get all edges for a specific vertex, since they factor into the calculation. My understanding was that sendMsg function would receive all relevant edges in participating vertices (all initially, declining as they converge and stop changing state), and I was planning to keep vertex edges associated to each vertex and propagate to other vertices that need to know about these edges.

What I'm finding is that not all edges get iterated on by sendMsg before sending messages to vprog. Even if I try to keep track of edges, I don't account all of them, leading to incorrect results.

The graph I'm testing on has edges between all nodes, one for each direction, and I'm using EdgeDirection.Both.

Anyone seen something similar, and have some suggestions?
Dan
Reply | Threaded
Open this post in threaded view
|

Re: GraphX: Get edges for a vertex

Takeshi Yamamuro
Hi, 

I think that there are two solutions;

1. Invalid edges send Iterator.empty messages in sendMsg of the Pregel API.
These messages make no effect on corresponding vertices.

2. Use GraphOps.(collectNeighbors/collectNeighborIds), not the Pregel API  so as to
handle active edge lists by yourself.
I think that it is hard to handle  edge active lists in Pregel APIs.

Thought?

Best regards,
takeshi

On Fri, Nov 14, 2014 at 7:32 AM, Daniil Osipov <[hidden email]> wrote:
Hello,

I'm attempting to implement a clustering algorithm on top of Pregel implementation in GraphX, however I'm hitting a wall. Ideally, I'd like to be able to get all edges for a specific vertex, since they factor into the calculation. My understanding was that sendMsg function would receive all relevant edges in participating vertices (all initially, declining as they converge and stop changing state), and I was planning to keep vertex edges associated to each vertex and propagate to other vertices that need to know about these edges.

What I'm finding is that not all edges get iterated on by sendMsg before sending messages to vprog. Even if I try to keep track of edges, I don't account all of them, leading to incorrect results.

The graph I'm testing on has edges between all nodes, one for each direction, and I'm using EdgeDirection.Both.

Anyone seen something similar, and have some suggestions?
Dan


mas
Reply | Threaded
Open this post in threaded view
|

Re: GraphX: Get edges for a vertex

mas
This post has NOT been accepted by the mailing list yet.
Hi,

Just to continue with the question.
I need to find the edges of one particular vertex. However, (collectNeighbors/collectNeighborIds) provides the neighbors/neighborids for all the vertices of the graph.
Any help in this regard will be highly appreciated.
Thanks,
Reply | Threaded
Open this post in threaded view
|

Re: GraphX: Get edges for a vertex

abhijeet.srivastava
This post has NOT been accepted by the mailing list yet.
Hi,

Will graphtriplets work for you? I circumvented the problem using triplets view , see if it helps.
e.g:
a triplet is of form ((srcId, (srcAttr)),(dstId,(dstAttr)),(attr)).
> Filter the triplet where the srcId (or dstID, or both) is the vertex you want.
> and then use the "attr" attribute (which is an edge between the two nodes) for the end result.

Thanks,