giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Koch <>
Subject Multiple node types in Giraph and doing a selective M/R over one of them
Date Mon, 28 Jan 2013 13:52:48 GMT

In Giraph is it possible to have different node types in a graph and have a
Map/Reduce only iterate over nodes of this type and their direct successors?

If it sounds a bit cryptic here is something more about our use-case:
We have different HBase tables which we want to "pseudo-join" in Map/Reduce
computations. The node types I mentioned above correspond to the respective
row-key types used in each of those tables, edges are generated by the fact
that the KeyValues in each table can contain row-key values found in one of
the other tables.

The graph would describe these relations. In a Map/Reduce I then want to be
able to iterate over all nodes of a given type while also having access to
a node's successor nodes in the same Mapper instance or better yet the same
map() call. One would then carry out additional Gets to retrieve the data
from the tables thus doing a fairly crude join.

The Graph is likely to change so it would be nice if it could be updated

Does all this sound like something that would be possible with Giraph?

Thank you,


View raw message