giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arjun Sharma <>
Subject Custom assignment of partitions to workers
Date Wed, 25 Mar 2015 18:51:32 GMT

I understand we can override the GraphPartitionerFactory class in order to
achieve custom partitioning of vertices over partitions. Is there a way to
do the same to enable assigning partitions to workers in a custom way
(e.g., partition n should be assigned to worker m)? The reason is that it
is more beneficial to have partitions that are close to each other in terms
of the graph structure placed close to each other on the same worker to
minimize network traffic. Otherwise, benefits of
overriding  GraphPartitionerFactory may be lost. Let us assume that there
is an external preprocessing step that outputs the desired assignment, so
can we materialize that assignment over Giraph workers?


View raw message