giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arjun Sharma <>
Subject Re: Custom assignment of partitions to workers
Date Wed, 25 Mar 2015 19:57:39 GMT
Thanks for your reply! I understand we can supply the
GraphPartitionerFactory class to Giraph using the
giraph.graphPartitionerFactoryClass class. Is there a way to supply the
MasterGraphPartitioner class?


On Wed, Mar 25, 2015 at 12:54 PM, Lukas Nalezenec <> wrote:
> Hi,
> There are two interfaces:
> WorkerGraphPartitioner - Maps vertexes to partitions
> MasterGraphPartitioner - Maps partitions to workers.
> So you need custom MasterGraphPartitioner.
> You dont need any external preprocessing step.
> Lukas
> On 25.3.2015 19:51, Arjun Sharma wrote:
> Hi,
> I understand we can override the GraphPartitionerFactory class in order
to achieve custom partitioning of vertices over partitions. Is there a way
to do the same to enable assigning partitions to workers in a custom way
(e.g., partition n should be assigned to worker m)? The reason is that it
is more beneficial to have partitions that are close to each other in terms
of the graph structure placed close to each other on the same worker to
minimize network traffic. Otherwise, benefits of overriding
 GraphPartitionerFactory may be lost. Let us assume that there is an
external preprocessing step that outputs the desired assignment, so can we
materialize that assignment over Giraph workers?
> Thanks!

View raw message