giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lukas Nalezenec <lukas.naleze...@firma.seznam.cz>
Subject Re: Custom assignment of partitions to workers
Date Wed, 25 Mar 2015 20:06:27 GMT
Its same.
GraphPartitionerFactory has got two methods. One for 
WorkerGraphPartitioner and second for MasterGraphPartitioner.

public interface GraphPartitionerFactory ...  {
   MasterGraphPartitioner<I, V, E> createMasterGraphPartitioner();
   WorkerGraphPartitioner<I, V, E> createWorkerGraphPartitioner();
}

Lukas

On 25.3.2015 20:57, Arjun Sharma wrote:
> Thanks for your reply! I understand we can supply the 
> GraphPartitionerFactory class to Giraph using the 
> giraph.graphPartitionerFactoryClass class. Is there a way to supply 
> the MasterGraphPartitioner class?
>
> Thanks!
>
> On Wed, Mar 25, 2015 at 12:54 PM, Lukas Nalezenec 
> <lukas.nalezenec@firma.seznam.cz 
> <mailto:lukas.nalezenec@firma.seznam.cz>> wrote:
> >
> > Hi,
> >
> > There are two interfaces:
> > WorkerGraphPartitioner - Maps vertexes to partitions
> > MasterGraphPartitioner - Maps partitions to workers.
> > So you need custom MasterGraphPartitioner.
> > You dont need any external preprocessing step.
> >
> > Lukas
> >
> >
> > On 25.3.2015 19:51, Arjun Sharma wrote:
> >
> > Hi,
> >
> > I understand we can override the GraphPartitionerFactory class in 
> order to achieve custom partitioning of vertices over partitions. Is 
> there a way to do the same to enable assigning partitions to workers 
> in a custom way (e.g., partition n should be assigned to worker m)? 
> The reason is that it is more beneficial to have partitions that are 
> close to each other in terms of the graph structure placed close to 
> each other on the same worker to minimize network traffic. Otherwise, 
> benefits of overriding  GraphPartitionerFactory may be lost. Let us 
> assume that there is an external preprocessing step that outputs the 
> desired assignment, so can we materialize that assignment over Giraph 
> workers?
> >
> > Thanks!
> >
> >


Mime
View raw message