incubator-giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Avery Ching <ach...@apache.org>
Subject Re: Comparing BSP and MR
Date Fri, 09 Dec 2011 07:21:40 GMT
Hi Praveen,

Answers inline.  Hope that helps!

Avery

On 12/8/11 10:16 PM, Praveen Sripati wrote:
> Hi,
>
> I know about MapReduce/Hadoop and trying to get myself around 
> BSP/Hama-Giraph by comparing MR and BSP.
>
> - Map Phase in MR is similar to Computation Phase in BSP. BSP allows 
> for process to exchange data in the communication phase, but there is 
> no communication between the mappers in the Map Phase. Though the data 
> flows from Map tasks to Reducer tasks. Please correct me if I am 
> wrong. Any other significant differences?
I suppose you can think of it that way.  I like to compare a BSP 
superstep to a MapReduce job since it's computation and communication.
> - After going through the documentation for Hama and Giraph, noticed 
> that they both use Hadoop as the underlying framework. In both Hama 
> and Giraph an MR Job is submitted. Does each superstep in BSP 
> correspond to a Job in MR? Where are the incoming, outgoing messages 
> and state stored - HDFS or HBase or Local or pluggable?
>
My understanding of Hama is that they have their own BSP framework.  
Giraph can be run on a Hadoop installation, it does not have its own 
computational framework.  A Giraph job is submitted to a Hadoop 
installation as a Map-only job.  Hama will have its own BSP lauching 
framework.

In Giraph, the state is stored all in memory.  Graphs are loaded/stored 
through VertexInputFormat/VertexOutputFormat (very similar to Hadoop).  
You could implement your own VertexInputFormat/VertexOutputFormat to use 
HDFS, HBase, etc. as your graph stable storage.

> - If a Vertex is deactivated and again activated after receiving a 
> message, does is run on the same node or a different node in the cluster?
>
In Giraph, vertices can move around workers between supersteps.  A 
vertex will run on the worker that it is assigned to.

> Regards,
> Praveen


Mime
View raw message