giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Praveen Sripati <>
Subject How does scaling work in Giraph?
Date Thu, 28 Jun 2012 13:34:27 GMT
During the 24th minute of the recent Hadoop Summit Video [1] Avery Ching
talks about how Giraph is made scalable. I am interested in Hama which is
also based on the BSP model and would like to know more details on how
Giraph is made scalable.

Basically, at the end of each super step, the BSP tasks sends some metrics
to the master and the master partitions the data in the most loaded BSP
tasks and uses the free map available slot to process them.

1) Where is the code for the above logic? I am new to Giraph.

2) What is the logic behind the partitioning of the data in the master
after the super step? Let's say that the data has been partitioned using
Hash partitioning.

3) Similarly will Giraph also scale down? Will the partitions be merged?


[1] -

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message