Mailing-List: contact dev-help@giraph.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@giraph.apache.org
Message-ID: <4FF15534.5060906@apache.org>
Date: Mon, 02 Jul 2012 01:00:52 -0700
From: Avery Ching <aching@apache.org>
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7;
 rv:13.0) Gecko/20120614 Thunderbird/13.0.1
MIME-Version: 1.0
To: dev@giraph.apache.org
Subject: Re: How does scaling work in Giraph?
References: 
 <CADYHM8xrnbKbXe7P7mR1b4oVX9mzh=nnEUE1oxV1GjZpQdKjuA@mail.gmail.com>
 <CADYHM8xtTBEDufAunJpOKpnoUv3CK1=d5xdMa0OrnwNf3f-SCA@mail.gmail.com>
In-Reply-To: 
 <CADYHM8xtTBEDufAunJpOKpnoUv3CK1=d5xdMa0OrnwNf3f-SCA@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Praveen, response inline.  Hope it's helpful.

On 6/30/12 10:47 AM, Praveen Sripati wrote:
> Could someone respond to the below mail please?
>
> Thanks,
> Praveen
>
> On Thu, Jun 28, 2012 at 7:04 PM, Praveen Sripati
> <praveensripati@gmail.com>wrote:
>
>> During the 24th minute of the recent Hadoop Summit Video [1] Avery Ching
>> talks about how Giraph is made scalable. I am interested in Hama which is
>> also based on the BSP model and would like to know more details on how
>> Giraph is made scalable.
>>
>> Basically, at the end of each super step, the BSP tasks sends some metrics
>> to the master and the master partitions the data in the most loaded BSP
>> tasks and uses the free map available slot to process them.
>>
>> 1) Where is the code for the above logic? I am new to Giraph.
See BspWorker#finishSuperstep()

>> 2) What is the logic behind the partitioning of the data in the master
>> after the super step? Let's say that the data has been partitioned using
>> Hash partitioning.
See GraphPartitionerFactory
>> 3) Similarly will Giraph also scale down? Will the partitions be merged?

This is totally up to the implementation of GraphPartitionerFactory.

>> Thanks,
>> Praveen
>>
>> [1] - http://www.youtube.com/watch?v=b5Qmz4zPj-M
>>