Mailing-List: contact dev-help@hama.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hama.apache.org
Received-SPF: pass (athena.apache.org: domain of micle.bu@gmail.com designates
 74.125.83.42 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CALis4o4SKN4zpOOCQ2ggfQ7TsGX4_9UZPc+L_nPsLmZvwSL0Lg@mail.gmail.com>
References: 
 <CAH3Bz6W=cnHrDgP-Kun-3dQN6OMXmUDTTJaMW0Cb7Qrg8M==-A@mail.gmail.com>
	<CALis4o4SKN4zpOOCQ2ggfQ7TsGX4_9UZPc+L_nPsLmZvwSL0Lg@mail.gmail.com>
Date: Wed, 17 Apr 2013 23:46:31 +0800
Message-ID: 
 <CAH3Bz6W4FChdHVR-UcyEDk9EUnwwmNVx4U7GcQBxcZK7DYaQiA@mail.gmail.com>
Subject: Re: Data locality in Hama
From: Micle Bu <micle.bu@gmail.com>
To: dev@hama.apache.org
Content-Type: multipart/alternative; boundary=089e01635046cfb7d004da9063e2

--089e01635046cfb7d004da9063e2
Content-Type: text/plain; charset=ISO-8859-1

Thanks Suraj!

Good, another way is to select the groom server that has most blocks/parts
of split(when splitSize > HDFS blocksize, split may has several HDFS
blocks) to obtain maximum locality!
Taking hostname in different rack is another good hint.

Micle Bu


On Wed, Apr 17, 2013 at 11:29 PM, Suraj Menon <surajsmenon@apache.org>wrote:

> Good catch! Yes the logic is to find the first groom server that has the
> split and has available slots for execution.
> You might note that depending on the HDFS allocation, this hostname might
> not be in the same rack. You are welcome to fix this.
>
>
> On Wed, Apr 17, 2013 at 11:15 AM, Micle Bu <micle.bu@gmail.com> wrote:
>
> > Hi all,
> >
> > I'm learning data locality in Hama, and found there is a
> > BestEffortDataLocalTaskAllocator class for this purpose. It's a good idea
> > to assign task to the groom which contains its split,
> getGroomToSchedule()
> > play this role.
> >
> > Well, in getGroomToSchedule(), the code like:
> >
> > GroomServerStatus groom = grooms.get(location);
> > ...
> > if (taskInGroom < groom.getMaxTasks() &&
> > location.equals(groom.getGroomHostName())) {
> >         return groom.getGroomHostName();
> > }
> >
> > It seems that location.equals(groom.getGroomHostName() is always true, so
> > it just select the first groom which contains split? Am i right?
> >
> > Thanks in advance!
> >
> > Micle Bu
> >
>

--089e01635046cfb7d004da9063e2--