Mailing-List: contact mapreduce-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-dev@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of harsh@cloudera.com designates
 209.85.160.48 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAO0LRM9b2jMyu3j=Mbx1eDaQwSEyZJY0eT2JX=d78p80X9W8tQ@mail.gmail.com>
References: 
 <CAO0LRM9b2jMyu3j=Mbx1eDaQwSEyZJY0eT2JX=d78p80X9W8tQ@mail.gmail.com>
From: Harsh J <harsh@cloudera.com>
Date: Wed, 1 Feb 2012 19:10:01 +0530
Message-ID: 
 <CAOcnVr2purppi689VvAnSaf0suYPbWF0WVD1ZFd5yZhRwAPChg@mail.gmail.com>
Subject: Re: regarding to sort and reducer
To: mapreduce-dev@hadoop.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Samaneh,

Sorry for the late response. Inline, some of what I can offer.

On Sat, Jan 21, 2012 at 10:51 PM, Samaneh Shokuhi
<samaneh.shokuhi@gmail.com> wrote:
> Hi All,

Welcome!

> I am very new to hadoop and going to do some research on it regarding to =
my
> master thesis. First of all What i want to do is to know the fuctionality
> of sort =A0and shuffle and to run an applition while hadoop included =A0a=
nd not
> included sort part.
> I need to know which class in hadoop is taking care of sort ?

Are you looking for the sort mechanism or the algorithm?

This is an excellent presentation on the MR sort/shuffle/merge layers
that I recommend reading:
http://www.slideshare.net/hadoopusergroup/ordered-record-collection

> Another thing i need to know is the functianlity of reducer and to find o=
ut
> the possibility of sending message from one reducer to another one and
> doing kind of work stealing between reducers.

You probably want to read ReduceTask class, but this functionality is
not present today. Perhaps easier to do with the new MR2 framework,
detailed in http://developer.yahoo.com/blogs/hadoop/posts/2011/03/mapreduce=
-nextgen-scheduler/

> Since i am very new to hadoop and it has alot of modules ,i need to know
> which project should i look at it.
> Also i ll appriciate you to let me know if you have any comment on this
> idea.

You need to look at the hadoop-mapreduce-project in trunk for all
things MR today. It also uses some generic components from the
hadoop-common project. See
http://wiki.apache.org/hadoop/HowToContribute for more details.

Please feel free to mail the lists with any specific questions you
have as you go ahead!

--=20
Harsh J
Customer Ops. Engineer
Cloudera | http://tiny.cloudera.com/about