Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of harsh@cloudera.com designates
 209.85.223.178 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAOhq9BV_EHi4c0ETumb0djAyHM+NDBsWxfwtSerEYQ7Ayt2jhA@mail.gmail.com>
References: 
 <CAOhq9BV_EHi4c0ETumb0djAyHM+NDBsWxfwtSerEYQ7Ayt2jhA@mail.gmail.com>
From: Harsh J <harsh@cloudera.com>
Date: Wed, 14 Aug 2013 10:58:39 +0530
Message-ID: 
 <CAOcnVr0StuGPQOq-p01fQYNgExuOG84Q8J0RHa4UC0P4LggXyQ@mail.gmail.com>
Subject: Re: Reduce Task Clarification
To: "<user@hadoop.apache.org>" <user@hadoop.apache.org>
Content-Type: text/plain; charset=ISO-8859-1

Are you looking to do a secondary sort under a grouped key?

A reduce() is called once for each globally unique map() emitted key,
along with all values grouped for it. To sort the grouped data, you
need to use a separate sort comparator and perform the 'secondary
sort'.

On Wed, Aug 14, 2013 at 1:21 AM, Sam Garrett <sam@actionx.com> wrote:
> I am working on a MapReduce job where I would like to have the output sorted
> by a LongWritable value. I read the Anatomy of a MapReduce Run in the
> Definitive Guide and it didn't say explicitly whether reduce() gets called
> only once per map output key. If it does get called only once I was thinking
> that I could use this:
> http://hadoop.apache.org/docs/current/api/org/apache/hadoop/mapreduce/Job.html#setSortComparatorClass(java.lang.Class)
> to do the sorting.
>
> Thank you for your time.
>
> --
> Sam Garrett
> ActionX, NYC


-- 
Harsh J