Mailing-List: contact core-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of taiping.du@gmail.com
 designates 209.85.128.187 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=message-id:date:from:to:subject:mime-version:content-type;
        b=ODKAQUVXs38jEeNZYWyHxmd5C4bhaCYI8tEK0gpQN45KLBy6q7hDnTF0yIOyepPCOy
         A6ZhwOYOP31kUOuyexyJM//3ZdcTTbNeZiTxzLOzgIZKQgRpSb40Gm/Ty3OaLyHxle6w
         tH4oNumBQHPSY0iTS/cRD2jaO3ksx5iJ6z9Sc=
Message-ID: <710ef8220808260039l68b8ab03u5e3fd306c47e9458@mail.gmail.com>
Date: Tue, 26 Aug 2008 00:39:33 -0700
From: "charles du" <taiping.du@gmail.com>
To: core-user@hadoop.apache.org
Subject: questions on sorting big files and sorting order
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----=_Part_1650_20847000.1219736373690"

------=_Part_1650_20847000.1219736373690
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

Hi all:

I would like to sort a large number of records in a big file based on a
given field (key).

If I run just one reducer, it works fine because the reducer will sort all
records based on the key. To increase the sorting performance, I would like
to run multiple reducers, how can I guarantee the order of records that got
partitioned to different reducers?

Also, the default order is ascending. How can I program my reducer to output
records in descending order? My key could be IntWritable, or Text. Thanks.


-- 
tp

------=_Part_1650_20847000.1219736373690--