From core-user-return-11052-apmail-hadoop-core-user-archive=hadoop.apache.org@hadoop.apache.org Mon Jan 05 23:59:32 2009 Return-Path: Delivered-To: apmail-hadoop-core-user-archive@www.apache.org Received: (qmail 89902 invoked from network); 5 Jan 2009 23:59:32 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 5 Jan 2009 23:59:32 -0000 Received: (qmail 22213 invoked by uid 500); 5 Jan 2009 23:59:26 -0000 Delivered-To: apmail-hadoop-core-user-archive@hadoop.apache.org Received: (qmail 21597 invoked by uid 500); 5 Jan 2009 23:59:25 -0000 Mailing-List: contact core-user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: core-user@hadoop.apache.org Delivered-To: mailing list core-user@hadoop.apache.org Received: (qmail 21586 invoked by uid 99); 5 Jan 2009 23:59:25 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 05 Jan 2009 15:59:25 -0800 X-ASF-Spam-Status: No, hits=1.2 required=10.0 tests=SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [76.96.62.48] (HELO QMTA05.westchester.pa.mail.comcast.net) (76.96.62.48) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 05 Jan 2009 23:59:17 +0000 Received: from OMTA14.westchester.pa.mail.comcast.net ([76.96.62.60]) by QMTA05.westchester.pa.mail.comcast.net with comcast id zvVH1a00R1HzFnQ55zywaa; Mon, 05 Jan 2009 23:58:56 +0000 Received: from [10.1.12.52] ([209.131.62.115]) by OMTA14.westchester.pa.mail.comcast.net with comcast id zzya1a0032VBGtd3azygnG; Mon, 05 Jan 2009 23:58:51 +0000 Message-Id: <148383D5-4C24-48F9-8923-6C951D8F9B49@apache.org> From: Owen O'Malley To: core-user@hadoop.apache.org In-Reply-To: <93dd73db0901051407x21d6f68am2f43ff185621986b@mail.gmail.com> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Subject: Re: correct pattern for using setOutputValueGroupingComparator? Date: Mon, 5 Jan 2009 15:58:32 -0800 References: <93dd73db0901051407x21d6f68am2f43ff185621986b@mail.gmail.com> X-Mailer: Apple Mail (2.930.3) X-Virus-Checked: Checked by ClamAV on apache.org This is exactly what the setOutputValueGroupingComparator is for. Take a look at HADOOP-4545, for an example using the secondary sort. If you are using trunk or 0.20, look at src/examples/org/apache/hadoop/ examples/SecondarySort.java. The checked in example uses the new map/ reduce api that was introduced in 0.20. -- Owen