Return-Path: X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 3D33AD878 for ; Mon, 17 Sep 2012 09:24:08 +0000 (UTC) Received: (qmail 47942 invoked by uid 500); 17 Sep 2012 09:24:03 -0000 Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org Received: (qmail 47484 invoked by uid 500); 17 Sep 2012 09:23:58 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 47454 invoked by uid 99); 17 Sep 2012 09:23:57 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 17 Sep 2012 09:23:57 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=FSL_RCVD_USER,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of lin.yang.jason@gmail.com designates 209.85.220.176 as permitted sender) Received: from [209.85.220.176] (HELO mail-vc0-f176.google.com) (209.85.220.176) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 17 Sep 2012 09:23:51 +0000 Received: by vcbgb22 with SMTP id gb22so495967vcb.35 for ; Mon, 17 Sep 2012 02:23:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=JyDcywi2dfrSuuq7dxKlMgOpYBFwlahXCzeJ6NyFQhw=; b=mj+1muyG8xemV4QBLaw1tW65PNMvoFGz0HapOCWuNq004xHfqcsJbP+GuQXKLvqAc2 726kgspas78zbkBu5Mj57dsUeqY0GgqS3+lzUCK5mPNwVN9uyIWSiK0lFysjrzmm6xiG GZRiU7By1bzHi6i6Z2ixdqTdccr4ncVUF+apSm05Hq9g3F1Ai1TMWAF+lFjME1ZZenRJ H+DrKgRGJBp0gpfhyzecv2RAOg1MVU7lp9ZstmFm6zAfe1ghmMVRMBdXzWdezDOqzjkH 3YoYWLZrMzws6yxwFu4Mr5UalCerR1PwaG1ms5Syfkao4X7eXZMqmgF4FkXXG2BdyeDg Vt9Q== Received: by 10.58.64.103 with SMTP id n7mr7368833ves.35.1347873810248; Mon, 17 Sep 2012 02:23:30 -0700 (PDT) MIME-Version: 1.0 Received: by 10.58.132.176 with HTTP; Mon, 17 Sep 2012 02:23:09 -0700 (PDT) In-Reply-To: References: From: Jason Yang Date: Mon, 17 Sep 2012 17:23:09 +0800 Message-ID: Subject: Re: How to output according to the key in reducer? To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=047d7b67415aa9736604c9e25318 X-Virus-Checked: Checked by ClamAV on apache.org --047d7b67415aa9736604c9e25318 Content-Type: text/plain; charset=ISO-8859-1 All right, thanks~ 2012/9/17 feng lu > Hi > Maybe you can refer to > > http://hadoop.apache.org/docs/r1.0.3/api/org/apache/hadoop/mapred/lib/MultipleSequenceFileOutputFormat.html > > or > > > http://hadoop.apache.org/docs/r1.0.3/api/org/apache/hadoop/mapred/lib/MultipleTextOutputFormat.html > > example like this > > public static class GeneratorOutputFormat extends > MultipleSequenceFileOutputFormat { > // generate a filename based on the segnum stored for this text > protected String generateFileNameForKeyValue(IntWritable key, Text > value, > String name) { > return "ID_" + key.get() + ".dat"; > } > > } > > On Mon, Sep 17, 2012 at 3:16 PM, Jason Yang wrote: > >> Hi, all >> >> I was wondering how to write all the input with the same key to a single >> file in the reducer ? >> >> say, I got some intermediate outputs from mappers like that: >> key value >> -------------------- >> 1 annie >> 2 Jason >> 1 andy >> 2 Joey >> 1 andrew >> ... >> >> and I would like write all the intermediate outputs with key 1 to file >> "ID_1.dat", and all the intermediate outputs with key 2 to file "ID_2.data". >> >> how could I achieve that? >> >> -- >> YANG, Lin >> >> > > > -- > Don't Grow Old, Grow Up... :-) > -- YANG, Lin --047d7b67415aa9736604c9e25318 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable All right, thanks~

2012/9/17 feng lu <amuseme.lu@gmail.com>
Hi=A0
Maybe you can refer to=A0
http://hadoop.apache.org/docs/r1.0.3/api/= org/apache/hadoop/mapred/lib/MultipleSequenceFileOutputFormat.html

or=A0


example like this

=A0 pub= lic static class GeneratorOutputFormat extends
=A0 =A0 =A0 Multip= leSequenceFileOutputFormat<IntWritable,Text> {
=A0 =A0 // g= enerate a filename based on the segnum stored for this text
=A0 =A0 protected String generateFileNameForKeyValue(IntWritable key, = Text value,
=A0 =A0 =A0 =A0 String name) {
=A0 =A0 =A0 = return "ID_" + key.get() + ".dat";
=A0 =A0 }

=A0 }

On Mon, Sep 17, 2012 at 3:16= PM, Jason Yang <lin.yang.jason@gmail.com> wrote:
=
Hi, all

I was wondering how to write all the input with = the same key to a single file in the reducer ?

say= , I got some intermediate outputs from mappers like that:=A0
key = =A0 =A0 value
--------------------
1 =A0 annie
2 =A0 Jason
=
1 =A0 andy
2 =A0 Joey
1 =A0 andrew
...

and I would like write all the=A0intermediate outpu= ts with key 1 to file "ID_1.dat", and all the intermediate output= s with key 2 to file "ID_2.data".

how could I achieve that?

--
YANG, Lin




--
Don't Grow= Old, Grow Up... :-)



--
YANG, L= in

--047d7b67415aa9736604c9e25318--