hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Yang <lin.yang.ja...@gmail.com>
Subject Re: How to output according to the key in reducer?
Date Mon, 17 Sep 2012 09:23:09 GMT
All right, thanks~

2012/9/17 feng lu <amuseme.lu@gmail.com>

> Hi
> Maybe you can refer to
>
> http://hadoop.apache.org/docs/r1.0.3/api/org/apache/hadoop/mapred/lib/MultipleSequenceFileOutputFormat.html
>
> or
>
>
> http://hadoop.apache.org/docs/r1.0.3/api/org/apache/hadoop/mapred/lib/MultipleTextOutputFormat.html
>
> example like this
>
>   public static class GeneratorOutputFormat extends
>       MultipleSequenceFileOutputFormat<IntWritable,Text> {
>     // generate a filename based on the segnum stored for this text
>     protected String generateFileNameForKeyValue(IntWritable key, Text
> value,
>         String name) {
>       return "ID_" + key.get() + ".dat";
>     }
>
>   }
>
> On Mon, Sep 17, 2012 at 3:16 PM, Jason Yang <lin.yang.jason@gmail.com>wrote:
>
>> Hi, all
>>
>> I was wondering how to write all the input with the same key to a single
>> file in the reducer ?
>>
>> say, I got some intermediate outputs from mappers like that:
>> key     value
>> --------------------
>> 1   annie
>> 2   Jason
>> 1   andy
>> 2   Joey
>> 1   andrew
>> ...
>>
>> and I would like write all the intermediate outputs with key 1 to file
>> "ID_1.dat", and all the intermediate outputs with key 2 to file "ID_2.data".
>>
>> how could I achieve that?
>>
>> --
>> YANG, Lin
>>
>>
>
>
> --
> Don't Grow Old, Grow Up... :-)
>



-- 
YANG, Lin

Mime
View raw message