hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "tim robertson" <timrobertson...@gmail.com>
Subject Re: Newbie: multiple output files
Date Sun, 23 Nov 2008 13:25:13 GMT
Hi Jeremy,

Thank you very much!  Exactly what I was looking for

Cheers,

Tim


On Sun, Nov 23, 2008 at 2:21 PM, Jeremy Chow <coderplay@gmail.com> wrote:
> Hi Tim,
>
> You can write a class inherit from org.apache.hadoop.mapred.lib.
> MultipleOutputFormat. Override method generateFileNameForKeyValue() like
> this
>
>
>   1.     @Override
>   2.     protected
>    String generateFileNameForKeyValue(K key, V value, String name) {
>   3.         return name + "_" + value.toString();
>   4.     }
>
>
> you can also check out http://coderplay.javaeye.com/blog/191188 for example.
>
> On Sun, Nov 23, 2008 at 9:12 PM, tim robertson <timrobertson100@gmail.com>wrote:
>
>> Hi,
>>
>> Can someone please point me at the best way to create multiple output
>> files based on the Key outputted from the Map?  So I end up with no
>> reduction, but a file per Key outputted in the Mapping phase, ideally
>> with the Key as the file name.
>>
>> Many thanks,
>>
>> Tim
>>
>
> Re
>
> --
> My research interests are distributed systems, parallel computing and
> bytecode based virtual machine.
>
> http://coderplay.javaeye.com
>

Mime
View raw message