hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Rosenstrauch <dar...@darose.net>
Subject Re: Merge Reducers Outputs
Date Wed, 27 Jul 2011 14:55:58 GMT
On 07/26/2011 06:52 PM, Mohamed Riadh Trad wrote:
> Dear All,
>
> Is it possible to set up a task with multiple reducers and merge reducers outputs into
one single file?
>
> Bests,
>
> Trad Mohamed Riadh, M.Sc, Ing.

Not within the map-reduce job, but you can merge it after the job is 
done.  At my previous company we used FileUtil.copyMerge() to do this, 
and it worked quite well.

See:

http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/fs/FileUtil.html#copyMerge%28org.apache.hadoop.fs.FileSystem,%20org.apache.hadoop.fs.Path,%20org.apache.hadoop.fs.FileSystem,%20org.apache.hadoop.fs.Path,%20boolean,%20org.apache.hadoop.conf.Configuration,%20java.lang.String%29

DR

Mime
View raw message