hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 丛林 <congli...@gmail.com>
Subject Re: How to merge several SequenceFile into one?
Date Thu, 12 May 2011 11:15:39 GMT
Dear Jason,

If the order of the keys in sequence file is not important to me, in
other words, the sort process is not necessary, how can I stop the
distributed sort to save the consumption of resource?

Thanks for your suggestion.

Best Wishes,

-Lin

2011/5/12 jason <urgisb@gmail.com>:
> M/R job with a single reducer would do the job. This way you can
> utilize distributed sort and merge/combine/dedupe key/values as you
> wish.
>
> On 5/11/11, 丛林 <conglin02@gmail.com> wrote:
>> Hi all,
>>
>> There is lots of SequenceFile in HDFS, how can I merge them into one
>> SequenceFile?
>>
>> Thanks for you suggestion.
>>
>> -Lin
>>
>

Mime
View raw message