Mailing-List: contact mapreduce-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of conglin02@gmail.com designates
 209.85.210.176 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type:content-transfer-encoding;
        b=Hm1apmHJLZEpFTWOodOApg8c+VOpWBZNMHXcw01VKD9JtoY6e0RuSdAj5ysrhFnHn6
         J9PfswUIY+8AGbPSmvRNQgwXQr/UgiUiNluHtueI4J+I7i0vRXqhaRA2pWuOcvSv1Tfu
         Ql0KQfGK54y3UCUGL1Nd/1bDW6+ORuZX5KYxc=
MIME-Version: 1.0
In-Reply-To: <BANLkTikcYR9hdCDKharp_+pc_BnUhfHR4w@mail.gmail.com>
References: <BANLkTikNF-QxE0jtQd17AXwEzOABgkZi5A@mail.gmail.com>
	<BANLkTikcYR9hdCDKharp_+pc_BnUhfHR4w@mail.gmail.com>
Date: Thu, 12 May 2011 19:15:39 +0800
Message-ID: <BANLkTimgf7+0Uts-7Tk2yODO_xcJr6aBJA@mail.gmail.com>
Subject: Re: How to merge several SequenceFile into one?
From: =?GB2312?B?tNTB1g==?= <conglin02@gmail.com>
To: mapreduce-user@hadoop.apache.org
Content-Type: text/plain; charset=GB2312
Content-Transfer-Encoding: quoted-printable

Dear Jason,

If the order of the keys in sequence file is not important to me, in
other words, the sort process is not necessary, how can I stop the
distributed sort to save the consumption of resource?

Thanks for your suggestion.

Best Wishes,

-Lin

2011/5/12 jason <urgisb@gmail.com>:
> M/R job with a single reducer would do the job. This way you can
> utilize distributed sort and merge/combine/dedupe key/values as you
> wish.
>
> On 5/11/11, =B4=D4=C1=D6 <conglin02@gmail.com> wrote:
>> Hi all,
>>
>> There is lots of SequenceFile in HDFS, how can I merge them into one
>> SequenceFile?
>>
>> Thanks for you suggestion.
>>
>> -Lin
>>
>