Mailing-List: contact common-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of jason.hadoop@gmail.com
 designates 209.85.212.196 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=F889RWGQAqL1XyL7sYAJfGEjtOEFb//kAl+9vuzrHXu4YZvDB8OKh2IU1gHcnjUSPC
         y2CueiMFdfbUjz3MBDufeWNVkTeZkIXzOxqkbbNqzZb0u8wCvG4XbiQ9SV1zSdqLHiWf
         52e49kniYBuMukfJ1+qbM1c8OlVTFy3tq8HVI=
MIME-Version: 1.0
In-Reply-To: <c7d45fc70907081655h4dcd4284j2624ef14edc62d46@mail.gmail.com>
References: <445c748b0907081513r278b7157l250a8f2c0f28211f@mail.gmail.com>
	 <1E45DBB0-A0D7-40D4-B457-4D84274C8471@apache.org>
	 <c7d45fc70907081655h4dcd4284j2624ef14edc62d46@mail.gmail.com>
Date: Wed, 8 Jul 2009 22:54:24 -0700
Message-ID: <314098690907082254n69254779rf4804bfc012dba8a@mail.gmail.com>
Subject: Re: Merging many output files from reducer
From: jason hadoop <jason.hadoop@gmail.com>
To: common-user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=0016e64f5f4ce9e0cf046e3f7c73

--0016e64f5f4ce9e0cf046e3f7c73
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

In the example code from Pro Hadoop, is a sample map reduce job that uses
mapside join to merge the files into a single output.
It is part of the chapter 9 examples.

On Wed, Jul 8, 2009 at 4:55 PM, Ted Dunning <ted.dunning@gmail.com> wrote:

> On Wed, Jul 8, 2009 at 3:38 PM, Owen O'Malley <omalley@apache.org> wrote:
>
> >
> > On Jul 8, 2009, at 3:13 PM, Pankil Doshi wrote:
> >
> >  Can anyone guide me to merge my output files from reducer to single file
> >> in
> >> HDFS.
> >>
> >
> > The usual approach is to leave them as separate files.
>
>
> Also, the need to merge often arises from a need to import the data into an
> external database.  That doesn't sound like your need because you already
> know and have rejected dfs -cat.
>
> It may help to think of the containing directory as the actual file and the
> files inside that directory as no more interesting than the inodes and
> blocks that make up a normal unix file.
>


-- 
Pro Hadoop, a book to guide you from beginner to hadoop mastery,
http://www.amazon.com/dp/1430219424?tag=jewlerymall
www.prohadoopbook.com a community for Hadoop Professionals

--0016e64f5f4ce9e0cf046e3f7c73--