Mailing-List: contact common-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of ted.xu.ml@gmail.com
 designates 209.85.219.217 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=RTNufWaSZgVCF7qAr6qUfoFU8VYap2+/45v63iGwCRnDtGUlvqMmBY+IQLipHj3I9m
         3s9lwJTUa+Y0dB5sNtrbSpvXKqL80yoUf47WN+I70Be/Wngn2+aKAEKmEsDQHMQMC2m2
         d4DaTsRgPnZElOaosFpXjgg11LKMHUvG/k2Xg=
MIME-Version: 1.0
In-Reply-To: <24774844.647271259972118270.JavaMail.root@zimbra>
References: <24774844.647271259972118270.JavaMail.root@zimbra>
Date: Sat, 5 Dec 2009 09:51:56 +0800
Message-ID: <ce6ab8460912041751n49f08bf3y1a13cf72e2999aae@mail.gmail.com>
Subject: Re: Problem while writing a file from Reducer
From: Ted Xu <ted.xu.ml@gmail.com>
To: common-user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=0016e6da7ae31ff2f70479f1785b

--0016e6da7ae31ff2f70479f1785b
Content-Type: text/plain; charset=ISO-8859-1

Hello Parth,

Once I met a similar problem while using user-defined recordwriter in
reducer. That time I forgot to close the recordwriter in the close phase of
reducer.

2009/12/5 Parth J. Brahmbhatt <parth@ccs.neu.edu>

> Hello All,
>
> I am creating 2 files in the constructor of my reducer and I store the file
> Handles as member variables. I write some data in these files for each call
> to reduce method. For some reason the files are created but only the larger
> file has data(The larger file is 400mb).The smaller file is created but no
> data is written to it (The size should be around 4MB). I checked the hadoop
> configuration and the buffer-size parameter is 4096.
>
> I padded some 100MB data to the smaller file and than this file gets around
> 64MB data. I am clueless what is happening and if there is a solution to
> this. Please share if you have faced similar problem or if you know the
> solution.
>
> Thanks,
> Parth Brahmbhatt
>

Best Regards,

Ted Xu

--0016e6da7ae31ff2f70479f1785b--