Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of jayunit100@gmail.com
 designates 209.85.214.172 as permitted sender)
References: 
 <CALiVDr_ZHiYHfB8Ta+vEmSK4QecT62mqgM+WHNkrsx-sG_GXDg@mail.gmail.com>
Mime-Version: 1.0 (1.0)
In-Reply-To: 
 <CALiVDr_ZHiYHfB8Ta+vEmSK4QecT62mqgM+WHNkrsx-sG_GXDg@mail.gmail.com>
Content-Type: multipart/alternative;
	boundary=Apple-Mail-9795B762-0049-4D2A-ABB0-019D79541150
Content-Transfer-Encoding: 7bit
Message-Id: <8849328B-D716-4CBF-AA93-17BF04E51D7B@gmail.com>
Cc: "user@hadoop.apache.org" <user@hadoop.apache.org>
From: Jay Vyas <jayunit100@gmail.com>
Subject: Re: Writing intermediate key,value pairs to file and read it again
Date: Sat, 20 Apr 2013 11:06:18 -0500
To: "user@hadoop.apache.org" <user@hadoop.apache.org>


--Apple-Mail-9795B762-0049-4D2A-ABB0-019D79541150
Content-Type: text/plain;
	charset=us-ascii
Content-Transfer-Encoding: quoted-printable

How many intermediate keys? If small enough, you can keep them in memory.  I=
f large, you can just wait for the job to finish and siphon them into your j=
ob as input with the MultipleInputs API.


On Apr 20, 2013, at 10:43 AM, Vikas Jadhav <vikascjadhav87@gmail.com> wrote:=


> Hello,
> Can anyone help me in following issue
> Writing intermediate key,value pairs to file and read it again
>=20
> let us say i have to write each intermediate pair received @reducer to a f=
ile and again read that as key value pair again and use it for processing
>=20
> I found IFile.java file which has reader and writer but i am not able unde=
rstand how to use it for example. I dont understand Counter value as last pa=
rameter "spilledRecordsCounter"=20
>=20
>=20
> Thanks.
>=20
>=20
> --=20
>=20
>=20
>   Regards,
>    Vikas

--Apple-Mail-9795B762-0049-4D2A-ABB0-019D79541150
Content-Type: text/html;
	charset=utf-8
Content-Transfer-Encoding: 7bit

<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body dir="auto"><div>How many intermediate keys? If small enough, you can keep them in memory. &nbsp;If large, you can just wait for the job to finish and siphon them into your job as input with the MultipleInputs API.<br><br><br></div><div><br>On Apr 20, 2013, at 10:43 AM, Vikas Jadhav &lt;<a href="mailto:vikascjadhav87@gmail.com">vikascjadhav87@gmail.com</a>&gt; wrote:<br><br></div><blockquote type="cite"><div><div dir="ltr">Hello,<br clear="all"><div>Can anyone help me in following issue<br>Writing intermediate key,value pairs to file and read it again<br></div><div><br>let us say i have to write each intermediate pair received @reducer to a file and again read that as key value pair again and use it for processing<br>
<br></div><div>I found IFile.java file which has reader and writer but i am not able understand how to use it for example. I dont understand Counter value as last parameter "spilledRecordsCounter" <br><br><br></div>
<div>Thanks.</div><div><br><br></div><div>-- <br><div dir="ltr"><div><b><font color="#ff0000"><br></font></b></div><b><font color="#ff0000"><div><b><font color="#ff0000"><br></font></b></div>&nbsp; Regards,</font></b><div><b><font color="#ff0000">&nbsp; &nbsp;Vikas </font></b></div>
</div>
</div></div>
</div></blockquote></body></html>
--Apple-Mail-9795B762-0049-4D2A-ABB0-019D79541150--