Return-Path: X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id CA8CDF3BF for ; Tue, 23 Apr 2013 05:36:37 +0000 (UTC) Received: (qmail 8364 invoked by uid 500); 23 Apr 2013 05:36:33 -0000 Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org Received: (qmail 8005 invoked by uid 500); 23 Apr 2013 05:36:32 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 7988 invoked by uid 99); 23 Apr 2013 05:36:31 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 23 Apr 2013 05:36:31 +0000 X-ASF-Spam-Status: No, hits=1.7 required=5.0 tests=FREEMAIL_ENVFROM_END_DIGIT,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of vikascjadhav87@gmail.com designates 209.85.216.169 as permitted sender) Received: from [209.85.216.169] (HELO mail-qc0-f169.google.com) (209.85.216.169) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 23 Apr 2013 05:36:24 +0000 Received: by mail-qc0-f169.google.com with SMTP id t2so123323qcq.28 for ; Mon, 22 Apr 2013 22:36:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:content-type; bh=LiCyjaTkzuu/exlEH1HkV6Em/L+z9uRFkWXghO7ziR8=; b=XeEhcmnA1du6/P2QmNDAxEF3Ktt7jpr7HOEZhIbnAEUXiWiaRIyyeQEvzkR8MZQzoY 81moU8eZrxHuDRzZMQClpOCj0+UGYxqSdl7ZhxcRtATPh+drSBOxqup8Ql9wAqBnzyi5 VczRSaOilIJ+ql3fxIFdAJebe3jhFvsmJP2hZcrQWcq2rXgKTO6qsfVCr8op3JW7TLyl Cvv1cMmRrf40lhiUAEuQrC6jheiheu4nxjiH+UyxLXuPjGimGv01xE7+p+OQ9MX9uQyQ zjbonZzRohQGA4RmkYsGbRpVRzT4wCk0fU8R5nNNoxV26iEQ6bx4iyJGNAnQ2JsCfMSx 2Txg== MIME-Version: 1.0 X-Received: by 10.224.37.196 with SMTP id y4mr17199264qad.60.1366695364106; Mon, 22 Apr 2013 22:36:04 -0700 (PDT) Received: by 10.49.16.170 with HTTP; Mon, 22 Apr 2013 22:36:04 -0700 (PDT) In-Reply-To: References: Date: Tue, 23 Apr 2013 11:06:04 +0530 Message-ID: Subject: Re: Writing intermediate key,value pairs to file and read it again From: Vikas Jadhav To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=001a11c29ff2b19e8704db008f92 X-Virus-Checked: Checked by ClamAV on apache.org --001a11c29ff2b19e8704db008f92 Content-Type: text/plain; charset=ISO-8859-1 ya i looked @ chain reducer but it will not help in my case Now i am able to write each key value pair received @ reducer(using IFile.Writer ) but when i try to read it using IFile.Reader it reads all data not single key value pair Example: *IFile.Reader reader = new* IFile.Reader(context.getConfiguration(), localFs, file, *null*, myReadCounter); DataInputBuffer keyRead = new DataInputBuffer(); DataInputBuffer valueRead = new DataInputBuffer(); reader.next(keyRead, valueRead); when i say keyRead.getData() it print all key value pair in that buffer i want single key value pair. On Sun, Apr 21, 2013 at 9:53 AM, Azuryy Yu wrote: > you would look at chain reducer java doc, which meet your requirement. > > --Send from my Sony mobile. > On Apr 20, 2013 11:43 PM, "Vikas Jadhav" wrote: > >> Hello, >> Can anyone help me in following issue >> Writing intermediate key,value pairs to file and read it again >> >> let us say i have to write each intermediate pair received @reducer to a >> file and again read that as key value pair again and use it for processing >> >> I found IFile.java file which has reader and writer but i am not able >> understand how to use it for example. I dont understand Counter value as >> last parameter "spilledRecordsCounter" >> >> >> Thanks. >> >> >> -- >> * >> * >> * >> >> Regards,* >> * Vikas * >> > -- * * * Regards,* * Vikas * --001a11c29ff2b19e8704db008f92 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
ya i looked @ chain reducer but it will not help in m= y case
=A0
Now i am able to write each key value pair = received @ reducer(using IFile.Writer )
but when i try to read it= using IFile.Reader it reads all data not single key value pair
=A0
Example:

=A0

IFile.Reader= =A0reader=A0 =3D =A0new IFile.Reader<KEYIN, VALUEIN>(conte= xt.getConfiguration(), localFs, file, null, myReadCounter);

DataInputBuffer keyRead =3D new DataInputBuffer();<= /font>

DataInputBuffer valueRead =3D new DataInp= utBuffer();

reader.next(keyRead, valueRead);

=A0

when i say keyRead.getData()

=A0it print all key value pair in that b= uffer i want single=A0 key value pair.

=A0

=A0



On Sun, Apr 21, 2013 at 9:53 AM, Azuryy Yu <azuryyyu@gma= il.com> wrote:

you would look at chain reduc= er java doc, which meet your requirement.

--Send from my Sony mobile.

On Apr 20, 2013 11:43 PM, "Vikas Jadhav&quo= t; <vikasc= jadhav87@gmail.com> wrote:
Hello,
Can anyone help me in followi= ng issue
Writing intermediate key,value pairs to file and read it again<= br>

let us say i have to write each intermediate pair receive= d @reducer to a file and again read that as key value pair again and use it= for processing

I found IFile.java file which has reader and writer but i am= not able understand how to use it for example. I dont understand Counter v= alue as last parameter "spilledRecordsCounter"


Thanks.


--
<= font color=3D"#ff0000">

=A0 Regards,=A0 =A0Vikas



--


=A0 Regar= ds,
=A0 =A0Vikas
--001a11c29ff2b19e8704db008f92--