Mailing-List: contact general-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: general@hadoop.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; s=serpent; d=yahoo-inc.com; c=nofws; q=dns;
	h=received:from:to:date:subject:thread-topic:thread-index:
	message-id:in-reply-to:accept-language:content-language:
	x-ms-has-attach:x-ms-tnef-correlator:acceptlanguage:content-type:mime-version;
	b=1y5HrgROHPxKhdKv/SIMby5c+FL+Q36MBhPeK0t/uBT4oInBTFqqwNPjFhvHXAdM
From: Amareshwari Sri Ramadasu <amarsri@yahoo-inc.com>
To: "general@hadoop.apache.org" <general@hadoop.apache.org>
Date: Tue, 22 Dec 2009 10:01:52 +0530
Subject: Re: InputFormat related question...
Thread-Topic: InputFormat related question...
Thread-Index: AcqCZTE+Kk7yIDiHRs+vcGmey7IrsgAWn3E1
Message-ID: <C7564990.EA0B%amarsri@yahoo-inc.com>
In-Reply-To: <1eabbac30912210942w714b1ab7x3aa4b1d4207f667f@mail.gmail.com>
Accept-Language: en-US
Content-Language: en
acceptlanguage: en-US
Content-Type: multipart/alternative;
	boundary="_000_C7564990EA0Bamarsriyahooinccom_"
MIME-Version: 1.0

--_000_C7564990EA0Bamarsriyahooinccom_
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

Hi,

If you want map task to process two lines at a time, you need to write a Re=
cordReader which constructs two lines per record. LineRecordReader makes on=
e line as one record.
You can extend NLineInputFormat for generating splits and return your new R=
ecordReader for reading records from split.
Hope this helps you.

Thanks
Amareshwari

On 12/21/09 11:12 PM, "Something Something" <mailinglists19@gmail.com> wrot=
e:

In my application I have a file in this format:

The first line of the file contains the data to be processed, and *each* of
the remaining lines contain parameters that will be used to slice & dice th=
e
data in various ways.  In other words, each mapper needs two lines - the 1s=
t
line from this file that contains data and another line that contains
parameters.

I looked at NLineInputFormat which can be used for "parameter sweeps", but
it's not quite what I want.  I believe this format returns N no. of
consecutive lines to the mapper, correct?

What's the best way to handle this case?  Do I have to write a special
InputFormat class?  Please help.  Thanks.


--_000_C7564990EA0Bamarsriyahooinccom_--