hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Armstrong <john.armstr...@ccri.com>
Subject Re: Question about how input data is presented to the map function
Date Fri, 16 Sep 2011 13:34:57 GMT
On Fri, 16 Sep 2011 08:26:35 -0500, harry lippy <harrylippy@gmail.com>
wrote:
>  The keys are file offsets into the input file.  My question:  how did
the
> 'are presented to the map function as key-value pairs' happen?  I've run
> the
> example on the input file using the java Mapper, Reducer, and the code
that
> runs the job - none of which seems, to my novice eye, to massage the
input
> from the file to the map function in the (file offset, line of data from
> file) key-value format - and the results are correct.

There are actually MANY classes in the framework floating around, most of
which you Don't Need to Know About on a day-to-day basis.  One of them is
called an InputFormat, which handles getting input and parsing it into
records.  These classes can all be replaced with appropriate
configurations, but the defaults are usually pretty good for most purposes.

Mime
View raw message