hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron Kimball" <aa...@cloudera.com>
Subject Re: Map input records(on JobTracker website) increasing and decreasing
Date Mon, 05 Jan 2009 20:24:28 GMT
The actual number of input records is most likely steadily increasing. The
counters on the web site are inaccurate until the job is complete; their
values will fluctuate wildly. I'm not sure why this is.

- Aaron

On Mon, Jan 5, 2009 at 8:34 AM, Saptarshi Guha <saptarshi.guha@gmail.com>wrote:

> Hello,
> When I check the job tracker web page, and look at the Map Input
> records read,the map input records goes up to say 1.4MN and then drops
> to 410K and then goes up again.
> The same happens with input/output bytes and output records.
> Why is this? Is there something wrong with the mapper code? In my map
> function, i assume I have received one line of input.
> The oscillatory behavior does not occur for tiny datasets, but for 1GB
> of data (tiny for others) i see this happening.
> Thank s
> Saptarshi
> --
> Saptarshi Guha - saptarshi.guha@gmail.com

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message