hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Keren Ouaknine <ker...@gmail.com>
Subject Re: Measuring time in a Hadoop MapReduce job.
Date Mon, 04 Jul 2011 09:12:35 GMT
Hello Amit,

There are several ways, the one I am currently using for measurements is
based on the job history file.
This file has a line for each event, thus the last map event (indicating on
the completion of the map task) would be the last record in time with both
string "MAP" and "TASK_FINISHED". Similarly for the last reduce event: look
for the last record in time with both string "REDUCE" and "TASK_FINISHED".

If you find a better way, please update me!


On Mon, Jul 4, 2011 at 12:04 PM, Amit Sangroya <sangroyaamit@gmail.com>wrote:

> Hi,
> I am trying to monitor the time to complete a map phase and reduce
> phase in hadoop. Is there any way to measure the time taken to
> complete map and reduce phase in a cluster.
> Thanks,
> Amit

Keren Ouaknine
Cell: +972 54 2565404
Web: www.kereno.com

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message