hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Efe Gencer <efegen...@gmail.com>
Subject Which Subphases Do Times on JobHistory Web UI Cover
Date Tue, 24 Sep 2013 21:53:17 GMT
Hi All,

In JobHistory Web UI under Job > "Map Tasks" I see something as follows:
...
Started: <start time>
Finished: <finish time>
Elapsed: 12 mins, 5sec
Diagnostics:
*Average Map Time*: 1 mins, 40 sec
Average Reduce Time: 12 sec
Average Shuffle Time: 10 mins, 8 sec
Average Merge Time: 1 sec
...

1) I wonder which sub-map phases "Map Time" contains (e.g. map function,
sort, spill, merge, read and transfer corresponding filesplit from HDFS)
2) Does Reduce time covers Shuffle Time? What else does it cover? (write to
hdfs, etc)
3) Is Average Merge time related with map or reduce? (since they both have
merge phases)

Best,
Efe

Mime
View raw message