hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Devaraj Das (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-1101) Add more statistics in the web-ui to do with tasks
Date Thu, 15 Mar 2007 05:23:09 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-1101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Devaraj Das updated HADOOP-1101:
--------------------------------

    Attachment: 1101.patch

This patch does the following (in analysejobhistory.jsp):
1) Splits the Reduce phase into two - shuffle and the actual "reduce". It displays the data
for each seperately.
2) Displays the average time, and the transition time for the last task for each of the phases
(map, shuffle, reduce).

> Add more statistics in the web-ui to do with tasks
> --------------------------------------------------
>
>                 Key: HADOOP-1101
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1101
>             Project: Hadoop
>          Issue Type: Improvement
>            Reporter: Devaraj Das
>         Assigned To: Devaraj Das
>         Attachments: 1101.patch
>
>
> Currently, the web-ui does not display all the statistics needed to analyze performance
of a job in the MR framework. Things like the average time the maps took to finish is missing.
Similarly, the min/max/average for shuffle is missing. It would also be nice to add the statistics
for major transitions like the time when the last map finished, the time when the last shuffle/copy
finished, etc. The rate of reading map input and writing reduce output will be helpful too.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message