hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod Kumar Vavilapalli (Updated) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (MAPREDUCE-3001) Map Reduce JobHistory and AppMaster UI should have ability to display task specific counters.
Date Wed, 28 Sep 2011 14:00:52 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-3001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Vinod Kumar Vavilapalli updated MAPREDUCE-3001:

    Status: Open  (was: Patch Available)

Really cool patch, great job!

Particularly I just loved singlejobcounters page. It helps us to compare across all tasks
things like memory usage, input records etc.

Minor improvement, not caused by your patch: The Counter Group section on the counters-page
isn't going well with the rest of the page looks-wise. In the minimum we should fix the wrapping,
right now names of some counters like org.​apache.​hadoop.​mapreduce.​lib.​output.​FileOutputFormatCounter
are not getting displayed completely.

One bug I ran into with failed job with failed tasks:
Caused by: java.lang.NullPointerException
        at org.apache.hadoop.mapreduce.v2.app.webapp.CountersBlock.render(CountersBlock.java:85)
        at org.apache.hadoop.yarn.webapp.view.HtmlBlock.render(HtmlBlock.java:64)
        at org.apache.hadoop.yarn.webapp.view.HtmlBlock.renderPartial(HtmlBlock.java:74)
        at org.apache.hadoop.yarn.webapp.View.render(View.java:210)
        at org.apache.hadoop.yarn.webapp.view.HtmlPage$Page.subView(HtmlPage.java:47)
        at org.apache.hadoop.yarn.webapp.hamlet.HamletImpl$EImp._v(HamletImpl.java:117)
        at org.apache.hadoop.yarn.webapp.hamlet.Hamlet$TD._(Hamlet.java:843)
        at org.apache.hadoop.yarn.webapp.view.TwoColumnLayout.render(TwoColumnLayout.java:54)
        at org.apache.hadoop.yarn.webapp.view.HtmlPage.render(HtmlPage.java:80)
        at org.apache.hadoop.yarn.webapp.Controller.render(Controller.java:210)
        at org.apache.hadoop.mapreduce.v2.app.webapp.AppController.taskCounters(AppController.java:123)
        at org.apache.hadoop.mapreduce.v2.hs.webapp.HsController.taskCounters(HsController.java:120)
        ... 32 more

You will need to check for absent counters for failed attempts, tasks.

Thanks for doing this!
> Map Reduce JobHistory and AppMaster UI should have ability to display task specific counters.
> ---------------------------------------------------------------------------------------------
>                 Key: MAPREDUCE-3001
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3001
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: jobhistoryserver, mrv2
>    Affects Versions: 0.23.0, 0.24.0
>            Reporter: Robert Joseph Evans
>            Assignee: Robert Joseph Evans
>            Priority: Blocker
>             Fix For: 0.23.0, 0.24.0
>         Attachments: MR-3001.txt
> Map Reduce JobHistory and AppMaster UI should have ability to display task specific counters.
 I think the best way to do this is to include in the Nav Block a task specific section with
task links when a task is selected.  Counters is already set up to deal with a task passed

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message