hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amar Kamat (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3200) Job history counters should be avaible on the UI.
Date Mon, 15 Sep 2008 14:39:44 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12631038#action_12631038
] 

Amar Kamat commented on HADOOP-3200:
------------------------------------

bq. I am trying to get map-tasks counters and reduce-tasks counters separately
I think comparing the final job counter should suffice. Job counter encompasses all the maps
and reduce counters. Is there any specific reason why you are matching them separately?
bq. May be we could just change parseHistoryFromFs to accept multiple listeners
I dont see any need for a separate listener, atleast in this jira. I think we should simply
recover the counter wherever the counter is expected (like _TASK_FINISH_) and add it to the
global job counter. The only change required will be 
{code}
Counters jobCounter = new Counters();
...
...
// On a task/tip event
Counters newCounter = Counters.fromEscapedCompactString(task.get(Keys.COUNTERS));
jobCounter.incrAllCounters(newCounter);
{code}
Introducing new code just for this is probably not worth it. I think we should reuse the available
listener. I agree that we should support multiple listeners but it makes sense when the listeners
are having separate, well defined functionalities. in this case one would be a subpart of
other, logically.

> Job history counters should be avaible on the UI.
> -------------------------------------------------
>
>                 Key: HADOOP-3200
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3200
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.16.0
>            Reporter: Amareshwari Sriramadasu
>            Assignee: Vinod K V
>         Attachments: HADOOP-3200-20080915.1.txt
>
>
> Job history is logging counters. But they are not visible on the UI. 
> Job history parser and UI should be modified to view counters.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message