hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Scott Chen (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-2125) Put map-reduce framework counters to JobTrackerMetricsInst
Date Wed, 03 Nov 2010 19:04:26 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-2125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12927959#action_12927959

Scott Chen commented on MAPREDUCE-2125:

Hey Luke, Thanks for the reply.
bq. I also meant the metrics names regenerated on every doUpdate with regex replace and lowercase.

I think they are OK because it happens only once every 5 second on the entire JT. There is
no need to optimize them.

The problem is that many of the intermediate jobs don't need the additional getJobCounter
call. If you put it in the default instrumentation class, it becomes mandatory additional
expensive call for every job. I'd be more comfortable if you provide a way to turn it off,
preferably by default.
I see. I will think of any elegant way to allow turning it off. Let me know if you have any
good suggestions :)

> Put map-reduce framework counters to JobTrackerMetricsInst
> ----------------------------------------------------------
>                 Key: MAPREDUCE-2125
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2125
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: jobtracker
>    Affects Versions: 0.22.0
>            Reporter: Scott Chen
>            Assignee: Scott Chen
>             Fix For: 0.22.0
>         Attachments: MAPREDUCE-2125-v2.txt, MAPREDUCE-2125.txt
> We have lots of useful information in the framework counters including #spills, filesystem
read and write.
> It will be nice to put them all in the jobtracker metrics to get a global view of all
these numbers.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message