hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Luke Lu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-10090) Jobtracker metrics not updated properly after execution of a mapreduce job
Date Fri, 15 Nov 2013 00:17:20 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-10090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13823152#comment-13823152
] 

Luke Lu commented on HADOOP-10090:
----------------------------------

[~cnauroth]: That's a good idea. I actually thought about doing that for HADOOP-8050, as MetricsSystemImpl
is already implementing MetricsSource in trunk.

OTOH, this seems to be no longer an issue in trunk due to YARN-1043, which disabled sparse
update completely. I was not aware of that until now. In retrospect, I probably should've
done that by default, as all the sinks except the first (and then sole) production sink that
the system was designed for cannot handle sparse updates anyway. I still cringe at wasting
resource snapshotting all the "failure" metrics that don't change much. Maybe I'll scratch
that itch one day. 

> Jobtracker metrics not updated properly after execution of a mapreduce job
> --------------------------------------------------------------------------
>
>                 Key: HADOOP-10090
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10090
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: metrics
>    Affects Versions: 1.2.1
>            Reporter: Ivan Mitic
>            Assignee: Ivan Mitic
>         Attachments: HADOOP-10090.branch-1.patch, OneBoxRepro.png
>
>
> After executing a wordcount mapreduce sample job, jobtracker metrics are not updated
properly. Often times the response from the jobtracker has higher number of job_completed
than job_submitted (for example 8 jobs completed and 7 jobs submitted). 
> Issue reported by Toma Paunovic.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message