Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-issues@hadoop.apache.org
Date: Fri, 15 Nov 2013 21:19:21 +0000 (UTC)
From: "Ivan Mitic (JIRA)" <jira@apache.org>
To: common-issues@hadoop.apache.org
Message-ID: <JIRA.12678588.1384181312613.83037.1384550361603@arcas>
In-Reply-To: <JIRA.12678588.1384181312613@arcas>
References: <JIRA.12678588.1384181312613@arcas>
Subject: [jira] [Commented] (HADOOP-10090) Jobtracker metrics not updated
 properly after execution of a mapreduce job
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-10090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13824105#comment-13824105 ] 

Ivan Mitic commented on HADOOP-10090:
-------------------------------------

Thanks Chris and Luke.

Luke, I tend to agree with the risk part.

I guess we have a couple of options on the table:
#1: We lock the source and use this to guarantee consistency of metrics reported thru sinks.
#2. We go with my first patch attached (I just have address the feedback). This will guarantee that JMX returns back consistent state. Sinks on the other hand can get back partial results (same as what we have today in branch-1).
#3. We go with my first patch + YARN-1043. This will address the problem from this Jira + consistent results for sinks. 

Given our conversation from above, my preference is to go with #2. If we believe that YARN-1043 makes sense for branch-1, I would prefer to do it as a separate Jira.

Thoughts?

It's definitely not easy to make the call... at least on my side... :)

PS. Shall we do a trunk patch too?
I believe we'll want to do a trunk patch too. Haven't tried to repro the problem there yet though. 

> Jobtracker metrics not updated properly after execution of a mapreduce job
> --------------------------------------------------------------------------
>
>                 Key: HADOOP-10090
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10090
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: metrics
>    Affects Versions: 1.2.1
>            Reporter: Ivan Mitic
>            Assignee: Ivan Mitic
>         Attachments: HADOOP-10090.branch-1.2.patch, HADOOP-10090.branch-1.patch, OneBoxRepro.png
>
>
> After executing a wordcount mapreduce sample job, jobtracker metrics are not updated properly. Often times the response from the jobtracker has higher number of job_completed than job_submitted (for example 8 jobs completed and 7 jobs submitted). 
> Issue reported by Toma Paunovic.


--
This message was sent by Atlassian JIRA
(v6.1#6144)