Mailing-List: contact hadoop-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hadoop-dev@lucene.apache.org
Message-ID: <12075170.1172609225779.JavaMail.jira@brutus>
Date: Tue, 27 Feb 2007 12:47:05 -0800 (PST)
From: "Doug Cutting (JIRA)" <jira@apache.org>
To: hadoop-dev@lucene.apache.org
Subject: [jira] Commented: (HADOOP-1048) Sort500 failing since counters
 patch went in
In-Reply-To: <3804994.1172604185523.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-1048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12476375 ] 

Doug Cutting commented on HADOOP-1048:
--------------------------------------

Another possible optimization in JobInProgress#updateTaskStatus might be to keep, for each task, its previous counter values, then compute the difference between those and the new values, and update global counters incrementally rather than recomputing them from scratch each time.  This might not actually reduce the amount of computation much, but it might greatly decrease the amount of allocated objects, and object allocations are considerably more expensive than arithmetic.

> Sort500 failing since counters patch went in
> --------------------------------------------
>
>                 Key: HADOOP-1048
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1048
>             Project: Hadoop
>          Issue Type: Bug
>          Components: mapred
>            Reporter: David Bowen
>
> Nigel wrote:
>   We have a problem
>   Looks like counters patch is "breaking" things
>   Sort500 has been failing since the patch went in
>   Looking at the JT logs, there are 0 "Call queue overflow" message before the patch went in and 15000+ after the patch went in
>   Looks like the counters stuff is overwhelming the JT 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.