Mailing-List: contact hadoop-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hadoop-dev@lucene.apache.org
Message-ID: <24139375.1198729963776.JavaMail.jira@brutus>
Date: Wed, 26 Dec 2007 20:32:43 -0800 (PST)
From: "Amar Kamat (JIRA)" <jira@apache.org>
To: hadoop-dev@lucene.apache.org
Subject: [jira] Commented: (HADOOP-2284) BasicTypeSorterBase.compare calls
 progress on each compare
In-Reply-To: <4407269.1196114683342.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12554515 ] 

Amar Kamat commented on HADOOP-2284:
------------------------------------

Yeah. It should be some fraction of {{mapred.task.timeout}}. I was thinking like *2%*. So that we make sure that sufficient attempts are made to declare the progress (in this case 50 attempts) and 50 progress indications while sorting should not be a problem. no?

> BasicTypeSorterBase.compare calls progress on each compare
> ----------------------------------------------------------
>
>                 Key: HADOOP-2284
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2284
>             Project: Hadoop
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Owen O'Malley
>            Assignee: Devaraj Das
>             Fix For: 0.16.0
>
>
> The inner loop of the sort is calling progress on each compare. I think it would make more sense to call progress in the sort rather than the compare or at most every 10000 compares. In the performance numbers, the call to progress as part of the sort are consuming 12% of the total cpu time when running word count under the local runner.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.