Mailing-List: contact hadoop-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hadoop-dev@lucene.apache.org
Message-ID: <21419253.1196107543045.JavaMail.jira@brutus>
Date: Mon, 26 Nov 2007 12:05:43 -0800 (PST)
From: "Amar Kamat (JIRA)" <jira@apache.org>
To: hadoop-dev@lucene.apache.org
Subject: [jira] Updated: (HADOOP-1984) some reducer stuck at copy phase and
 progress extremely slowly
In-Reply-To: <4690776.1191331971788.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/HADOOP-1984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Amar Kamat updated HADOOP-1984:
-------------------------------

    Status: Patch Available  (was: Open)

Taking into consideration Arun's comment, hard coding the computations for base=2. The way it works now is as follows
1. the first notification is sent to the jobtracker after {{max-backoff}} time. The backoff values are ({{4, 8, 16, 32 ..... k}}) such that {{4 + 8 + 16 + 32 .... k ~ max-backoff}},  {{max-backoff}} can be set using {{mapred.reduce.copy.backoff}} [default is 300 sec].
2. subsequent notifications are also sent after {{max-backoff}} time but the backoff values now are ({{max-backoff/2, max-backoff/2}}) , i.e 2 retries
3. at the max the reducer waits for {{3 * max-backoff}} before the maps get re-executed. 
Comments ?


> some reducer stuck at copy phase and progress extremely slowly
> --------------------------------------------------------------
>
>                 Key: HADOOP-1984
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1984
>             Project: Hadoop
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.16.0
>            Reporter: Runping Qi
>            Assignee: Amar Kamat
>            Priority: Critical
>             Fix For: 0.16.0
>
>         Attachments: HADOOP-1984-simple.patch, HADOOP-1984-simple.patch, HADOOP-1984-simple.patch, HADOOP-1984.patch
>
>
> In many cases, some reducers got stuck at copy phase, progressing extremely slowly.
> The entire cluster seems doing nothing. This causes a very bad long tails of otherwise well tuned map/red jobs.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.