Return-Path: Delivered-To: apmail-lucene-hadoop-dev-archive@locus.apache.org Received: (qmail 29696 invoked from network); 5 Jul 2006 18:23:19 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (209.237.227.199) by minotaur.apache.org with SMTP; 5 Jul 2006 18:23:19 -0000 Received: (qmail 16157 invoked by uid 500); 5 Jul 2006 18:23:18 -0000 Delivered-To: apmail-lucene-hadoop-dev-archive@lucene.apache.org Received: (qmail 16121 invoked by uid 500); 5 Jul 2006 18:23:18 -0000 Mailing-List: contact hadoop-dev-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hadoop-dev@lucene.apache.org Delivered-To: mailing list hadoop-dev@lucene.apache.org Received: (qmail 16112 invoked by uid 99); 5 Jul 2006 18:23:18 -0000 Received: from asf.osuosl.org (HELO asf.osuosl.org) (140.211.166.49) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 05 Jul 2006 11:23:18 -0700 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests= X-Spam-Check-By: apache.org Received: from [209.237.227.198] (HELO brutus.apache.org) (209.237.227.198) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 05 Jul 2006 11:23:17 -0700 Received: from brutus (localhost [127.0.0.1]) by brutus.apache.org (Postfix) with ESMTP id AADAF4103EC for ; Wed, 5 Jul 2006 18:21:30 +0000 (GMT) Message-ID: <28456345.1152123690696.JavaMail.jira@brutus> Date: Wed, 5 Jul 2006 18:21:30 +0000 (GMT+00:00) From: "Mahadev konar (JIRA)" To: hadoop-dev@lucene.apache.org Subject: [jira] Updated: (HADOOP-343) In case of dead task tracker, the copy mapouts try copying all mapoutputs from this tasktracker In-Reply-To: <21351816.1152122969991.JavaMail.jira@brutus> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N [ http://issues.apache.org/jira/browse/HADOOP-343?page=all ] Mahadev konar updated HADOOP-343: --------------------------------- Attachment: bugfix.patch This patch updates its mapoutput locations in case of any failures on copying from the task tracker. So, in case a copy failed from a task tracker, the map outputs corresponding to this node will be marked stale. The ReduceTask will ask for the mapout locations again for these stale map outputs. This patch also fixes a bug wherein the tasks keep polling the job tracker for map otputs "in a loop" without sleeping/waiting. With this fix the tasks will poll evry MIN_POLL_INTERVAL before querying the job tracker. > In case of dead task tracker, the copy mapouts try copying all mapoutputs from this tasktracker > ----------------------------------------------------------------------------------------------- > > Key: HADOOP-343 > URL: http://issues.apache.org/jira/browse/HADOOP-343 > Project: Hadoop > Type: Bug > Components: mapred > Versions: 0.5.0 > Reporter: Mahadev konar > Assignee: Mahadev konar > Attachments: bugfix.patch > > In case of a dead task tracker, the reduces which do not have the updated map out locations try copygin files from this node and since there are failures on copying, this leads to backoff and slowing down of the copy pahse. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira