Return-Path: X-Original-To: apmail-hadoop-common-user-archive@www.apache.org Delivered-To: apmail-hadoop-common-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 3F06DDBE6 for ; Thu, 23 Aug 2012 10:45:54 +0000 (UTC) Received: (qmail 71291 invoked by uid 500); 23 Aug 2012 10:45:49 -0000 Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org Received: (qmail 71110 invoked by uid 500); 23 Aug 2012 10:45:49 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 71081 invoked by uid 99); 23 Aug 2012 10:45:48 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 23 Aug 2012 10:45:48 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=5.0 tests=RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of Jan.Lukavsky@firma.seznam.cz designates 77.75.74.246 as permitted sender) Received: from [77.75.74.246] (HELO posta.szn.cz) (77.75.74.246) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 23 Aug 2012 10:45:43 +0000 Received: from [10.0.2.22] (10.0.2.22) by posta.szn.cz (10.0.3.149) with Microsoft SMTP Server id 14.2.298.4; Thu, 23 Aug 2012 12:45:20 +0200 Message-ID: <503609C1.80205@firma.seznam.cz> Date: Thu, 23 Aug 2012 12:45:21 +0200 From: =?ISO-8859-1?Q?Jan_Lukavsk=FD?= User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:14.0) Gecko/20120714 Thunderbird/14.0 MIME-Version: 1.0 To: Subject: Re: Running map tasks after all reduces have finished References: <5035F6ED.4000104@firma.seznam.cz> In-Reply-To: Content-Type: text/plain; charset="ISO-8859-1"; format=flowed Content-Transfer-Encoding: 8bit X-Originating-IP: [10.0.2.22] X-Virus-Checked: Checked by ClamAV on apache.org Hi, sorry I forgot to mention. We are using cdh3u3. Jan On 23.8.2012 12:08, Harsh J wrote: > Hey Jan, > > What version/distribution of Hadoop are you noticing this on? > > On Thu, Aug 23, 2012 at 2:55 PM, Jan Lukavsk� > wrote: >> Hi all, >> >> we are seeing strange behaviour of JobTracker in the following scenario: >> - job finishes map phase and starts reduce >> - after the shuffle phase of all reducers we loose a tasktracker, that >> doesn't run any reducer - so all remaining reducers are still running in the >> reduce phase >> - map tasks that were running on the lost tasktracker are rescheduled >> - reduces may finish earlier than the rescheduled map tasks and so the job >> is blocked waiting for the maps to finish, although their output is simple >> discarded >> >> Is this behaviour a bug or feature? :) I haven't found any JIRA that would >> describe it, if there exists one can anyone point me out? >> >> Thanks, >> Jan >> > > -- Jan Lukavsk� program�tor Seznam.cz, a.s. Radlick� 608/2 15000, Praha 5 jan.lukavsky@firma.seznam.cz http://www.seznam.cz