Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id C81DFDBE8 for ; Wed, 29 Aug 2012 13:41:32 +0000 (UTC) Received: (qmail 85740 invoked by uid 500); 29 Aug 2012 13:41:28 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 85634 invoked by uid 500); 29 Aug 2012 13:41:28 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 85627 invoked by uid 99); 29 Aug 2012 13:41:28 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 29 Aug 2012 13:41:28 +0000 X-ASF-Spam-Status: No, hits=-2.3 required=5.0 tests=RCVD_IN_DNSWL_MED,SPF_HELO_PASS,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of thealy@bnl.gov designates 130.199.3.132 as permitted sender) Received: from [130.199.3.132] (HELO smtpgw.bnl.gov) (130.199.3.132) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 29 Aug 2012 13:41:20 +0000 X-BNL-policy-q: X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AoYQAJIbPlCCx/7+/2dsb2JhbABFiFG1RgEFeBEsFg8JAwIBAgFFEwgBAYgJu1SORYMcA6Vrgn8 X-IronPort-AV: E=Sophos;i="4.80,334,1344225600"; d="scan'208";a="4962427" Received: from t5.itd.bnl.gov (HELO [130.199.254.254]) ([130.199.254.254]) by smtpgw.sec.bnl.local with ESMTP/TLS/DHE-RSA-CAMELLIA256-SHA; 29 Aug 2012 09:40:59 -0400 Message-ID: <503E1BEB.7060003@bnl.gov> Date: Wed, 29 Aug 2012 09:40:59 -0400 From: Terry Healy User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:14.0) Gecko/20120714 Thunderbird/14.0 MIME-Version: 1.0 To: user@hadoop.apache.org Subject: Delays in worker node jobs References: <53903ED9-146F-40FF-BD3B-183E1A95A3AB@yahoo.com> In-Reply-To: <53903ED9-146F-40FF-BD3B-183E1A95A3AB@yahoo.com> X-Enigmail-Version: 1.5a1pre Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org Running 1.0.2, in this case on Linux. I was watching the processes / loads on one TaskTracker instance and noticed that it completed it's first 8 map tasks and reported 8 free slots (the max for this system). It then waited doing nothing for more than 30 seconds before the next "batch" of work came in and started running. Likewise it also has relatively long periods with all 8 cores running at or near idle. There are no jobs failing or obvious errors in the TaskTracker log. What could be causing this? Should I increase the number of map jobs to greater than number of cores to try and keep it busier? -Terry