Return-Path: Delivered-To: apmail-hadoop-core-dev-archive@www.apache.org Received: (qmail 45987 invoked from network); 3 Feb 2009 03:24:29 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 3 Feb 2009 03:24:29 -0000 Received: (qmail 33441 invoked by uid 500); 3 Feb 2009 03:24:22 -0000 Delivered-To: apmail-hadoop-core-dev-archive@hadoop.apache.org Received: (qmail 33410 invoked by uid 500); 3 Feb 2009 03:24:22 -0000 Mailing-List: contact core-dev-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: core-dev@hadoop.apache.org Delivered-To: mailing list core-dev@hadoop.apache.org Received: (qmail 33399 invoked by uid 99); 3 Feb 2009 03:24:22 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Feb 2009 19:24:22 -0800 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.140] (HELO brutus.apache.org) (140.211.11.140) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 03 Feb 2009 03:24:20 +0000 Received: from brutus (localhost [127.0.0.1]) by brutus.apache.org (Postfix) with ESMTP id 817B6234C48D for ; Mon, 2 Feb 2009 19:23:59 -0800 (PST) Message-ID: <321025168.1233631439512.JavaMail.jira@brutus> Date: Mon, 2 Feb 2009 19:23:59 -0800 (PST) From: "Nathan Marz (JIRA)" To: core-dev@hadoop.apache.org Subject: [jira] Created: (HADOOP-5160) Hadoop reduce scheduler sometimes leaves machines idle MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org Hadoop reduce scheduler sometimes leaves machines idle ------------------------------------------------------ Key: HADOOP-5160 URL: https://issues.apache.org/jira/browse/HADOOP-5160 Project: Hadoop Core Issue Type: Bug Components: mapred Reporter: Nathan Marz I have a MapReduce application with number of reducers equal to the number of machines in the cluster (and with speculative execution turned off). However, Hadoop schedules multiple reduces to run on single machines and leaves other machines idle. This causes contention and seriously slows down the job. Hadoop should employ the simple heuristic of utilizing as many machines as possible when scheduling reduces. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.