Return-Path: 
 <user-return-1027-apmail-hadoop-mapreduce-user-archive=hadoop.apache.org@hadoop.apache.org>
X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org
Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id 38EF2DAE5
	for <apmail-hadoop-mapreduce-user-archive@minotaur.apache.org>;
 Mon,  3 Sep 2012 14:19:42 +0000 (UTC)
Received: (qmail 61294 invoked by uid 500); 3 Sep 2012 14:19:37 -0000
Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org
Received: (qmail 61201 invoked by uid 500); 3 Sep 2012 14:19:37 -0000
Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:user-help@hadoop.apache.org>
List-Unsubscribe: <mailto:user-unsubscribe@hadoop.apache.org>
List-Post: <mailto:user@hadoop.apache.org>
List-Id: <user.hadoop.apache.org>
Reply-To: user@hadoop.apache.org
Delivered-To: mailing list user@hadoop.apache.org
Received: (qmail 61190 invoked by uid 99); 3 Sep 2012 14:19:37 -0000
Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136)
    by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 03 Sep 2012 14:19:37 +0000
X-ASF-Spam-Status: No, hits=1.5 required=5.0
	tests=FSL_RCVD_USER,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS
X-Spam-Check-By: apache.org
Received-SPF: pass (athena.apache.org: domain of abhay.ratnaparkhi@gmail.com
 designates 74.125.83.48 as permitted sender)
Received: from [74.125.83.48] (HELO mail-ee0-f48.google.com) (74.125.83.48)
    by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 03 Sep 2012 14:19:31 +0000
Received: by eekd41 with SMTP id d41so2211751eek.35
        for <user@hadoop.apache.org>; Mon, 03 Sep 2012 07:19:09 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20120113;
        h=mime-version:date:message-id:subject:from:to:content-type;
        bh=W7sg3uaYifGAwMABMWVevQJr4ghRPwemvhxZ87N31xA=;
        b=QxhlnqDeZuSIJewblwRGyh2zG6tyr4RbFxVg293y8HDW5AY+9gYqQtVi2VLYwcIg64
         mL5FBeq4DoMjTXR+h/u4i10Zmo818fHo3GJ2Nwyu47uEvoGwikbH+fDitNhZqb8k8AOT
         gLoyFCg4r4iJG0GYXmUAuoq03lp2kO7y0ufrD4H9VhcSuTJ+j/81V4Eg3XFimXOPDbo/
         Bfq53lzmEV9I/1t/esUUdi9+XIwiVK0CAQM35AbulroRt7lqAKiKDWJk8vHfBItcHCiK
         IFtnkRVC5W8xZnNDK//yUefgUiTXKktA5uyzQi6ZtxZl/5O+btUavSrkNRDTH//19JcZ
         EAVA==
MIME-Version: 1.0
Received: by 10.14.179.136 with SMTP id h8mr21624072eem.6.1346681949939; Mon,
 03 Sep 2012 07:19:09 -0700 (PDT)
Received: by 10.14.223.136 with HTTP; Mon, 3 Sep 2012 07:19:09 -0700 (PDT)
Date: Mon, 3 Sep 2012 19:49:09 +0530
Message-ID: 
 <CAM2hXiUAJfPiR6423kKioSo1=xm0iBwpEU+6QvXC_qPxcXOiXA@mail.gmail.com>
Subject: knowing the nodes on which reduce tasks will run
From: Abhay Ratnaparkhi <abhay.ratnaparkhi@gmail.com>
To: user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=047d7b603e9240677a04c8ccd36c
X-Virus-Checked: Checked by ClamAV on apache.org

--047d7b603e9240677a04c8ccd36c
Content-Type: text/plain; charset=ISO-8859-1

Hello,

How can one get to know the nodes on which reduce tasks will run?

One of my job is running and it's completing all the map tasks.
My map tasks write lots of intermediate data. The intermediate directory is
getting full on all the nodes.
If the reduce task take any node from cluster then It'll try to copy the
data to same disk and it'll eventually fail due to Disk space related
exceptions.

I have added few more tasktracker nodes in the cluster and now want to run
reducer on new nodes only.
Is it possible to choose a node on which the reducer will run? What's the
algorithm hadoop uses to get a new node to run reducer?

Thanks in advance.

Bye
Abhay

--047d7b603e9240677a04c8ccd36c
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Hello,<div><br></div><div>How can one get to know the nodes on which reduce=
 tasks will run?</div><div><br></div><div>One of my job is running and it&#=
39;s completing all the map tasks.</div><div>My map tasks write lots of int=
ermediate data. The intermediate directory is getting full on all the nodes=
.=A0</div>
<div>If the reduce task take any node from cluster then It&#39;ll try to co=
py the data to same disk and it&#39;ll eventually fail due to Disk space re=
lated exceptions.</div><div><br></div><div>I have added few more tasktracke=
r nodes in the cluster and now want to run reducer on new nodes only.</div>
<div>Is it possible to choose a node on which the reducer will run? What&#3=
9;s the algorithm hadoop uses to get a new node to run reducer?</div><div><=
br></div><div>Thanks in advance.</div><div><br></div><div>Bye</div><div>
Abhay</div>

--047d7b603e9240677a04c8ccd36c--