Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of rahul.rec.dgp@gmail.com
 designates 209.85.220.174 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAEFMBxR+ajQ8YJ5jqH+KSQPq_p9Qe00_mbkPHNnOEFs844v0yQ@mail.gmail.com>
References: 
 <CAEFMBxR+ajQ8YJ5jqH+KSQPq_p9Qe00_mbkPHNnOEFs844v0yQ@mail.gmail.com>
From: Rahul Bhattacharjee <rahul.rec.dgp@gmail.com>
Date: Fri, 13 Sep 2013 22:15:59 +0530
Message-ID: 
 <CAO7hTbN8oBLzC7=dTNh0G94sqJFtyFKpQB06ue+zjpCuyN91dg@mail.gmail.com>
Subject: Re: Map JVMs not being terminated after MAP jobs complettion.
To: "user@hadoop.apache.org" <user@hadoop.apache.org>
Content-Type: multipart/alternative; boundary=bcaec5016625027cd404e646982c

--bcaec5016625027cd404e646982c
Content-Type: text/plain; charset=UTF-8

After the map task finishes , the VM terminates and all resources are
reclaimable.

Reduces might start pulling data from completed map tasks much before all
the maps are finished. However there is a property using which you can
delay the reduce step till all the maps are completed.

Rahul


On Fri, Sep 13, 2013 at 12:42 PM, Manu Reddy <manubharghav@gmail.com> wrote:

> Hi,
>
> I am running a memory intensive task on my cluster. The reducer class
> should load more than 2gb into memory. I was running into memory issues as
> the Map JVMS were not terminated after the completion of map tasks and
> still were holding onto memory.
>
> Is this normal? What can I do to ensure that my reduce function gets
> called only after all the map jvms are down?
>
>
> --
> Manu Reddy,
> mobile no: 091-8143405225,
> Hyderabad.
>

--bcaec5016625027cd404e646982c
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div class=3D"gmail_default" style=3D"font-family:georgia,=
serif;color:rgb(0,0,0)">After the map task finishes , the VM terminates and=
 all resources are reclaimable.<br><br></div><div class=3D"gmail_default" s=
tyle=3D"font-family:georgia,serif;color:rgb(0,0,0)">

Reduces might start pulling data from completed map tasks much before all t=
he maps are finished. However there is a property using which you can delay=
 the reduce step till all the maps are completed.<br><br></div><div class=
=3D"gmail_default" style=3D"font-family:georgia,serif;color:rgb(0,0,0)">

Rahul<br></div></div><div class=3D"gmail_extra"><br><br><div class=3D"gmail=
_quote">On Fri, Sep 13, 2013 at 12:42 PM, Manu Reddy <span dir=3D"ltr">&lt;=
<a href=3D"mailto:manubharghav@gmail.com" target=3D"_blank">manubharghav@gm=
ail.com</a>&gt;</span> wrote:<br>

<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex"><div dir=3D"ltr"><div><div>Hi,<br><br></div>=
I am running a memory intensive task on my cluster. The reducer class shoul=
d load more than 2gb into memory. I was running into memory issues as the M=
ap JVMS were not terminated after the completion of map tasks and still wer=
e holding onto memory. <br>


<br></div>Is this normal? What can I do to ensure that my reduce function g=
ets called only after all the map jvms are down?<span class=3D"HOEnZb"><fon=
t color=3D"#888888"><br><br clear=3D"all"><div><div><div><br>-- <br>Manu Re=
ddy,<div>

mobile no: 091-8143405225,</div><div>
Hyderabad.</div>
</div></div></div></font></span></div>
</blockquote></div><br></div>

--bcaec5016625027cd404e646982c--