hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rahul Bhattacharjee <rahul.rec....@gmail.com>
Subject Re: Map JVMs not being terminated after MAP jobs complettion.
Date Fri, 13 Sep 2013 16:45:59 GMT
After the map task finishes , the VM terminates and all resources are

Reduces might start pulling data from completed map tasks much before all
the maps are finished. However there is a property using which you can
delay the reduce step till all the maps are completed.


On Fri, Sep 13, 2013 at 12:42 PM, Manu Reddy <manubharghav@gmail.com> wrote:

> Hi,
> I am running a memory intensive task on my cluster. The reducer class
> should load more than 2gb into memory. I was running into memory issues as
> the Map JVMS were not terminated after the completion of map tasks and
> still were holding onto memory.
> Is this normal? What can I do to ensure that my reduce function gets
> called only after all the map jvms are down?
> --
> Manu Reddy,
> mobile no: 091-8143405225,
> Hyderabad.

View raw message