hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Navraj S. Chohan" <nlak...@gmail.com>
Subject Re: Mapper Process Duration
Date Thu, 04 Feb 2010 22:34:05 GMT
Nevermind,
I had set the reuse to the wrong value. It seems that setting the reuse to 0
acts the same way as setting it to -1.


On Thu, Feb 4, 2010 at 2:52 PM, Navraj S. Chohan <nlake44@gmail.com> wrote:

> Hello,
> I have a question about mapred.Child processes. Even though a mapper is
> finished I see that the process (from ps) stays around longer than reported
> on the hadoop MR webpage.
> What is the mapper process doing after it has reported that it is
> finished?
> To illustrate my question: I see that one mapper reports it finished in 9
> seconds but from logging ps output every second, I see it last for 24
> seconds before exiting. I essentially see this for each mapper.
>
> Lastly, where can I find information on how exactly the map reduce
> framework reuses JVMs. The reason I'm asking is because I see that with
> reuse on (mapred.job.reuse.jvm.num.tasks set to -1), the pid's change for
> each new mapper. How can this be without starting a new JVM?
> Thanks!
>
> --
> Navraj S. Chohan
> nlake44@gmail.com
>



-- 
Navraj S. Chohan
nlake44@gmail.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message