hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Navraj S. Chohan" <nlak...@gmail.com>
Subject Mapper Process Duration
Date Thu, 04 Feb 2010 19:52:08 GMT
Hello,
I have a question about mapred.Child processes. Even though a mapper is
finished I see that the process (from ps) stays around longer than reported
on the hadoop MR webpage.
What is the mapper process doing after it has reported that it is finished?
To illustrate my question: I see that one mapper reports it finished in 9
seconds but from logging ps output every second, I see it last for 24
seconds before exiting. I essentially see this for each mapper.

Lastly, where can I find information on how exactly the map reduce framework
reuses JVMs. The reason I'm asking is because I see that with reuse on
(mapred.job.reuse.jvm.num.tasks set to -1), the pid's change for each new
mapper. How can this be without starting a new JVM?
Thanks!

-- 
Navraj S. Chohan
nlake44@gmail.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message