hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shrinivas Joshi <jshrini...@gmail.com>
Subject JVM reuse and log files
Date Wed, 30 Mar 2011 20:17:03 GMT
It seems like when JVM reuse is enabled map task log data is not getting
written to their corresponding log files; log data from certain map tasks
gets appended to log files corresponding to some other map task.

For example, I have a case here where 8 map JVMs are running simultaneously
and all syslog data from map task 9, 17 and 25 gets appended in to log file
for map task 0. Whereas no syslog file gets generated in
attempt_*m_000009_0/ , attempt_*m_000017_0/ and attempt_*m_000025_0/
folders. This job creates 32 map tasks.
This behavior might also be applicable to reduce log files, however, in our
case total # of reduce tasks is not more than max reduce JVMs running at the
same time and hence it might not be manifesting.

BTW, this is on Apache distro 0.21.0.


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message