hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-5694) MR AM container syslog is empty
Date Tue, 21 Jan 2014 17:32:21 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13877635#comment-13877635
] 

Jason Lowe commented on MAPREDUCE-5694:
---------------------------------------

I recently noticed that AM logs were being truncated, see MAPREDUCE-5730, and it appears to
be caused by this change.  The patch is shutting down the LogManager in the finally block
of the main thread, but the main method does not wait around for the MRAppMaster to complete.
 This means under normal conditions the log system is being shutdown sometime shortly after
startup, effectively disabling the AM logs entirely after that point.

> MR AM container syslog is empty  
> ---------------------------------
>
>                 Key: MAPREDUCE-5694
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5694
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>    Affects Versions: 2.2.0
>            Reporter: Mohammad Kamrul Islam
>            Assignee: Mohammad Kamrul Islam
>             Fix For: 2.4.0
>
>         Attachments: MAPREDUCE-5694.1.patch
>
>
> When the property "mapreduce.task.userlog.limit.kb" is set non-zero in mapped-site.xml,
AM container syslog remains empty. Without the log, it is hard to identify the cause of any
MR AM failure.
> However, if "mapreduce.task.userlog.limit.kb"  is not set (or defaulted to 0), syslog
contents are correct.
>   
> Bug details:
> For  non zero "mapreduce.task.userlog.limit.kb", the code caches the log contents into
memory, waited to be written to file when close() method is called at the end(Ref: ContainerLogAppender.java).

> Explicit call of LogManager.shutdown() from the main() (REF:MRAppMaster.java), ultimately
call the *Appender.close().
> Root Cause: 
> LogManager.shutdown() is not being called from MRAppMaster.java.
> Similar steps were taken correctly in YarnChild.java.
>   
>  



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message