hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rajiv Chittajallu (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-8149) cap space usage of default log4j rolling policy
Date Tue, 20 Mar 2012 19:45:50 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-8149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13233690#comment-13233690
] 

Rajiv Chittajallu commented on HADOOP-8149:
-------------------------------------------

bq. 5GB seems like a reasonable number to me. I'm excited that our defaults may no longer
fill up our hard disks when stuff is awry.

from my experience in running hadoop, having sufficient logs helped us in debugging issues
when there are issues. With the current logging and error reporting of hadoop daemons, I wouldn't
recommend log4j purging logs, especially by space, to any one.
 
                
> cap space usage of default log4j rolling policy 
> ------------------------------------------------
>
>                 Key: HADOOP-8149
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8149
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: conf
>            Reporter: Patrick Hunt
>            Assignee: Patrick Hunt
>         Attachments: HADOOP-8149.patch, HADOOP-8149.patch, HADOOP-8149.patch
>
>
> I've seen several critical production issues because logs are not automatically removed
after some time and accumulate. Changes to Hadoop's default log4j file appender would help
with this.
> I recommend we move to an appender which:
> 1) caps the max file size (configurable)
> 2) caps the max number of files to keep (configurable)
> 3) uses rolling file appender rather than DRFA, see the warning here:
> http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/DailyRollingFileAppender.html
> Specifically: "DailyRollingFileAppender has been observed to exhibit synchronization
issues and data loss."
> We'd lose (based on the default log4j configuration) the daily rolling aspect, however
increase reliability.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message