hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eli Collins (Updated) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-8149) cap space usage of default log4j rolling policy
Date Thu, 29 Mar 2012 21:05:26 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-8149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Eli Collins updated HADOOP-8149:
--------------------------------

          Resolution: Fixed
       Fix Version/s: 2.0.0
    Target Version/s:   (was: 0.23.3)
        Release Note: Hadoop log files are now rolled by size instead of date (daily) by default.
Tools that depend on the log file name format will need to be updated. Users who would like
to maintain the previous settings of hadoop.root.logger and hadoop.security.logger can use
their current log4j.properties files and update the HADOOP_ROOT_LOGGER and HADOOP_SECURITY_LOGGER
environment variables to use DRFA and DRFAS respectively.  (was: Hadoop log files are now
rolled by size instead of date (daily) by default. Tools that depend on the log file name
format will need to be updated. Users who would like to maintain the previous settings can
use their current log4j.properties files and update the   HADOOP_ROOT_LOGGER and HADOOP_SECURITY_LOGGER
environment variables to use DRFA and DRFAS respectively.)
              Status: Resolved  (was: Patch Available)

I've committed this and merged to branch-2. 

Thanks Pat! Thank you Kihwal and Rajiv for the feedback!
                
> cap space usage of default log4j rolling policy 
> ------------------------------------------------
>
>                 Key: HADOOP-8149
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8149
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: conf
>            Reporter: Patrick Hunt
>            Assignee: Patrick Hunt
>             Fix For: 2.0.0
>
>         Attachments: HADOOP-8149.patch, HADOOP-8149.patch, HADOOP-8149.patch, HADOOP-8149.patch,
HADOOP-8149.patch
>
>
> I've seen several critical production issues because logs are not automatically removed
after some time and accumulate. Changes to Hadoop's default log4j file appender would help
with this.
> I recommend we move to an appender which:
> 1) caps the max file size (configurable)
> 2) caps the max number of files to keep (configurable)
> 3) uses rolling file appender rather than DRFA, see the warning here:
> http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/DailyRollingFileAppender.html
> Specifically: "DailyRollingFileAppender has been observed to exhibit synchronization
issues and data loss."
> We'd lose (based on the default log4j configuration) the daily rolling aspect, however
increase reliability.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message