hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marcelo Vanzin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3680) Allows customized audit logging in HDFS FSNamesystem
Date Fri, 05 Oct 2012 23:42:03 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13470788#comment-13470788
] 

Marcelo Vanzin commented on HDFS-3680:
--------------------------------------

Just to end the "what does log4j do" discussion, here's what happens when log4j appenders
throw exceptions/errors:

2012-10-05 16:32:05,748 WARN org.apache.hadoop.ipc.Server: IPC Server handler 7 on 8020, call
org.apache.hadoop.hdfs.protocol.ClientProtocol.getFileInfo from 172.29.110.222:48751: error:
java.lang.RuntimeException: testing exception handling.
java.lang.RuntimeException: testing exception handling.
        at test.AppenderBase.append(AppenderBase.java:72)
        at org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)
        at org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
        at org.apache.log4j.Category.callAppenders(Category.java:206)
        at org.apache.log4j.Category.forcedLog(Category.java:391)
        at org.apache.log4j.Category.log(Category.java:856)
        at org.apache.commons.logging.impl.Log4JLogger.info(Log4JLogger.java:199)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.logAuditEvent(FSNamesystem.java:258)

2012-10-05 16:32:08,820 WARN org.apache.hadoop.ipc.Server: IPC Server handler 11 on 8020,
call org.apache.hadoop.hdfs.protocol.ClientProtocol.getFileInfo from 172.29.110.222:48754:
error: java.lang.Error: testing error handling.
java.lang.Error: testing error handling.
        at test.AppenderBase.append(AppenderBase.java:74)
        at org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)
        at org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
        at org.apache.log4j.Category.callAppenders(Category.java:206)
        at org.apache.log4j.Category.forcedLog(Category.java:391)
        at org.apache.log4j.Category.log(Category.java:856)
        at org.apache.commons.logging.impl.Log4JLogger.info(Log4JLogger.java:199)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.logAuditEvent(FSNamesystem.java:258)

log4j appenders in general use log4j's ErrorHandler when errors occur, instead of throwing
exceptions. The default one (http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/helpers/OnlyOnceErrorHandler.html)
just drops errors after printing the first one to stderr. So the NameNode is not shut down;
it just keeps running, and audit logs are silently dropped. Which is another reason why I'll
maintain that the "what to do when logging an audit fails" issue is not particular to my patch.
                
> Allows customized audit logging in HDFS FSNamesystem
> ----------------------------------------------------
>
>                 Key: HDFS-3680
>                 URL: https://issues.apache.org/jira/browse/HDFS-3680
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: name-node
>    Affects Versions: 2.0.0-alpha
>            Reporter: Marcelo Vanzin
>            Assignee: Marcelo Vanzin
>            Priority: Minor
>         Attachments: accesslogger-v1.patch, accesslogger-v2.patch, hdfs-3680-v3.patch,
hdfs-3680-v4.patch, hdfs-3680-v5.patch, hdfs-3680-v6.patch, hdfs-3680-v7.patch, hdfs-3680-v8.patch
>
>
> Currently, FSNamesystem writes audit logs to a logger; that makes it easy to get audit
logs in some log file. But it makes it kinda tricky to store audit logs in any other way (let's
say a database), because it would require the code to implement a log appender (and thus know
what logging system is actually being used underneath the fa├žade), and parse the textual
log message generated by FSNamesystem.
> I'm attaching a patch that introduces a cleaner interface for this use case.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message