hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andy Isaacson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3680) Allows customized audit logging in HDFS FSNamesystem
Date Wed, 01 Aug 2012 23:54:03 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13427016#comment-13427016

Andy Isaacson commented on HDFS-3680:

Given the designed use case for audit loggers -- they are creating business critical immutable
logs of filesystem access -- I do not agree that the NN should always continue to operate
when an audit logger fails.

The purpose of audit logging is, "*every* time a user accesses a file, an audit log entry
is generated".  Not "well, most of the time, unless some system had a hiccup, in which case
we just silently allow access without audit logs".  Think of it as similar to an authentication
system; if PAM is unable to access the LDAP server, it returns DENY, not ALLOW.

Yes, this adds points of failure to the system -- if a fail-closed audit log fails, then the
NN will stop allowing access. For customers that need this capability, that may be an acceptable
trade-off; that's a decision that should be left to the customer, and for which we should
establish sensible defaults.

I think the simplest solution is for the NN to simply shutdown if the AuditLogger fails. 
This probably needs a warning in fairly big letters.

If there's user demand for fail-open audit logs, that can be done in the AuditLogger implementation,
by catching all exceptions and swallowing them.  Or we could add separate configurations for
"fallible audit loggers" and "critical audit loggers".

(I'm willing to listen to contrary opinions in this area -- if you think I'm wrong about this,
please do say so!)
> Allows customized audit logging in HDFS FSNamesystem
> ----------------------------------------------------
>                 Key: HDFS-3680
>                 URL: https://issues.apache.org/jira/browse/HDFS-3680
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: name-node
>    Affects Versions: 2.0.0-alpha
>            Reporter: Marcelo Vanzin
>            Assignee: Marcelo Vanzin
>            Priority: Minor
>         Attachments: accesslogger-v1.patch, accesslogger-v2.patch, hdfs-3680-v3.patch,
hdfs-3680-v4.patch, hdfs-3680-v5.patch
> Currently, FSNamesystem writes audit logs to a logger; that makes it easy to get audit
logs in some log file. But it makes it kinda tricky to store audit logs in any other way (let's
say a database), because it would require the code to implement a log appender (and thus know
what logging system is actually being used underneath the fa├žade), and parse the textual
log message generated by FSNamesystem.
> I'm attaching a patch that introduces a cleaner interface for this use case.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message