hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marcelo Vanzin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3680) Allows customized audit logging in HDFS FSNamesystem
Date Mon, 30 Jul 2012 19:55:36 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13425162#comment-13425162

Marcelo Vanzin commented on HDFS-3680:

Hi Suresh, thanks for the comments. Replies below.

bq. Existing LOG could handle different levels, such as trace, debug, info, warn etc. I understand
we probably use info level logs now. Should we consider adding such levels to FSAccessLogger?

IMO "trace", "debug" et al are related to the logger implementation, not the audit event.
The audit event already has information about what it's about; e.g., access allowed / denied,
etc. The logger can choose to map that information to something that makes sense in the target;
for example, logging denied events with "warning" level. But such level wouldn't make much
sense in a different implementation (say, for example, writing to a database). 

bq. Why call it FSAccessLogger and not AuditLogger? AuditLogger seems to be a more generic

Fair enough, will change.

bq. You cannot make this InterfaceAudience.Public given HdfsFileStatus and UserGroupInformation
are not Public.

That poses an issue, though. Would there be resistance to make those two classes public? The
problem with them not being public is that it would then require the information to be exposed
in some other way: either a new class that just provides the same information (= code duplication,
overhead to create the copy), or a string (difficult to parse, overhead to create the string).

bq. Do not catch blanket Exception. Instead catch the specific exception you want to handle.

Are you OK with catching RuntimeException? I'm really being paranoid here, because I don't
want a buggy AccessLogger to suddenly cause the NameNode to go down. Alternatively, the access
logger could be disabled when it throws an exception (similar to how HBase disables coprocessors
when they throw unexpected exceptions).

bq. Please consider adding a separate test instead of adding this to TestFSNamesystem.java

Any particular reason why? The test is testing functionality of FSNamesystem (instantiating
and using custom AccessLoggers), so it makes sense to me that it's part of FSNamesystem's

bq. Mock may be better than TestAccessLogger implementation. If you still want to use TestAccessLogger
make it private class.

I can't mock because FSNamesystem instantiates the access logger using Class.forName(). I
also believe I cannot make it private for the same reason: FSNamesystem trying to call the
(now private) constructor would cause an IllegalAccessException.

> Allows customized audit logging in HDFS FSNamesystem
> ----------------------------------------------------
>                 Key: HDFS-3680
>                 URL: https://issues.apache.org/jira/browse/HDFS-3680
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: name-node
>    Affects Versions: 2.0.0-alpha
>            Reporter: Marcelo Vanzin
>            Assignee: Marcelo Vanzin
>            Priority: Minor
>         Attachments: accesslogger-v1.patch, accesslogger-v2.patch, hdfs-3680-v3.patch,
> Currently, FSNamesystem writes audit logs to a logger; that makes it easy to get audit
logs in some log file. But it makes it kinda tricky to store audit logs in any other way (let's
say a database), because it would require the code to implement a log appender (and thus know
what logging system is actually being used underneath the fa├žade), and parse the textual
log message generated by FSNamesystem.
> I'm attaching a patch that introduces a cleaner interface for this use case.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message