hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suresh Srinivas (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3680) Allows customized audit logging in HDFS FSNamesystem
Date Thu, 02 Aug 2012 00:34:03 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13427039#comment-13427039

Suresh Srinivas commented on HDFS-3680:

bq.  I don't think you can have it both ways: either misbehaving loggers are removed, or they're
always called...
I think you are missing my point. Misbehaving loggers should be removed. But given the importance
of audit logging, if the error condition associated with that logger is fixed, we need a mechanism
to reinstate that logger back without having to restart the namenode. By the same token, we
also need to discuss how a logger that was removed can get back in sync with the logs it might
have missed. If it is complicated to implement, then lets discuss how important such a functionality

bq. Well, let me rephrase. Talking about a separate daemon to serve audit logs is moot because,
well, it wouldn't be HDFS code anymore. Anyone can do that without requiring a single line
change in HDFS.
If we conclude that it is a better solution, why not.

bq.  it is possible today to have a custom audit logger inside the NameNode without my patch
Now I see your earlier point. However since this is adding support in HDFS, lets consider
the use cases we need to support. 

bq. we could add separate configurations for "fallible audit loggers" and "critical audit
I was thinking along the same lines. But why would one use fallible audit logger, to build
what functionality, and what guarantees they can provide with such a solution. Are there valid
"best effort only", "might be incomplete" solutions around audit logging? This made me conclude
that only thing that is really useful is "critical audit loggers".

> Allows customized audit logging in HDFS FSNamesystem
> ----------------------------------------------------
>                 Key: HDFS-3680
>                 URL: https://issues.apache.org/jira/browse/HDFS-3680
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: name-node
>    Affects Versions: 2.0.0-alpha
>            Reporter: Marcelo Vanzin
>            Assignee: Marcelo Vanzin
>            Priority: Minor
>         Attachments: accesslogger-v1.patch, accesslogger-v2.patch, hdfs-3680-v3.patch,
hdfs-3680-v4.patch, hdfs-3680-v5.patch
> Currently, FSNamesystem writes audit logs to a logger; that makes it easy to get audit
logs in some log file. But it makes it kinda tricky to store audit logs in any other way (let's
say a database), because it would require the code to implement a log appender (and thus know
what logging system is actually being used underneath the fa├žade), and parse the textual
log message generated by FSNamesystem.
> I'm attaching a patch that introduces a cleaner interface for this use case.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message