hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3680) Allows customized audit logging in HDFS FSNamesystem
Date Fri, 20 Jul 2012 16:31:35 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13419323#comment-13419323

Todd Lipcon commented on HDFS-3680:

Daryn: decent point. Do you think we should just attach a big warning to the interface saying
"make sure you do queueing", or should we build the queue into the core code itself?

The issue with doing our own queueing is that it's still nice to have the log4j logs interleave
in-order with the other related log messages. I wouldn't want the log to be async. But definitely
reporting to a JDBC-backed DB should be async via a queue.

Another thing to think through with the queue: we probably don't want it to be unbounded for
fear of OOME. And I imagine some organizations may have a strict audit requirement that limits
the possibility of lost audit events. In such a case, operations probably _should_ block on
a full queue - else events could be permanently stuck in the queue if the underlying DB is
down, and then lost if the NN crashes.

Marcelo, any thoughts?
> Allows customized audit logging in HDFS FSNamesystem
> ----------------------------------------------------
>                 Key: HDFS-3680
>                 URL: https://issues.apache.org/jira/browse/HDFS-3680
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: name-node
>    Affects Versions: 2.0.0-alpha
>            Reporter: Marcelo Vanzin
>            Assignee: Marcelo Vanzin
>            Priority: Minor
>         Attachments: accesslogger-v1.patch, accesslogger-v2.patch
> Currently, FSNamesystem writes audit logs to a logger; that makes it easy to get audit
logs in some log file. But it makes it kinda tricky to store audit logs in any other way (let's
say a database), because it would require the code to implement a log appender (and thus know
what logging system is actually being used underneath the fa├žade), and parse the textual
log message generated by FSNamesystem.
> I'm attaching a patch that introduces a cleaner interface for this use case.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message