hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daryn Sharp (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3680) Allows customized audit logging in HDFS FSNamesystem
Date Thu, 02 Aug 2012 19:18:03 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13427538#comment-13427538

Daryn Sharp commented on HDFS-3680:

Apologies if I wasn't clear, but by RPC I meant using the existing RPC server framework used
by all hadoop daemons.  That's where we can leverage secure auth, retries, timeouts, failover,

* Are audit logs written synchronously?

Consensus seems to be yes, I'm inclined to agree.

* If yes, is the extra latency for the RPC call acceptable?

I'd assume it can't be that bad since most of hadoop uses it.

* If no, how do we reliably know that audit logs have been written?

The RPC proxies implement synchronous calls.

* What's an acceptable timeout for the RPC call?

It's configurable, so it's based on what the particular user will tolerate?

bq.   I just don't think that, in the end, it's much different than allowing 3rd party code
into the NN

Sure it is, as noted before, it buffers the NN against 3rd party code calling exit, segfaulting
in JNI, memory and fd leaks, OOM, etc.  If the daemon misbehaves or blows up then it shouldn't
compromise the integrity of the namespace.
> Allows customized audit logging in HDFS FSNamesystem
> ----------------------------------------------------
>                 Key: HDFS-3680
>                 URL: https://issues.apache.org/jira/browse/HDFS-3680
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: name-node
>    Affects Versions: 2.0.0-alpha
>            Reporter: Marcelo Vanzin
>            Assignee: Marcelo Vanzin
>            Priority: Minor
>         Attachments: accesslogger-v1.patch, accesslogger-v2.patch, hdfs-3680-v3.patch,
hdfs-3680-v4.patch, hdfs-3680-v5.patch
> Currently, FSNamesystem writes audit logs to a logger; that makes it easy to get audit
logs in some log file. But it makes it kinda tricky to store audit logs in any other way (let's
say a database), because it would require the code to implement a log appender (and thus know
what logging system is actually being used underneath the fa├žade), and parse the textual
log message generated by FSNamesystem.
> I'm attaching a patch that introduces a cleaner interface for this use case.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message