hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suresh Srinivas (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3680) Allows customized audit logging in HDFS FSNamesystem
Date Fri, 05 Oct 2012 04:42:50 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13470019#comment-13470019

Suresh Srinivas commented on HDFS-3680:

bq. What do you mean "symlink is being done"? I'm not creating any symlinks. I'm creating
a FileStatus object based on an HdfsFileStatus, which is a private audience class and thus
cannot be used in the public audience AuditLogger.

Why is the following code part of this patch?
+    if (stat != null) {
+      Path symlink = stat.isSymlink() ? new Path(stat.getSymlink()) : null;
+      Path path = dst != null ? new Path(dst) : new Path(src);
+      status = new FileStatus(stat.getLen(), stat.isDir(),
+          stat.getReplication(), stat.getBlockSize(), stat.getModificationTime(),
+          stat.getAccessTime(), stat.getPermission(), stat.getOwner(),
+          stat.getGroup(), symlink, path);
+    }

bq. IOException would be one of many exceptions custom audit loggers could throw. So I don't
see why it should be special-cased here. My opinion is that audit loggers in general shouldn't
throw exceptions; if they really want to, having to throw a RuntimeException indicates that
it's not really an expected case.
Given loggers are going to some kind of io, to database, or some server etc. IOException should
be expected and seems like a logical thing to throw and not RunTimeException. The user of
audit logger needs to deal with that exception.

bq. I don't think there was an ultimate decision; my current patch follows the "things work
just like before" approach: currently, if the logging system throws some kind of exception,
it fails the request, just like the new code does. If that's not desired, then it can be changed,
but I think that's outside the scope of this patch.
I do not think it is outside the scope of this patch. Current logger could fail, on system
failures. However here it may fail because poorly written code, network errors etc. I thought
we decided NN will abort [here|https://issues.apache.org/jira/browse/HDFS-3680?focusedCommentId=13427039&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13427039],
and you seem to agree [here|https://issues.apache.org/jira/browse/HDFS-3680?focusedCommentId=13427550&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13427550].
Let me know if I missed any discussion that decided contrary to that.

hdfs-default.xml document does not cover my previous comment:
bq.  The document should include how to set it up, the expectation from the audit log implementation
and the impact of this configuration when things do not work correctly.

> Allows customized audit logging in HDFS FSNamesystem
> ----------------------------------------------------
>                 Key: HDFS-3680
>                 URL: https://issues.apache.org/jira/browse/HDFS-3680
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: name-node
>    Affects Versions: 2.0.0-alpha
>            Reporter: Marcelo Vanzin
>            Assignee: Marcelo Vanzin
>            Priority: Minor
>         Attachments: accesslogger-v1.patch, accesslogger-v2.patch, hdfs-3680-v3.patch,
hdfs-3680-v4.patch, hdfs-3680-v5.patch, hdfs-3680-v6.patch, hdfs-3680-v7.patch, hdfs-3680-v8.patch
> Currently, FSNamesystem writes audit logs to a logger; that makes it easy to get audit
logs in some log file. But it makes it kinda tricky to store audit logs in any other way (let's
say a database), because it would require the code to implement a log appender (and thus know
what logging system is actually being used underneath the fa├žade), and parse the textual
log message generated by FSNamesystem.
> I'm attaching a patch that introduces a cleaner interface for this use case.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message