hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gera Shegalov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6888) Remove audit logging of getFIleInfo()
Date Fri, 22 Aug 2014 20:00:13 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14107394#comment-14107394

Gera Shegalov commented on HDFS-6888:

Hi [~airbots], sorry for being unclear.

[~kihwal] suggests:
bq. We could have logAuditEvent() check cmd against "getfileinfo" or a *collection of such
commands* and log at debug level. 

Picking this idea up can you introduce some conf like dfs.audit.<loglevel>.cmdlist=getfileinfo,anotherLogFloodingCmd,...

In {{o.a.h.hdfs.server.namenode.FSNamesystem.DefaultAuditLogger#initialize}} you could read
the list using {{auditDebugCmds conf.getTrimmedStrings("dfs.audit.debug.cmdlist"}} and use
it for filtering. Currenly v2 hardcodes "getfileinfo".

--- hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
+++ hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
@@ -359,6 +359,9 @@ private void logAuditEvent(boolean succeeded,
       UserGroupInformation ugi, InetAddress addr, String cmd, String src,
       String dst, HdfsFileStatus stat) {
     FileStatus status = null;
+    if(cmd.equals("getfileinfo") && !auditLog.isDebugEnabled()) {
+      return;
+    }
     if (stat != null) {
       Path symlink = stat.isSymlink() ? new Path(stat.getSymlink()) : null;
       Path path = dst != null ? new Path(dst) : new Path(src);

Also auditLog.isDebugEnabled() is a cheaper check, and should be done before {{equals}}

> Remove audit logging of getFIleInfo()
> -------------------------------------
>                 Key: HDFS-6888
>                 URL: https://issues.apache.org/jira/browse/HDFS-6888
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.5.0
>            Reporter: Kihwal Lee
>            Assignee: Chen He
>              Labels: log
>         Attachments: HDFS-6888-2.patch, HDFS-6888.patch
> The audit logging of getFileInfo() was added in HDFS-3733.  Since this is a one of the
most called method, users have noticed that audit log is now filled with this.  Since we now
have HTTP request logging, this seems unnecessary.

This message was sent by Atlassian JIRA

View raw message