hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gera Shegalov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6888) Remove audit logging of getFIleInfo()
Date Fri, 22 Aug 2014 20:00:13 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14107394#comment-14107394
] 

Gera Shegalov commented on HDFS-6888:
-------------------------------------

Hi [~airbots], sorry for being unclear.

[~kihwal] suggests:
bq. We could have logAuditEvent() check cmd against "getfileinfo" or a *collection of such
commands* and log at debug level. 

Picking this idea up can you introduce some conf like dfs.audit.<loglevel>.cmdlist=getfileinfo,anotherLogFloodingCmd,...

In {{o.a.h.hdfs.server.namenode.FSNamesystem.DefaultAuditLogger#initialize}} you could read
the list using {{auditDebugCmds conf.getTrimmedStrings("dfs.audit.debug.cmdlist"}} and use
it for filtering. Currenly v2 hardcodes "getfileinfo".

{code}
--- hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
+++ hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
@@ -359,6 +359,9 @@ private void logAuditEvent(boolean succeeded,
       UserGroupInformation ugi, InetAddress addr, String cmd, String src,
       String dst, HdfsFileStatus stat) {
     FileStatus status = null;
+    if(cmd.equals("getfileinfo") && !auditLog.isDebugEnabled()) {
+      return;
+    }
     if (stat != null) {
       Path symlink = stat.isSymlink() ? new Path(stat.getSymlink()) : null;
       Path path = dst != null ? new Path(dst) : new Path(src);
{code}

Also auditLog.isDebugEnabled() is a cheaper check, and should be done before {{equals}}




> Remove audit logging of getFIleInfo()
> -------------------------------------
>
>                 Key: HDFS-6888
>                 URL: https://issues.apache.org/jira/browse/HDFS-6888
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.5.0
>            Reporter: Kihwal Lee
>            Assignee: Chen He
>              Labels: log
>         Attachments: HDFS-6888-2.patch, HDFS-6888.patch
>
>
> The audit logging of getFileInfo() was added in HDFS-3733.  Since this is a one of the
most called method, users have noticed that audit log is now filled with this.  Since we now
have HTTP request logging, this seems unnecessary.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message