hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lei (Eddy) Xu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5369) Support negative caching of user-group mapping
Date Thu, 26 Jun 2014 00:04:25 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14044201#comment-14044201
] 

Lei (Eddy) Xu commented on HDFS-5369:
-------------------------------------

[~andrew.wang] Thanks for your comments and pointing to the issues raised from HADOOP-8088.
I would consider this patch as reference implementation, so that it has not yet addressed
the error handling of a transient error. I will to address it as well as SLAs after getting
some inputs. 

Moreover, the default value for negative cache timeout is still not clear to me. It might
need more field data to support an appropriate timeout value here.

> Support negative caching of user-group mapping
> ----------------------------------------------
>
>                 Key: HDFS-5369
>                 URL: https://issues.apache.org/jira/browse/HDFS-5369
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: namenode
>    Affects Versions: 2.2.0
>            Reporter: Andrew Wang
>         Attachments: HDFS-5369.000.patch
>
>
> We've seen a situation at a couple of our customers where interactions from an unknown
user leads to a high-rate of group mapping calls. In one case, this was happening at a rate
of 450 calls per second with the shell-based group mapping, enough to severely impact overall
namenode performance and also leading to large amounts of log spam (prints a stack trace each
time).
> Let's consider negative caching of group mapping, as well as quashing the rate of this
log message.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message