hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kihwal Lee (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4995) Make getContentSummary() less expensive
Date Fri, 18 Oct 2013 22:57:42 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13799628#comment-13799628

Kihwal Lee commented on HDFS-4995:

I've tested this with 100 threads creating, renaming deleting and also reader client calling
getContentSummary(). The threshold was set to 2 in this test, so that it does lock yielding
very frequently.

> Make getContentSummary() less expensive
> ---------------------------------------
>                 Key: HDFS-4995
>                 URL: https://issues.apache.org/jira/browse/HDFS-4995
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: namenode
>    Affects Versions: 0.23.9, 2.3.0
>            Reporter: Kihwal Lee
>         Attachments: HDFS-4995.trunk1.patch, HDFS-4995.trunk.patch
> When users call du or count DFS command, getContentSummary() method is called against
namenode. If the directory has many directories and files, it could hold the namesystem lock
for a long time. We've seen it taking over 20 seconds. Namenode should not allow regular users
to cause extended locking.

This message was sent by Atlassian JIRA

View raw message