hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kihwal Lee (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-4995) Make getContentSummary() less expensive
Date Tue, 29 Oct 2013 18:18:27 GMT

     [ https://issues.apache.org/jira/browse/HDFS-4995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Kihwal Lee updated HDFS-4995:

    Attachment: HDFS-4995.trunk.2.patch

Modified the patch to allow querying lock yield count. The test case checks it against the
expected number. 

We have tested this approach in production environment.

> Make getContentSummary() less expensive
> ---------------------------------------
>                 Key: HDFS-4995
>                 URL: https://issues.apache.org/jira/browse/HDFS-4995
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: namenode
>    Affects Versions: 0.23.9, 2.3.0
>            Reporter: Kihwal Lee
>         Attachments: HDFS-4995.trunk1.patch, HDFS-4995.trunk.2.patch, HDFS-4995.trunk.patch
> When users call du or count DFS command, getContentSummary() method is called against
namenode. If the directory has many directories and files, it could hold the namesystem lock
for a long time. We've seen it taking over 20 seconds. Namenode should not allow regular users
to cause extended locking.

This message was sent by Atlassian JIRA

View raw message