hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kihwal Lee (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4995) Make getContentSummary() less expensive
Date Wed, 13 Nov 2013 22:17:23 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13821914#comment-13821914

Kihwal Lee commented on HDFS-4995:

bq. Is switching the order of the calls completely safe? Is there any chance you can fall
through an inode reference to a directory that will yield?

Nope. computeContentSummary4Snapshot() will calculate the summary using a new context instance
with yield disabled, and then merge the individual counts.

> Make getContentSummary() less expensive
> ---------------------------------------
>                 Key: HDFS-4995
>                 URL: https://issues.apache.org/jira/browse/HDFS-4995
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: namenode
>    Affects Versions: 0.23.9, 2.3.0
>            Reporter: Kihwal Lee
>            Assignee: Kihwal Lee
>         Attachments: HDFS-4995.branch-0.23.3.patch, HDFS-4995.trunk.2.patch, HDFS-4995.trunk.3.patch,
HDFS-4995.trunk.patch, HDFS-4995.trunk1.patch
> When users call du or count DFS command, getContentSummary() method is called against
namenode. If the directory has many directories and files, it could hold the namesystem lock
for a long time. We've seen it taking over 20 seconds. Namenode should not allow regular users
to cause extended locking.

This message was sent by Atlassian JIRA

View raw message