Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Wed, 13 Nov 2013 22:17:23 +0000 (UTC)
From: "Kihwal Lee (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.12657854.1373921424915.69673.1384381043591@arcas>
In-Reply-To: <JIRA.12657854.1373921424915@arcas>
References: <JIRA.12657854.1373921424915@arcas>
Subject: [jira] [Commented] (HDFS-4995) Make getContentSummary() less
 expensive
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-4995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13821914#comment-13821914 ] 

Kihwal Lee commented on HDFS-4995:
----------------------------------

bq. Is switching the order of the calls completely safe? Is there any chance you can fall through an inode reference to a directory that will yield?

Nope. computeContentSummary4Snapshot() will calculate the summary using a new context instance with yield disabled, and then merge the individual counts.

> Make getContentSummary() less expensive
> ---------------------------------------
>
>                 Key: HDFS-4995
>                 URL: https://issues.apache.org/jira/browse/HDFS-4995
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: namenode
>    Affects Versions: 0.23.9, 2.3.0
>            Reporter: Kihwal Lee
>            Assignee: Kihwal Lee
>         Attachments: HDFS-4995.branch-0.23.3.patch, HDFS-4995.trunk.2.patch, HDFS-4995.trunk.3.patch, HDFS-4995.trunk.patch, HDFS-4995.trunk1.patch
>
>
> When users call du or count DFS command, getContentSummary() method is called against namenode. If the directory has many directories and files, it could hold the namesystem lock for a long time. We've seen it taking over 20 seconds. Namenode should not allow regular users to cause extended locking.


--
This message was sent by Atlassian JIRA
(v6.1#6144)