Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Fri, 23 Sep 2016 19:30:20 +0000 (UTC)
From: "Xiao Chen (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.12863470.1441992891000.651652.1474659020806@Atlassian.JIRA>
In-Reply-To: <JIRA.12863470.1441992891000@Atlassian.JIRA>
References: <JIRA.12863470.1441992891000@Atlassian.JIRA> <JIRA.12863470.1441992891726@arcas>
Subject: [jira] [Commented] (HDFS-9063) Correctly handle snapshot path for
 getContentSummary
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Fri, 23 Sep 2016 19:30:23 -0000


    [ https://issues.apache.org/jira/browse/HDFS-9063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15517352#comment-15517352 ] 

Xiao Chen commented on HDFS-9063:
---------------------------------

Hm, I'm a little confused by this comment:
bq. I.e., since we already have a number in the content summary to indicate the total number of snapshots, the number of snapshots is no longer added into the directory number. 
[~jingzhao], could you clarify? 

I have just chatted offline with Manoj. The snapshotXXXX in {{ContentSummary}}, added by HDFS-8986, was to track how many files/directories are created under a snapshot, so that {{count -x}} could exclude them and calculate a result of direct usage. It is *not* tracking how many snapshots have been taken on a directory.

Do we have number of snapshots visible to users? I only found {{SnapshotManager#snapshotCounter}} at the hdfs level, and {{DirectorySnapshottableFeature#getNumSnapshots}} at the dir level - neither seems to be propagating to any shell commands.

My 2 cents are this is will not reasonably impact quotas, so maybe good as-is. Nice to have this displayed too, if users want to get it absolutely accurate. Please share your thoughts, thanks!

> Correctly handle snapshot path for getContentSummary
> ----------------------------------------------------
>
>                 Key: HDFS-9063
>                 URL: https://issues.apache.org/jira/browse/HDFS-9063
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: namenode
>            Reporter: Jing Zhao
>            Assignee: Jing Zhao
>              Labels: incompatible
>             Fix For: 2.8.0, 3.0.0-alpha1
>
>         Attachments: HDFS-9063.000.patch, test.001.patch, test.002.patch
>
>
> The current getContentSummary implementation does not take into account the snapshot path, thus if we have the following ops:
> 1. create dirs /foo/bar
> 2. take snapshot s1 on /foo
> 3. create a 1 byte file /foo/bar/baz
> then "du /foo" and "du /foo/.snapshot/s1" can report same results for "bar", which is incorrect since the 1 byte file is not included in snapshot s1.
> In the meanwhile, the snapshot diff list size is no longer included in the computation result. This can bring minor incompatibility but is consistent with the change in HDFS-7728.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org