hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xiao Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9063) Correctly handle snapshot path for getContentSummary
Date Fri, 23 Sep 2016 19:30:20 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15517352#comment-15517352

Xiao Chen commented on HDFS-9063:

Hm, I'm a little confused by this comment:
bq. I.e., since we already have a number in the content summary to indicate the total number
of snapshots, the number of snapshots is no longer added into the directory number. 
[~jingzhao], could you clarify? 

I have just chatted offline with Manoj. The snapshotXXXX in {{ContentSummary}}, added by HDFS-8986,
was to track how many files/directories are created under a snapshot, so that {{count -x}}
could exclude them and calculate a result of direct usage. It is *not* tracking how many snapshots
have been taken on a directory.

Do we have number of snapshots visible to users? I only found {{SnapshotManager#snapshotCounter}}
at the hdfs level, and {{DirectorySnapshottableFeature#getNumSnapshots}} at the dir level
- neither seems to be propagating to any shell commands.

My 2 cents are this is will not reasonably impact quotas, so maybe good as-is. Nice to have
this displayed too, if users want to get it absolutely accurate. Please share your thoughts,

> Correctly handle snapshot path for getContentSummary
> ----------------------------------------------------
>                 Key: HDFS-9063
>                 URL: https://issues.apache.org/jira/browse/HDFS-9063
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: namenode
>            Reporter: Jing Zhao
>            Assignee: Jing Zhao
>              Labels: incompatible
>             Fix For: 2.8.0, 3.0.0-alpha1
>         Attachments: HDFS-9063.000.patch, test.001.patch, test.002.patch
> The current getContentSummary implementation does not take into account the snapshot
path, thus if we have the following ops:
> 1. create dirs /foo/bar
> 2. take snapshot s1 on /foo
> 3. create a 1 byte file /foo/bar/baz
> then "du /foo" and "du /foo/.snapshot/s1" can report same results for "bar", which is
incorrect since the 1 byte file is not included in snapshot s1.
> In the meanwhile, the snapshot diff list size is no longer included in the computation
result. This can bring minor incompatibility but is consistent with the change in HDFS-7728.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message