hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Binglin Chang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4667) Capture renamed files/directories in snapshot diff report
Date Mon, 26 Aug 2013 03:35:52 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13749814#comment-13749814

Binglin Chang commented on HDFS-4667:

Thanks for the comments, [~jingzhao]. 
About rename across snapshottable directory, I have some concerns:
1. We currently don't check the permissions of inodes outside the snapshottable directory,
is it legal to expose those information?
2. The diff is most likely to be used for incremental backup(using distcp), in this context,
for a rename operation, if the source directory is not in this snapshot root, it is better
to consider it as newly created, which is easier for distcp to handle.

bq. the rename outside of a snapshottable directory can be detected. 
Yes, I considered this approach, it can be a better way of generating diff, I tried this,
but currently we don't have a method to get an inode's full path given a specified snapshot,
e.g. FSDirectory.getFullPathName(INode inode, Snapshot snapshot). I tried to write one, but
I am not sure it is correct. Please review the newly attached the patch.

bq. Another possible advantage of this method is that we do not need to check the subtree
of a new-created directory while generating the diff report.

I find that all the directories(created/deleted) need to be traversed, because newly-created/deleted
directories may contain renamed dirs/files.

> Capture renamed files/directories in snapshot diff report
> ---------------------------------------------------------
>                 Key: HDFS-4667
>                 URL: https://issues.apache.org/jira/browse/HDFS-4667
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: datanode, namenode
>            Reporter: Jing Zhao
>            Assignee: Binglin Chang
>         Attachments: HDFS-4667.demo.patch, HDFS-4667.v1.patch
> Currently in the diff report we only show file/dir creation, deletion and modification.
After rename with snapshots is supported, renamed file/dir should also be captured in the
diff report.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message