hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suresh Srinivas (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4103) Support O(1) snapshot creation
Date Thu, 20 Dec 2012 06:01:14 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13536803#comment-13536803

Suresh Srinivas commented on HDFS-4103:

This is a long patch and a very laborious to review.

Here are the comments I have accumulated so far. I have few more files to review:
# INode#save2Snapshot() - please add more javadoc to the method to indicate "Inode is being
modified. The previous version of the inode needs to be recorded in the latest snapshot."
It may also be good to call the method, recordModification().
#* In a subsequent jira, do you think it is a good idea to have an interface called SnapshotTracker/SnapshotRecorder
in which we can have methods such as save2Snapshot etc?
# INodeDirectory.java
#* "the latest snapshot has been taken" you mean "the latest snapshot +that+ has been taken"
#* getChild() does not use isLastINode parameter
# INodeDirectoryWithQuota#setQuota - @param names are incorrect
# FSDirectory#unprotectedDelete - Can you create a jira to ensure when a file is deleted,
if it exists no in the latest snapshot but much older snapshots, the blocks are not deleted?
This can be done in another jira. Also:
#* If the deleted file has 3 blocks and snapshotted file has first two blocks only, do we
delete the 3rd block? This can be addressed in another jira as well.
# FSDirectory.java - replaceSelf4INodeDirectory() calls should be moved to INodeDirectoryWithQuota.setQuota()
# INodeDirectorySnapshot
#* format the #snapshotByNames javadoc comment
#* SnapshotDiff - please use <pre> and </pre> to ensure the javadoc is readable.
Consider making this static class by passing appropriate outer class state to it.

> Support O(1) snapshot creation
> ------------------------------
>                 Key: HDFS-4103
>                 URL: https://issues.apache.org/jira/browse/HDFS-4103
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: namenode
>    Affects Versions: Snapshot (HDFS-2802)
>            Reporter: Tsz Wo (Nicholas), SZE
>            Assignee: Tsz Wo (Nicholas), SZE
>         Attachments: h4103_20121129.patch, h4103_20121202b.patch, h4103_20121202.patch,
h4103_20121209b.patch, h4103_20121209.patch, h4103_20121210b.patch, h4103_20121210.patch,
h4103_20121211.patch, h4103_20121212b.patch, h4103_20121212.patch, h4103_20121213.patch, h4103_20121215b.patch,
h4103_20121215.patch, h4103_20121216.patch, h4103_20121217b.patch, h4103_20121217.patch, h4103_20121218.patch
> In our first snapshot implementation, snapshot creation runs in O(N) and occupies O(N)
memory space, where N = # files + # directories + # symlinks in the snapshot.  The advantages
of the implementation are that there is no additional cost for the modifications after snapshots
are created, and it leads to a simple implementation.
> In this JIRA, we optimize snapshot creation to O(1) although it introduces additional
cost in the modifications after snapshots are created.  Note that the INode is given as an
assumption, otherwise, there is a non-constant cost to find the INode.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message