hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jagane Sundar (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2802) Support for RW/RO snapshots in HDFS
Date Sat, 03 Nov 2012 20:09:12 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13490083#comment-13490083

Jagane Sundar commented on HDFS-2802:

First off, I think this is a critical feature for HDFS, and I would like to thank all of you
folks working on this. In order to make snapshots usable to end users, there is more work
to be done in the application software above HDFS and in the management software. So, getting
this feature into HDFS expeditiously is very important.

Having read the design by Suresh, Nicholas, Sanjay et. al, and the (counter?) proposal by
Aaron, Colin, Todd and Eli - it occurs to me that both designs share a great deal, and that
there are really clever ideas in both proposals. Acutely aware of the political sensitivity
of this JIRA, I want to say only one thing, hopefully, without offending any of the parties:
Considering the amount of work the layers above have to do, it may be in the best interest
of the Hadoop community to get this feature into HDFS quickly. Given that Suresh, Nicholas
et. al. seem to have working code, and seem much further ahead, it would serve the HDFS community
well if all of us could put our weight behind that proposal.

And now, on to my real feedback:
1. Having the ability to snapshot sub-directories of the file system, and to have different
snapshot policies for each 'snapshottable' directory is absolutely essential.
2. In your design document, you state that the administrator needs to configure a directory
as being snapshottable. It would be useful to allow users to be able to set directories in
their own home dir as 'snapshottable'. For example, the administrator provisions a new user
xyzzy and his home directory /user/xyzzy. xyzzy may then want to create a dir /user/xyzzy/hbase
and want to set it as snapshottable. xyzzy should not have to call the administrator to make
his /home/xyzzy/hbase directory snapshottable. I believe that this was discussed in the meetup
of last week in terms of an ACL. I think this is important.

Finally, a question:
I still don't understand the 'single client consistency' versus 'multiple client consistency'
issues discussed above. Regardless of how many HDFS client connections are active on the namenode,
a snapshot will capture the HDFS state at that PIT, right?
> Support for RW/RO snapshots in HDFS
> -----------------------------------
>                 Key: HDFS-2802
>                 URL: https://issues.apache.org/jira/browse/HDFS-2802
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: data-node, name-node
>            Reporter: Hari Mankude
>            Assignee: Hari Mankude
>         Attachments: HDFS-2802.20121101.patch, HDFS-2802-meeting-minutes-121101.txt,
HDFSSnapshotsDesign.pdf, snap.patch, snapshot-design.pdf, snapshot-design.tex, snapshot-one-pager.pdf,
Snapshots20121018.pdf, Snapshots20121030.pdf
> Snapshots are point in time images of parts of the filesystem or the entire filesystem.
Snapshots can be a read-only or a read-write point in time copy of the filesystem. There are
several use cases for snapshots in HDFS. I will post a detailed write-up soon with with more

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message