hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron T. Myers (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2802) Support for RW/RO snapshots in HDFS
Date Fri, 02 Nov 2012 19:03:16 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13489659#comment-13489659
] 

Aaron T. Myers commented on HDFS-2802:
--------------------------------------

bq. ** Current design solves single-client consistency but not multiple-client consistency.

This is with respect to multiple distinct DFSClients which are simultaneously writing to open
files at the time of snapshot creation. All of the designs discussed so far discuss single
client ordering of namespace operations/data writes, but not multiple clients. The implication
is that multi client consistency would likely require such far-reaching changes as to be unrealistic.

bq. *** Would admins like to restrict based on users as well as directories for snapshots?

We discussed the possibility of adding a configurable white list of users who were allowed
to create snapshots, and/or an ACL per snapshottable directory of users allowed to create
snapshots of that directory.

bq. ** Current design adds extra overhead for every inode,

The design for storing snapshots in the NN in the proposal I posted would add integer start_snap
and end_snap fields to every INode in the FS in order to track which snapshots a given INode
should be included in. This is probably the biggest difference between the design proposed
by Suresh/Nicholas and the design I've proposed. That proposal adds a "diff" per INodeDirectory
which has been snapshotted to keep track of changes made to that directory across different
snapshots. This has the advantage of saving NN memory space for files/directories which have
never had snapshots created of them.
                
> Support for RW/RO snapshots in HDFS
> -----------------------------------
>
>                 Key: HDFS-2802
>                 URL: https://issues.apache.org/jira/browse/HDFS-2802
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: data-node, name-node
>            Reporter: Hari Mankude
>            Assignee: Hari Mankude
>         Attachments: HDFS-2802.20121101.patch, HDFS-2802-meeting-minutes-121101.txt,
HDFSSnapshotsDesign.pdf, snap.patch, snapshot-design.pdf, snapshot-design.tex, snapshot-one-pager.pdf,
Snapshots20121018.pdf, Snapshots20121030.pdf
>
>
> Snapshots are point in time images of parts of the filesystem or the entire filesystem.
Snapshots can be a read-only or a read-write point in time copy of the filesystem. There are
several use cases for snapshots in HDFS. I will post a detailed write-up soon with with more
information.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message