hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sanjay Radia (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2802) Support for RW/RO snapshots in HDFS
Date Mon, 19 Nov 2012 06:02:59 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13500036#comment-13500036

Sanjay Radia commented on HDFS-2802:

Minutes of Meetup on Snapshots for HDFS
Date: 8 Nov 2012, Location: Hortonworks Offices
Attendance: approx 25 to 30 in person and about 7 calls into webex.
Format: Suresh presented slides (attached in this Jira HDFS-2802) and a discussion followed
along with the slides.

*Main areas of discussions*
# CLI - most felt that we shouldn't use the fs command, introduce a new command instead.
# Scheduling snapshots - most folks felt it was best to leave it outside
# File attributes (permissions, replication factor etc)
** Most folks felt that we should preserve the original attributes in the snapshots, however
no access time to be recorded for snapshots.
** # replicas is max of current and the snapshots
# Quotas
** Everyone agreed that  snapshot metadata should count into the regular snapshot quotas.
** Most felt that max snapshot count per snapshottable directory is not needed. A system level
snapshot count however may be necessary as system limit.
# Length of file open-for-writing: most agreed with the proposal to update length at hearbeats
and for applications that care about a specific length to do a SuperFlush/SuperSync.
# HBase and snapshots: HBase snapshots will be able to use the proprosed solution.
# Exclude list - most (everyone?) disliked the idea of a "Snapshot exclude directory list"
# Restores
   An interesting messy example was raised: A subtree was renamed to outside of snapshoted
subtree and then that snapshot is restored. What happens to the subtree that is  outside -
does it get deleted?
  Again this would be discussed further in Jiras
# Should we introduce the notion of volumes? This would incorporate 
**  a) snapshot able-dir, 
**  b) where quotas are specified,
**  c) not allow renames across volumes
It was noted that c) would break apps. 
Allen W and Sanjay were in favor, Most were not in favor of the rename restriction. This topic
to be discussed further in a Jira.

> Support for RW/RO snapshots in HDFS
> -----------------------------------
>                 Key: HDFS-2802
>                 URL: https://issues.apache.org/jira/browse/HDFS-2802
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: data-node, name-node
>            Reporter: Hari Mankude
>            Assignee: Hari Mankude
>         Attachments: HDFS-2802.20121101.patch, HDFS-2802-meeting-minutes-121101.txt,
HDFSSnapshotsDesign.pdf, snap.patch, snapshot-design.pdf, snapshot-design.tex, snapshot-one-pager.pdf,
Snapshots20121018.pdf, Snapshots20121030.pdf, Snapshots.pdf
> Snapshots are point in time images of parts of the filesystem or the entire filesystem.
Snapshots can be a read-only or a read-write point in time copy of the filesystem. There are
several use cases for snapshots in HDFS. I will post a detailed write-up soon with with more

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message