hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Allen Wittenauer (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-12996) DataNode Replica Trash
Date Tue, 16 Jan 2018 02:29:03 GMT

    [ https://issues.apache.org/jira/browse/HDFS-12996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16326698#comment-16326698

Allen Wittenauer commented on HDFS-12996:

bq. Also the design looks very similar to Checkpointing/Snapshots.

The fact that this JIRA even exists points that snapshots are/were a failure.  On other file
systems, snapshots are exactly the recovery model for these types of deletes.


Reading through the doc, there are handful of spots where I see the use cases are extremely
limited.  But I'm really left with a basic question:

Why isn't there an option to just have the NN automatically do a snapshot for deletes over
a certain size instead and then automatically delete these snapshots after X amount time?
 Wouldn't that add the protection that is being requested while avoiding the requirement to
restart the NN? 

> DataNode Replica Trash
> ----------------------
>                 Key: HDFS-12996
>                 URL: https://issues.apache.org/jira/browse/HDFS-12996
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>            Reporter: Hanisha Koneru
>            Assignee: Hanisha Koneru
>            Priority: Major
>         Attachments: DataNode_Replica_Trash_Design_Doc.pdf
> DataNode Replica Trash will allow administrators to recover from a recent delete request
that resulted in catastrophic loss of user data. This is achieved by placing all invalidated
blocks in a replica trash on the datanode before completely purging them from the system.
The design doc is attached here.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message