hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shashikant Banerjee (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-13102) Implement SnapshotSkipList class to store Multi level DirectoryDiffs
Date Fri, 02 Feb 2018 18:30:00 GMT
Shashikant Banerjee created HDFS-13102:

             Summary: Implement SnapshotSkipList class to store Multi level DirectoryDiffs
                 Key: HDFS-13102
                 URL: https://issues.apache.org/jira/browse/HDFS-13102
             Project: Hadoop HDFS
          Issue Type: Improvement
            Reporter: Shashikant Banerjee
            Assignee: Shashikant Banerjee

HDFS-11225 explains an issue where deletion of older snapshots can take a very long time in
case the no of snapshot diffs is quite large for directories. For any directory under a snapshot,
to construct the children list , it needs to combine all the diffs from that particular snapshot
to the last snapshotDiff record and reverseApply to the current children list of the directory
on live fs. This can take  a significant time if the no of snapshot diffs are quite large
and changes per diff is significant.

This Jira proposes to store the Directory diffs in a SnapshotSkip list, where we store multi
level DirectoryDiffs. At each level, the Directory Diff will be cumulative diff of k snapshot

where k is the level of a node in the list. 


This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message