hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rakesh R (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8643) Add snapshot names list to SnapshottableDirectoryStatus
Date Thu, 06 Aug 2015 11:55:04 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14659892#comment-14659892

Rakesh R commented on HDFS-8643:

bq. Instead of directly adding name list to SnapshottableDirectoryStatus, I think it may be
better to directly enhance the lsSnapshottableDir cmd by making multiple RPC calls (i.e.,
do "ls xxx/.snapshot" for each snapshottable dir). Note that we have RPC packet limitation
and a large number of snapshot list or some long snapshot names can cause issue.

[~jingzhao], I agree with you. *lsSnapshottableDir* cmd is using the {{DistributedFileSystem#getSnapshottableDirListing()}}
API which is returning {{SnapshottableDirectoryStatus[]}} array, the entire snapshot list.
I'm thinking to improve DistributedFileSystem API, that would be helpful to the users also.
How about exposing {{BatchedRemoteIterator<Long, SnapshottableDirectoryStatus>}} to
return batch of snapshots. Also, the same can be used in lsSnapshottableDir cmd implementation.

Following are the initial thoughts to support the batch operation,
# Expose new API in {{DistributedFileSystem#listSnapshottableDirs()}}
# Add new API in {{ClientProtocol#listSnapshottableDirs(prevId)}}
# Expose configuration for batch size {{dfs.namenode.list.snapshottable.dirs.num.responses}},
defaulting to 100
# Modify {{SnapshotManager#snapshottables}} datastructure to {{java.util.TreeMap}}, this will
help to make the navigation easy through {{TreeMap.tailMap(prevId, false)}} function.

> Add snapshot names list to SnapshottableDirectoryStatus
> -------------------------------------------------------
>                 Key: HDFS-8643
>                 URL: https://issues.apache.org/jira/browse/HDFS-8643
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Rakesh R
>            Assignee: Rakesh R
>         Attachments: HDFS-8643-00.patch, HDFS-8643-01.patch
> The idea of this jira to enhance {{SnapshottableDirectoryStatus}} by adding {{snapshotNames}}
attribute into it, presently it has the {{snapshotNumber}}. IMHO this would help the users
to get the list of snapshot names created. Also, the snapshot names can be used while renaming
or deleting the snapshots.
> {code}
> org.apache.hadoop.hdfs.protocol.SnapshottableDirectoryStatus.java
>   /**
>    * @return Snapshot names for the directory.
>    */
>   public List <String> getSnapshotNames() {
>     return snapshotNames;
>   }
> {code}

This message was sent by Atlassian JIRA

View raw message