hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rakesh R (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-8959) Provide an iterator-based API for listing all the snapshottable directories
Date Wed, 26 Aug 2015 13:26:45 GMT

     [ https://issues.apache.org/jira/browse/HDFS-8959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Rakesh R updated HDFS-8959:
---------------------------
    Issue Type: Improvement  (was: Bug)

> Provide an iterator-based API for listing all the snapshottable directories
> ---------------------------------------------------------------------------
>
>                 Key: HDFS-8959
>                 URL: https://issues.apache.org/jira/browse/HDFS-8959
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Rakesh R
>            Assignee: Rakesh R
>         Attachments: HDFS-8959-00.patch
>
>
> Presently {{DistributedFileSystem#getSnapshottableDirListing()}} is sending all the {{SnapshottableDirectoryStatus[]}}
array to the clients. Now the client should have enough space to hold it in memory. There
could be chance that the client JVMs running out of memory because of this. Also, some time
back there was a [comment|https://issues.apache.org/jira/browse/HDFS-8643?focusedCommentId=14658800&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14658800]
about RPC packet limitation and a large number of snapshot list can again cause issues.
> I believe iterator based {{DistributedFileSystem#listSnapshottableDirs()}} API would
be a good addition!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message