hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Junping Du (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-8959) Provide an iterator-based API for listing all the snapshottable directories
Date Fri, 06 Jan 2017 21:54:59 GMT

     [ https://issues.apache.org/jira/browse/HDFS-8959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Junping Du updated HDFS-8959:
-----------------------------
    Target Version/s:   (was: 2.8.0)

> Provide an iterator-based API for listing all the snapshottable directories
> ---------------------------------------------------------------------------
>
>                 Key: HDFS-8959
>                 URL: https://issues.apache.org/jira/browse/HDFS-8959
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: hdfs-client
>            Reporter: Rakesh R
>            Assignee: Rakesh R
>         Attachments: HDFS-8959-00.patch, HDFS-8959-01.patch, HDFS-8959-02.patch
>
>
> Presently {{DistributedFileSystem#getSnapshottableDirListing()}} is sending all the {{SnapshottableDirectoryStatus[]}}
array to the clients. Now the client should have enough space to hold it in memory. There
could be chance that the client JVMs running out of memory because of this. Also, some time
back there was a [comment|https://issues.apache.org/jira/browse/HDFS-8643?focusedCommentId=14658800&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14658800]
about RPC packet limitation and a large number of snapshot list can again cause issues.
> I believe iterator based {{DistributedFileSystem#listSnapshottableDirs()}} API would
be a good addition!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message