hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xiao Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-10784) Implement WebHdfsFileSystem#listStatusIterator
Date Tue, 30 Aug 2016 04:43:20 GMT

    [ https://issues.apache.org/jira/browse/HDFS-10784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15448015#comment-15448015

Xiao Chen commented on HDFS-10784:

Thanks for the new patch, Andrew. Looks pretty good to me.
I have a question, and some nits.

- I see httpfs' documentation says {{HttpFs... And it is inteoperable with the webhdfs REST
HTTP API.}} Does this mean we have to implement this for httpfs too, in order to keep this

Nits (in nitness order, the later the nitter...):
- I think we should add this new API to {{WebHDFS.md}} with an example.
- Maybe we should also test the iterator on empty dir and / or dir with 1 entry.
- Extra line break at the beginning of {{JsonUtilClient#toDirectoryListing}} and {{StartAfterParam}}
class declaration.
- {{WebHdfsFileSystem}}: there're already com.google imports in the later part of the class,
let's put them together.

> Implement WebHdfsFileSystem#listStatusIterator
> ----------------------------------------------
>                 Key: HDFS-10784
>                 URL: https://issues.apache.org/jira/browse/HDFS-10784
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: webhdfs
>    Affects Versions: 2.6.4
>            Reporter: Andrew Wang
>            Assignee: Andrew Wang
>         Attachments: HDFS-10784.001.patch, HDFS-10784.002.patch
> It would be nice to implement the iterative listStatus in WebHDFS so client apps do not
need to buffer the full file list for large directories.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message