hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xiao Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-10784) Implement WebHdfsFileSystem#listStatusIterator
Date Tue, 30 Aug 2016 04:43:20 GMT

    [ https://issues.apache.org/jira/browse/HDFS-10784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15448015#comment-15448015
] 

Xiao Chen commented on HDFS-10784:
----------------------------------

Thanks for the new patch, Andrew. Looks pretty good to me.
I have a question, and some nits.

Question:
- I see httpfs' documentation says {{HttpFs... And it is inteoperable with the webhdfs REST
HTTP API.}} Does this mean we have to implement this for httpfs too, in order to keep this
interoperability?

Nits (in nitness order, the later the nitter...):
- I think we should add this new API to {{WebHDFS.md}} with an example.
- Maybe we should also test the iterator on empty dir and / or dir with 1 entry.
- Extra line break at the beginning of {{JsonUtilClient#toDirectoryListing}} and {{StartAfterParam}}
class declaration.
- {{WebHdfsFileSystem}}: there're already com.google imports in the later part of the class,
let's put them together.


> Implement WebHdfsFileSystem#listStatusIterator
> ----------------------------------------------
>
>                 Key: HDFS-10784
>                 URL: https://issues.apache.org/jira/browse/HDFS-10784
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: webhdfs
>    Affects Versions: 2.6.4
>            Reporter: Andrew Wang
>            Assignee: Andrew Wang
>         Attachments: HDFS-10784.001.patch, HDFS-10784.002.patch
>
>
> It would be nice to implement the iterative listStatus in WebHDFS so client apps do not
need to buffer the full file list for large directories.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message