hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nathan Roberts (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2316) webhdfs: a complete FileSystem implementation for accessing HDFS over HTTP
Date Fri, 21 Oct 2011 15:10:32 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13132732#comment-13132732

Nathan Roberts commented on HDFS-2316:

Hi Nicholas, some quick comments from first read:
* "<namenode>:<port>" and "http://<host>:<port>" seem to be used interchangeably.
We should be consistent where possible.
* Why doesn't "curl -i -L "http://<host>:<port>/webhdfs/<path>" just work?
Do we really need to specify op=OPEN for this very simple, common case?
* I believe "http://<datanode>:<path>" should be "http://<datanode>:<port>"
in append.
* Need format of responses spelled out.
* It would be nice if we could document the possible error responses as well.
* Since a single datanode will be performing the write of a potentially large file, does that
mean that file will have an entire copy on that node (due to block placement strategies)?
That doesn't seem desirable..
* Is a SHORT sufficient for buffersize?
* Do we need a renewlease? How will very slow writers be handled?
* Once I have file block locations, can I go directly to those datanodes to retrieve rather
than using content_range and always following a redirect?
* Do we need flush/sync?
> webhdfs: a complete FileSystem implementation for accessing HDFS over HTTP
> --------------------------------------------------------------------------
>                 Key: HDFS-2316
>                 URL: https://issues.apache.org/jira/browse/HDFS-2316
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>            Reporter: Tsz Wo (Nicholas), SZE
>            Assignee: Tsz Wo (Nicholas), SZE
>         Attachments: WebHdfsAPI20111020.pdf
> We current have hftp for accessing HDFS over HTTP.  However, hftp is a read-only FileSystem
and does not provide "write" accesses.
> In HDFS-2284, we propose to have webhdfs for providing a complete FileSystem implementation
for accessing HDFS over HTTP.  The is the umbrella JIRA for the tasks.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message