hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nathan Roberts (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2316) webhdfs: a complete FileSystem implementation for accessing HDFS over HTTP
Date Fri, 21 Oct 2011 15:10:32 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13132732#comment-13132732
] 

Nathan Roberts commented on HDFS-2316:
--------------------------------------

Hi Nicholas, some quick comments from first read:
* "<namenode>:<port>" and "http://<host>:<port>" seem to be used interchangeably.
We should be consistent where possible.
* Why doesn't "curl -i -L "http://<host>:<port>/webhdfs/<path>" just work?
Do we really need to specify op=OPEN for this very simple, common case?
* I believe "http://<datanode>:<path>" should be "http://<datanode>:<port>"
in append.
* Need format of responses spelled out.
* It would be nice if we could document the possible error responses as well.
* Since a single datanode will be performing the write of a potentially large file, does that
mean that file will have an entire copy on that node (due to block placement strategies)?
That doesn't seem desirable..
* Is a SHORT sufficient for buffersize?
* Do we need a renewlease? How will very slow writers be handled?
* Once I have file block locations, can I go directly to those datanodes to retrieve rather
than using content_range and always following a redirect?
* Do we need flush/sync?
                
> webhdfs: a complete FileSystem implementation for accessing HDFS over HTTP
> --------------------------------------------------------------------------
>
>                 Key: HDFS-2316
>                 URL: https://issues.apache.org/jira/browse/HDFS-2316
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>            Reporter: Tsz Wo (Nicholas), SZE
>            Assignee: Tsz Wo (Nicholas), SZE
>         Attachments: WebHdfsAPI20111020.pdf
>
>
> We current have hftp for accessing HDFS over HTTP.  However, hftp is a read-only FileSystem
and does not provide "write" accesses.
> In HDFS-2284, we propose to have webhdfs for providing a complete FileSystem implementation
for accessing HDFS over HTTP.  The is the umbrella JIRA for the tasks.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message