hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-1568) NameNode Schema for HttpFileSystem
Date Mon, 09 Jul 2007 20:08:04 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-1568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12511242
] 

Doug Cutting commented on HADOOP-1568:
--------------------------------------

> It will kill performance to require the copy planner to do a http head for each file

It should be significantly faster than the actual copy, no?  The copy will need to stat each
file at open, which is the same number of namenode requests.  So, yes, it might significantly
affect copy performance, but it shouldn't dominate.  Isn't that acceptable for a compatibility
tool?  Again, if we want to provide optimal performance, then we'll need to expose more of
the internals (like block locations) and then the tool will be more fragile.  If that's required,
then perhaps we should instead consider making the HDFS client and servers all support multiple
protocol versions.

> NameNode Schema for HttpFileSystem
> ----------------------------------
>
>                 Key: HADOOP-1568
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1568
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: fs
>            Reporter: Chris Douglas
>            Assignee: Chris Douglas
>
> This issue will track the design and implementation of (the first pass of) a servlet
on the namenode for querying its filesystem via HTTP. The proposed syntax for queries and
responses is as follows.
> *Query*
> {noformat}GET http://<nn>:<port>/ls.jsp[<?option>[&option]*] HTTP/1.1{noformat}
> Where _option_ may be any of the following query parameters:
> _path_ : String (default: '/')
> _recursive_ : boolean (default: false)
> _filter_ : String (default: none)
> *Response*
> The response will be returned as an XML document in the following format:
> {noformat}
> <listing path="..." recursive="(yes|no)" filter="..."
>          time="yyyy-MM-dd hh:mm:ss UTC" version="...">
>   <directory path="..."/>
>   <file path="..." modified="yyyy-MM-dd hh:mm:ss" blocksize="..."
>         replication="..." size="..."
>         dnurl="http://dn:port/streamFile?..."/>
> </listing>
> {noformat}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message