hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Douglas (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-1568) NameNode Schema for HttpFileSystem
Date Sat, 07 Jul 2007 02:29:04 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-1568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12510841
] 

Chris Douglas commented on HADOOP-1568:
---------------------------------------

Thoughts:

1) It isn't clear how one would format a list of block locations in a reply. One could request
the contents of a consecutive set of blocks by setting query parameters or using a range header
field, but XML provides a more expressive format for that association that might also permit
a consumer to contact multiple datanodes simultaneously, from different machines, etc. We
don't need or support it now, but situations where it could be useful are not difficult to
imagine.
2) Each request for file info is a separate HTTP HEAD request. A consumer needs to parse the
link to acquire interesting information about the file, including its name (as in the patch).
If we hope to avoid assumptions about the link syntax, a HEAD request is necessary for every
file in the listing. Granted, the link syntax is probably pretty stable, but parsing URIs
for file paths doesn't seem like a more future-proof solution than an XML schema.

Further, if we hope to implement (1), we would probably break the current implementation of
(2) or include block location URIs in header fields.

It seems prudent to restrict this servlet to meta-information, i.e. not returning file content
in response to a GET and leave that to StreamFile or some other facility. Some of the ideas
in this and in HADOOP-1563 seem very appropriate for StreamFile, though.

> NameNode Schema for HttpFileSystem
> ----------------------------------
>
>                 Key: HADOOP-1568
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1568
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: fs
>            Reporter: Chris Douglas
>            Assignee: Chris Douglas
>
> This issue will track the design and implementation of (the first pass of) a servlet
on the namenode for querying its filesystem via HTTP. The proposed syntax for queries and
responses is as follows.
> *Query*
> {noformat}GET http://<nn>:<port>/ls.jsp[<?option>[&option]*] HTTP/1.1{noformat}
> Where _option_ may be any of the following query parameters:
> _path_ : String (default: '/')
> _recursive_ : boolean (default: false)
> _filter_ : String (default: none)
> *Response*
> The response will be returned as an XML document in the following format:
> {noformat}
> <listing path="..." recursive="(yes|no)" filter="..."
>          time="yyyy-MM-dd hh:mm:ss UTC" version="...">
>   <directory path="..."/>
>   <file path="..." modified="yyyy-MM-dd hh:mm:ss" blocksize="..."
>         replication="..." size="..."
>         dnurl="http://dn:port/streamFile?..."/>
> </listing>
> {noformat}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message