hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-1563) Create FileSystem implementation to read HDFS data via http
Date Wed, 11 Jul 2007 18:49:04 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-1563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12511823
] 

Doug Cutting commented on HADOOP-1563:
--------------------------------------

I talked with Owen about this, and what he wants is more like a 'tar' format for the FileSystem
API, something that preserves standard properties, without being specific to the FileSystem
implementation.  The URI for this should be something like hftp://host:port/a/b/c, since,
while HTTP will be used as the transport, this will not be a FileSystem for arbitrary HTTP
urls.  Finally, we agreed that the FileSystem API should be altered, so that listStatus()
is the primary method, replacing both listPaths() and getStatus().  Whether or not my HttpFileSystem
(included above) is in fact ever used, that patch also has some cleanups to the FileSystem
API that should be committed.

> Create FileSystem implementation to read HDFS data via http
> -----------------------------------------------------------
>
>                 Key: HADOOP-1563
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1563
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: fs
>    Affects Versions: 0.14.0
>            Reporter: Owen O'Malley
>            Assignee: Chris Douglas
>         Attachments: httpfs.patch, httpfs2.patch
>
>
> There should be a FileSystem implementation that can read from a Namenode's http interface.
This would have a couple of useful abilities:
>   1. Copy using distcp between different versions of HDFS.
>   2. Use map/reduce inputs from a different version of HDFS. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message