hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tom White (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-571) Path should use URI syntax
Date Tue, 05 Dec 2006 21:38:23 GMT
    [ http://issues.apache.org/jira/browse/HADOOP-571?page=comments#action_12455763 ] 
Tom White commented on HADOOP-571:

While working on HADOOP-574 I came across what looks like a small bug in uri3.patch.

Here's a failing testcase (add to TestPath#testDots):

    assertEquals(new Path("/foo/bar", ".").toString(), "/foo/bar");

The actual value is "/foo/bar/" (note trailing slash, and the irony of HADOOP-778!).

I think it is easy to fix. Change the Path(Path parent, Path child)  constructor to use the
private initialize method as follows:
      URI resolvedUri = parentUri.resolve(child.uri);
      initialize(resolvedUri.getScheme(), resolvedUri.getAuthority(), normalizePath(resolvedUri.getPath()));


> Path should use URI syntax
> --------------------------
>                 Key: HADOOP-571
>                 URL: http://issues.apache.org/jira/browse/HADOOP-571
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: fs
>            Reporter: Doug Cutting
>         Assigned To: Doug Cutting
>             Fix For: 0.10.0
>         Attachments: uri.patch, uri2.patch, uri3.patch
> The following changes are proposed:
> 1. Add a factory/registry of FileSystem implementations.  Given a protocol, hostname
and port, it should be possible to get a FileSystem implementation.
> 2. Path's constructor should accept URI-formatted strings & a configuration.
> 3. A new Path method should be added: FileSystem.getFileSystem().  This returns the filesystem
named in the path or the default configured filesystem.
> 4. Most methods which currently take FileSystem and Path parameters can be changed to
take only Path.
> 5. Many FileSystem operations (create, open, delete, list, etc.) can become convenience
methods on Path.
> 6. A URLStreamHandler can be defined in terms of the FileSystem API, so that URLs for
any protocol with a registered FileSystem implementation can be accessed with a java.net.URL,
permitting FileSystem implementations to be used on the classpath, etc.
> It is tempting to try to replace Path with java.net.URL, but URL's methods are insufficient
for mapreduce.  We require directory listings, random access, location hints, etc., which
are not supported by existing URLStreamHandler implementations.  But we can expose all FileSystem
implementations for access with java.net.URL.
> (From a brainstorm with Owen.)

This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message