hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-960) DistributedFileSystem#makeQualified port inconsistency
Date Tue, 09 Feb 2010 17:37:29 GMT

    [ https://issues.apache.org/jira/browse/HDFS-960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12831550#action_12831550

Doug Cutting commented on HDFS-960:

> It breaks programs that assumes DistributedFileSystem#listStatus paths always have the
same form

The notion of a default port implies that a given file may have two, non-equal, unambiguous
URIs, one that specifies the port and one that does not.  A given file may have other non-equal
URIs too.  For example, one may use a hostname, a second an IP address, and a third an HFTP

However, like HTTP URLS, whenever possible we try to elide the default HDFS port from URIs.
 We could alternately try to always insert them, but eliding seemed more user-friendly, and,
regardless, both versions are unambiguous URIs for a single file.

One improvement might be to change Path#equals() to use a registry of default ports per scheme,
so that the Paths could be equal in this case.  Alternately we could abandon the idea of a
default port.

> DistributedFileSystem#makeQualified port inconsistency
> ------------------------------------------------------
>                 Key: HDFS-960
>                 URL: https://issues.apache.org/jira/browse/HDFS-960
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: hdfs client
>    Affects Versions: 0.20.1, 0.20.2, 0.21.0, 0.22.0
>            Reporter: Eli Collins
> DistributedFileSystem#makeQualified returns paths of form hdfs://host:port/path if the
port of the given path does not match the default NN port, and paths of form hdfs://host/path
if it does. Is this inentional? It breaks programs that assumes DistributedFileSystem#listStatus
paths always have the same form. I think the output of listStatus should not depend on whether
the default port is being used.  Here's the relevant code:
> {code}
>   /** Normalize paths that explicitly specify the default port. */
>   public Path makeQualified(Path path) {
>     URI thisUri = this.getUri();
>     URI thatUri = path.toUri();
>     String thatAuthority = thatUri.getAuthority();
>     if (thatUri.getScheme() != null
>         && thatUri.getScheme().equalsIgnoreCase(thisUri.getScheme())
>         && thatUri.getPort() == NameNode.DEFAULT_PORT
>         && thisUri.getPort() == -1
>         && thatAuthority.substring(0,thatAuthority.indexOf(":"))
>         .equalsIgnoreCase(thisUri.getAuthority())) {
>       path = new Path(thisUri.getScheme(), thisUri.getAuthority(),
>                       thatUri.getPath());
>     }
>     return super.makeQualified(path);
>   }
> {code}

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message