hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ryan Blue (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7031) Path#makeQualified copies authority when scheme does not match
Date Tue, 09 Sep 2014 03:33:29 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14126530#comment-14126530

Ryan Blue commented on HDFS-7031:

My application doesn't know what the incoming Path will look like, so it uses {{makeQualified}}
to fill in the details that may be missing. If there is no scheme/authority, then those should
be taken from the default URI (/some/path => hdfs://nn:8020/some/path). If there is only
a relative path, then that's filled in from the working directory Path (path => hdfs://nn:8020/path).
Because the URI argument is a "default", I think that behavior #1 is implied when the schemes
don't match, which fits with my interpretation of the other examples: qualifying a URI doesn't
change what the URI already contains, but fills in what's missing from it being fully qualified.
Behavior #2 would change the original URI's meaning, not just fill in defaults. That said,
I've added a work-around that only passes in the default FS URI if the schemes don't match
so my application is no longer affected.

I think your argument that {{FileSystem#makeQualified}} is a better fit for behavior #2 is
reasonable, but I don't think there is a reason to change the current behavior, which throws
an IllegalArgumentException that the FS is incorrect. That's why I'm using {{Path#makeQualified}},
because it seems to have the behavior I want, to default the values if not specified, in most

> Path#makeQualified copies authority when scheme does not match
> --------------------------------------------------------------
>                 Key: HDFS-7031
>                 URL: https://issues.apache.org/jira/browse/HDFS-7031
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: hdfs-client
>    Affects Versions: 2.3.0
>            Reporter: Ryan Blue
> I have an application that calls {{makeQualified}} that amounts to this:
> {code:java}
> new Path("file:/some/local/path").makeQualified(
>     URI.create("hdfs://nn:8020"), new Path("/"));
> {code}
> This unexpectedly produces {{file://nn:8020/some/local/path}}, using the authority section
from the default URI even though the path that is being qualified doesn't have a scheme that
matches the default URI.
> In {{Path}}, there is a check to see if the default URI should be used:
> {code:java}
>     if (scheme != null &&
>         (authority != null || defaultUri.getAuthority() == null))
>       return path;
> {code}
> I think this should be:
> {code:java}
>     // if the scheme matches and there is no authority, use the default
>     if (scheme != null && scheme.equals(defaultUri.getScheme()) &&
>         (authority != null || defaultUri.getAuthority() == null))
>       return path;
> {code}

This message was sent by Atlassian JIRA

View raw message