hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daryn Sharp (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-9357) Fallback to default authority if not specified in FileContext
Date Fri, 26 Apr 2013 15:04:15 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-9357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13642934#comment-13642934

Daryn Sharp commented on HADOOP-9357:

bq. Daryn. I am missing your comment. The current patch does exactly what your are proposing
- take the authority from the default fs IF the default fs is hdfs.
[~sanjay.radia], unless I'm looking at the wrong patch: if there's no authority, it's immediately
dropping it and relying on it being re-added later, instead of just adding the authority immediately.
 I'm unclear which is the more desirable behavior of this method.

bq. The URI RFC can assert that a notion of default authority is sensible because one can
arrange for a default for each of the schemes.

The default uri is providing the default authority for a scheme.  Given it's not flexible
to support multiple schemes, it is arguable doing the right thing for one specific case.

I agree specifying /path is preferable to hdfs:///path and assuming the defaultFS must be
hdfs, else it breaks.  My big concern is the backwards incompatibility since this has been
long standing behavior in hadoop.  Perhaps a deprecated warning should be emitted in 2.x,
and we can remove it in 3.x?

> Fallback to default authority if not specified in FileContext
> -------------------------------------------------------------
>                 Key: HADOOP-9357
>                 URL: https://issues.apache.org/jira/browse/HADOOP-9357
>             Project: Hadoop Common
>          Issue Type: Bug
>    Affects Versions: 2.0.0-alpha
>            Reporter: Andrew Wang
>            Assignee: Andrew Wang
>            Priority: Minor
>         Attachments: hadoop-9357-1.patch, hadoop-9357-2.patch, hadoop-9357-3.patch, hadoop-9357-testfixup.patch
> Currently, FileContext adheres rather strictly to RFC2396 when it comes to parsing absolute
URIs (URIs with a scheme). If a user asks for a URI like "hdfs:///tmp", FileContext will error
while FileSystem will add the authority of the default FS (e.g. turn it into "hdfs://defaultNN:port/tmp").

> This is technically correct, but FileSystem's behavior is nicer for users and okay based
on 5.2.3 in the RFC, so lets do it in FileContext too:
> {noformat}
> For backwards
> compatibility, an implementation may work around such references
> by removing the scheme if it matches that of the base URI and the
> scheme is known to always use the  syntax.  The parser
> can then continue with the steps below for the remainder of the
> reference components.  Validating parsers should mark such a
> misformed relative reference as an error.
> {noformat}

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message