hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "john lilley (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-8353) FileSystem.resolvePath() fails for Windows UNC paths
Date Fri, 08 May 2015 13:35:59 GMT
john lilley created HDFS-8353:

             Summary: FileSystem.resolvePath() fails for Windows UNC paths
                 Key: HDFS-8353
                 URL: https://issues.apache.org/jira/browse/HDFS-8353
             Project: Hadoop HDFS
          Issue Type: Bug
          Components: HDFS
    Affects Versions: 2.6.0, 2.4.0
         Environment: Windows 8, x64
Java 1.7
            Reporter: john lilley

FileSystem.resolvePath() fails with Windows UNC path. This has a knock-on effect with Parquet
file access in local filesystem, because Parquet has no open-using-stream API that we've been
able to find. Take this simple test: 

public class Scratch {
  public static void main(String[] args) {
    // Note that this path must exist
    java.net.URI uriWithAuth = java.net.URI.create("file://host/share/file");
    try {
      FileSystem fs = FileSystem.get(uriWithAuth, new Configuration());
      fs.resolvePath(new Path(uriWithAuth));
    } catch (Exception ex) {

The resolvePath() call will fail in FileSystem.checkPath():
  protected void checkPath(Path path) {
    URI uri = path.toUri();
    String thatScheme = uri.getScheme();
    if (thatScheme == null)                // fs is relative
    URI thisUri = getCanonicalUri();
    String thisScheme = thisUri.getScheme();
    //authority and scheme are not case sensitive
    if (thisScheme.equalsIgnoreCase(thatScheme)) {// schemes match
      String thisAuthority = thisUri.getAuthority();
      String thatAuthority = uri.getAuthority();
      if (thatAuthority == null &&                // path's authority is null
          thisAuthority != null) {                // fs has an authority
        URI defaultUri = getDefaultUri(getConf());
        if (thisScheme.equalsIgnoreCase(defaultUri.getScheme())) {
          uri = defaultUri; // schemes match, so use this uri instead
        } else {
          uri = null; // can't determine auth of the path
      if (uri != null) {
        // canonicalize uri before comparing with this fs
        uri = canonicalizeUri(uri);
        thatAuthority = uri.getAuthority();
        if (thisAuthority == thatAuthority ||       // authorities match
            (thisAuthority != null &&
    throw new IllegalArgumentException("Wrong FS: "+path+
                                       ", expected: "+this.getUri());

The problem is that thisAuthority is null, and thatAuthority gets "host". There is no logic
for dealing with that case. In fact, this method seems broken in several ways. There are at
least two problems: 
-- For UNC paths like file://host/share/... , the authority does not need to match, at least
for Windows UNC paths. All of these paths are the same file system: "file:///F:/folder/file",
"file://host1/share/file", "file://host2/share/file". 
-- The test thisAuthority == thatAuthority violates Java 101. It should be thisAuthority.equals(thatAuthority)

-- hostnames are case-independent, so I think that the authority comparison should also be
case-insensitive, at least for UNC paths. 
-- I don't see any attempt to resolve hostnames to IP addresses, but that may simply be beyond
the scope of this method.

This message was sent by Atlassian JIRA

View raw message