hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Manoj Govindassamy (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13885) Implement getLinkTarget for ViewFileSystem
Date Fri, 06 Jan 2017 02:14:58 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15803245#comment-15803245

Manoj Govindassamy commented on HADOOP-13885:

The goal is to make ViewFileSystem support getLinkTarget() to resolve the target filesystem
paths, so that users of this file system can resolve their paths based on viewfs mounts. 

1. Now coming to whether the resolved path has to be returned as is OR make the resolved path
qualified it on top of viewfs mount path, ViewFileSystem already does this for "ls", "du",
getHomeDirectory(), etc., Yes, they could further be links to another file system or relative
one, but all the _path qualification_ does is it *replaces* the target filesystem prefix path
with the viewfs mount path (as defined in the config)


# hdfs dfs -ls  /
-r-xr-xr-x   - manoj staff          0 2017-01-05 17:32 /nn0
-r-xr-xr-x   - manoj staff          0 2017-01-05 17:32 /nn1
-r-xr-xr-x   - manoj staff          0 2017-01-05 17:32 /nn2
-r-xr-xr-x   - manoj staff          0 2017-01-05 17:32 /nn3

# hdfs dfs -ls -R /nn0/
drwxr-xr-x   - manoj supergroup          0 2017-01-05 17:23 /nn0/user
drwxr-xr-x   - manoj supergroup          0 2017-01-05 17:23 /nn0/user/manoj


Say, we have the following symbolic link in the target file system.
hdfs://localhost:52217/tmp/TestViewFileSystemHdfs/debug.link  ==> hdfs://localhost:52217/tmp/TestViewFileSystemHdfs/debug.log

Now, if the same target file system "hdfs://localhost:52217/tmp/TestViewFileSystemHdfs/" is
mounted on "/nn1/", then

With resolved path qualified to viewfs mount path:
/nn1/debug.link => /nn1/debug.log

With resolved path non qualified:
/nn1/debug.link => hdfs://localhost:52217/tmp/TestViewFileSystemHdfs/debug.log

I was following the {{getFileStatus()}} to model, to qualify returned paths. Let me know if
this raw target filesystem resolved path is ok and no more viewfs qualification is needed.

2. We can potentially get FileNotFoundExceptions from 2 places. 
* Case 1:  From ViewFileSystem internal InodeTree resolve which traverses the internal mount
tree leading to the final InodeLink which links the target filesystem, when the given path
doesn't map to a proper mountpoint configured.
* Case 2: From the TargetFileSystem {{getLinkTarget()}} when the given path is missing.

So, we want to return NotInMountPointException for the Case 1 above. Where as for the Case
2, we want to return the FNFE as is. 

In the attached patch v01, try block surrounds both Case 1 and Case 2 together. I will fix
this in the next patch revision so that only Case 1 returns NotInMountPointException and Case
2 returns FNFE.

[~andrew.wang], let me know your views on (1) and I will do necessary changes and upload the
next patch revision. Thanks for the review. 

> Implement getLinkTarget for ViewFileSystem
> ------------------------------------------
>                 Key: HADOOP-13885
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13885
>             Project: Hadoop Common
>          Issue Type: Task
>          Components: viewfs
>    Affects Versions: 3.0.0-alpha1
>            Reporter: Manoj Govindassamy
>            Assignee: Manoj Govindassamy
>         Attachments: HADOOP-13885.01.patch
> ViewFileSystem doesn't override FileSystem#getLinkTarget(). So, when view filesystem
is used to resolve the symbolic links, the default FileSystem implementation throws UnsupportedOperationException.
> The proposal is to define getLinkTarget() for ViewFileSystem and invoke the target FileSystem
for resolving the symbolic links. Path thus returned is preferred to be a viewfs qualified
path, so that it can be used again on the ViewFileSystem handle. 

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message