Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Fri, 6 Jan 2017 02:14:58 +0000 (UTC)
From: "Manoj Govindassamy (JIRA)" <jira@apache.org>
To: common-issues@hadoop.apache.org
Message-ID: <JIRA.13027093.1481316192000.660459.1483668898371@Atlassian.JIRA>
In-Reply-To: <JIRA.13027093.1481316192000@Atlassian.JIRA>
References: <JIRA.13027093.1481316192000@Atlassian.JIRA> <JIRA.13027093.1481316192356@arcas>
Subject: [jira] [Commented] (HADOOP-13885) Implement getLinkTarget for
 ViewFileSystem
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Fri, 06 Jan 2017 02:15:00 -0000


    [ https://issues.apache.org/jira/browse/HADOOP-13885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15803245#comment-15803245 ] 

Manoj Govindassamy commented on HADOOP-13885:
---------------------------------------------

The goal is to make ViewFileSystem support getLinkTarget() to resolve the target filesystem paths, so that users of this file system can resolve their paths based on viewfs mounts. 

1. Now coming to whether the resolved path has to be returned as is OR make the resolved path qualified it on top of viewfs mount path, ViewFileSystem already does this for "ls", "du", getHomeDirectory(), etc., Yes, they could further be links to another file system or relative one, but all the _path qualification_ does is it *replaces* the target filesystem prefix path with the viewfs mount path (as defined in the config)

{noformat}

# hdfs dfs -ls  /
-r-xr-xr-x   - manoj staff          0 2017-01-05 17:32 /nn0
-r-xr-xr-x   - manoj staff          0 2017-01-05 17:32 /nn1
-r-xr-xr-x   - manoj staff          0 2017-01-05 17:32 /nn2
-r-xr-xr-x   - manoj staff          0 2017-01-05 17:32 /nn3

# hdfs dfs -ls -R /nn0/
drwxr-xr-x   - manoj supergroup          0 2017-01-05 17:23 /nn0/user
drwxr-xr-x   - manoj supergroup          0 2017-01-05 17:23 /nn0/user/manoj

{noformat}

Say, we have the following symbolic link in the target file system.
hdfs://localhost:52217/tmp/TestViewFileSystemHdfs/debug.link  ==> hdfs://localhost:52217/tmp/TestViewFileSystemHdfs/debug.log

Now, if the same target file system "hdfs://localhost:52217/tmp/TestViewFileSystemHdfs/" is mounted on "/nn1/", then

With resolved path qualified to viewfs mount path:
/nn1/debug.link => /nn1/debug.log

With resolved path non qualified:
/nn1/debug.link => hdfs://localhost:52217/tmp/TestViewFileSystemHdfs/debug.log

I was following the {{getFileStatus()}} to model, to qualify returned paths. Let me know if this raw target filesystem resolved path is ok and no more viewfs qualification is needed.

2. We can potentially get FileNotFoundExceptions from 2 places. 
* Case 1:  From ViewFileSystem internal InodeTree resolve which traverses the internal mount tree leading to the final InodeLink which links the target filesystem, when the given path doesn't map to a proper mountpoint configured.
* Case 2: From the TargetFileSystem {{getLinkTarget()}} when the given path is missing.

So, we want to return NotInMountPointException for the Case 1 above. Where as for the Case 2, we want to return the FNFE as is. 

In the attached patch v01, try block surrounds both Case 1 and Case 2 together. I will fix this in the next patch revision so that only Case 1 returns NotInMountPointException and Case 2 returns FNFE.

[~andrew.wang], let me know your views on (1) and I will do necessary changes and upload the next patch revision. Thanks for the review. 


> Implement getLinkTarget for ViewFileSystem
> ------------------------------------------
>
>                 Key: HADOOP-13885
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13885
>             Project: Hadoop Common
>          Issue Type: Task
>          Components: viewfs
>    Affects Versions: 3.0.0-alpha1
>            Reporter: Manoj Govindassamy
>            Assignee: Manoj Govindassamy
>         Attachments: HADOOP-13885.01.patch
>
>
> ViewFileSystem doesn't override FileSystem#getLinkTarget(). So, when view filesystem is used to resolve the symbolic links, the default FileSystem implementation throws UnsupportedOperationException.
> The proposal is to define getLinkTarget() for ViewFileSystem and invoke the target FileSystem for resolving the symbolic links. Path thus returned is preferred to be a viewfs qualified path, so that it can be used again on the ViewFileSystem handle. 


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org