hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-9984) FileSystem#globStatus and FileSystem#listStatus should resolve symlinks by default
Date Mon, 30 Sep 2013 23:53:24 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-9984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Chris Nauroth updated HADOOP-9984:
----------------------------------

    Attachment: HADOOP-9984.011.patch

Hi, [~cmccabe].  I was still seeing failures on Mac/FreeBSD.  The problem was due to hard-coding
of {{`link'}} in the stat escape sequence, where the subsequent logic depends on exactly 7
characters length difference compared to the source path.

I'm attaching version 11 of the patch with a small change to make this insert the value of
the source {{Path#toString}}.  Tests are passing on Mac for me with this version of the patch.
 For reference, here is the incremental diff of version 10 to version 11:

{code}
[chris@Chriss-MacBook-Pro:ttys004] hadoop-common                                         
               (git:trunk)
> diff ~/patch/HADOOP-9984.010.patch ~/patch/HADOOP-9984.011.patch 
1098c1098
< index c2ec63c..580a509 100644
---
> index c2ec63c..df9460c 100644
1101c1101
< @@ -99,10 +99,10 @@ FileStatus getFileStatusForTesting() {
---
> @@ -99,10 +99,13 @@ FileStatus getFileStatusForTesting() {
1106c1106,1108
< +          "stat", derefFlag + "c", "%s,%F,%Y,%X,%a,%U,%G,%n,%N", path.toString() };
---
> +          "stat", derefFlag + "c",
> +          "%s,%F,%Y,%X,%a,%U,%G,%n,%N",
> +          path.toString() };
1110c1112,1113
< +          "stat", derefFlag + "f", "%z,%HT,%m,%a,%Op,%Su,%Sg,%N,`link' -> `%Y'",
---
> +          "stat", derefFlag + "f",
> +          "%z,%HT,%m,%a,%Op,%Su,%Sg,%N,`" + path.toString() + "' -> `%Y'",
1114c1117
< @@ -145,21 +145,32 @@ protected void parseExecResult(BufferedReader lines) throws IOException
{
---
> @@ -145,21 +148,32 @@ protected void parseExecResult(BufferedReader lines) throws IOException
{
{code}


> FileSystem#globStatus and FileSystem#listStatus should resolve symlinks by default
> ----------------------------------------------------------------------------------
>
>                 Key: HADOOP-9984
>                 URL: https://issues.apache.org/jira/browse/HADOOP-9984
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs
>    Affects Versions: 2.1.0-beta
>            Reporter: Colin Patrick McCabe
>            Assignee: Colin Patrick McCabe
>            Priority: Blocker
>         Attachments: HADOOP-9984.001.patch, HADOOP-9984.003.patch, HADOOP-9984.005.patch,
HADOOP-9984.007.patch, HADOOP-9984.009.patch, HADOOP-9984.010.patch, HADOOP-9984.011.patch
>
>
> During the process of adding symlink support to FileSystem, we realized that many existing
HDFS clients would be broken by listStatus and globStatus returning symlinks.  One example
is applications that assume that !FileStatus#isFile implies that the inode is a directory.
 As we discussed in HADOOP-9972 and HADOOP-9912, we should default these APIs to returning
resolved paths.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message