hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-9984) FileSystem#globStatus and FileSystem#listStatus should resolve symlinks by default
Date Wed, 02 Oct 2013 17:20:44 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-9984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13784174#comment-13784174
] 

Chris Nauroth commented on HADOOP-9984:
---------------------------------------

My test run on Windows looks good.

bq. If a backwards-incompatible change is unavoidable, we should do it before Hadoop 2 goes
GA (general availability). That means doing it now.

My understanding is that a breaking change will be done in 2.3.0 for HADOOP-9972, regardless
of what happens in this patch.  Is that not the case?  Do we expect to implement those new
APIs fully in the base class without requiring anything new of subclasses?

bq. Why is it easier to fix the out-of-tree filesytems later rather than sooner?

My concern isn't based on the difficulty of the change.  I agree with you that this is easy.
 My concern is based on timing.  We'd commit this, and then there would be some delay before
downstream projects notice that they need to react.  This is a bit tight as downstream projects
are trying to wrap up their own releases dependent on ours.  Delaying the interface change
to 2.3.0 increases time to react (but brings its own set of problems).

bq. Why tolerate all the major negative consequences you admit will happen?

I'm looking at it as a trade-off.  Both approaches have pros and cons, and I still can't decide.
 We need to move quickly though, so I promise to reply again later today.

Thanks for your thoughts on this, Colin.

> FileSystem#globStatus and FileSystem#listStatus should resolve symlinks by default
> ----------------------------------------------------------------------------------
>
>                 Key: HADOOP-9984
>                 URL: https://issues.apache.org/jira/browse/HADOOP-9984
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs
>    Affects Versions: 2.1.0-beta
>            Reporter: Colin Patrick McCabe
>            Assignee: Colin Patrick McCabe
>            Priority: Blocker
>         Attachments: HADOOP-9984.001.patch, HADOOP-9984.003.patch, HADOOP-9984.005.patch,
HADOOP-9984.007.patch, HADOOP-9984.009.patch, HADOOP-9984.010.patch, HADOOP-9984.011.patch,
HADOOP-9984.012.patch
>
>
> During the process of adding symlink support to FileSystem, we realized that many existing
HDFS clients would be broken by listStatus and globStatus returning symlinks.  One example
is applications that assume that !FileStatus#isFile implies that the inode is a directory.
 As we discussed in HADOOP-9972 and HADOOP-9912, we should default these APIs to returning
resolved paths.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message