hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-2566) need FileSystem#globStatus method
Date Wed, 16 Jan 2008 23:02:35 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-2566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12559723#action_12559723

Doug Cutting commented on HADOOP-2566:

> Is this what we wanted? I thought we wanted other way around.

I don't think it does that in all cases, but it does still appear to call getStatus() in places.
 I've not yet examined the logic to see if that's easily avoidable or not.  But it's not a
fatal problem at this point.  For this release the important thing is to have globStatus()
as the preferred, non-deprecated method.  Once we remove the status cache, during 0.17 development,
we'll soon find out whether the globStatus() implementation needs more work to perform well
without a cache, and fix that before 0.17 is released.  But that aspect shouldn't block this
for 0.16, since we still have the cache in 0.16.

> need FileSystem#globStatus method
> ---------------------------------
>                 Key: HADOOP-2566
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2566
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: fs
>            Reporter: Doug Cutting
>            Assignee: Hairong Kuang
>             Fix For: 0.16.0
>         Attachments: globStatus.patch, globStatus1.patch
> To remove the cache of FileStatus in DFSPath (HADOOP-2565) without hurting performance,
we must use file enumeration APIs that return FileStatus[] rather than Path[].  Currently
we have FileSystem#globPaths(), but that method should be deprecated and replaced with a FileSystem#globStatus().
> We need to deprecate FileSystem#globPaths() in 0.16 in order to remove the cache in 0.17.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message