hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Benoit Perroud (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-7344) globStatus doesn't grok groupings with a slash
Date Tue, 29 May 2012 14:13:23 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-7344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13284836#comment-13284836
] 

Benoit Perroud commented on HADOOP-7344:
----------------------------------------

I fall into this issue, and my (ugly) workaround is the following : 

Create a public class in the package {{org.apache.hadoop.fs}} to be able to access the package
restricted class {{GlobExpander}}

{noformat}
package org.apache.hadoop.fs;

public class PublicGlobExpander extends GlobExpander {
}
{noformat}

And have a {{FileSystemHelper}} class that contains at least the following function (basically
expanding the path and calling {{FS.globStatus}} on every components found) : 

{noformat}
    public static FileStatus[] globStatus(FileSystem fs, final Path pathPattern) throws IOException
{

        final String filename = pathPattern.toUri().getPath();

        final List<String> filePatterns = PublicGlobExpander.expand(filename);

        final List<FileStatus> statuses = new LinkedList<FileStatus>();

        for (final String filePattern : filePatterns) {
            final FileStatus[] tmpStatuses = fs.globStatus(new Path(filePattern));
            if (tmpStatuses == null) {
                continue;
            }
            for (final FileStatus status : tmpStatuses) {
                statuses.add(status);
            }
        }
        return statuses.toArray(new FileStatus[statuses.size()]);
    }
{noformat}

Really looking forward to see the a {{if (files == null) continue;}} at line 1277 !
                
> globStatus doesn't grok groupings with a slash
> ----------------------------------------------
>
>                 Key: HADOOP-7344
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7344
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs
>    Affects Versions: 0.23.0
>            Reporter: Daryn Sharp
>
> If a glob contains a grouping with a single item that contains a slash, ex. "{a/b}",
then globStatus throws {{"Illegal file pattern: Unclosed group near index 2"}} -- regardless
of whether the path exists.  However, if the glob set contains more than one item, ex. "{a/b,c}",
then it throws a {{NullPointerException}} from {{FileSystem.java:1277}}.
> {code}
> 1276: FileStatus[] files = globStatusInternal(new Path(filePattern), filter);
> 1277: for (FileStatus file : files) {
> 1278:   results.add(file);
> 1279: }
> {code}
> The method {{globStatusInternal}} can return null, so the iterator fails with the NPE.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message