hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Benoit Perroud (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-7344) globStatus doesn't grok groupings with a slash
Date Tue, 29 May 2012 14:13:23 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-7344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13284836#comment-13284836

Benoit Perroud commented on HADOOP-7344:

I fall into this issue, and my (ugly) workaround is the following : 

Create a public class in the package {{org.apache.hadoop.fs}} to be able to access the package
restricted class {{GlobExpander}}

package org.apache.hadoop.fs;

public class PublicGlobExpander extends GlobExpander {

And have a {{FileSystemHelper}} class that contains at least the following function (basically
expanding the path and calling {{FS.globStatus}} on every components found) : 

    public static FileStatus[] globStatus(FileSystem fs, final Path pathPattern) throws IOException

        final String filename = pathPattern.toUri().getPath();

        final List<String> filePatterns = PublicGlobExpander.expand(filename);

        final List<FileStatus> statuses = new LinkedList<FileStatus>();

        for (final String filePattern : filePatterns) {
            final FileStatus[] tmpStatuses = fs.globStatus(new Path(filePattern));
            if (tmpStatuses == null) {
            for (final FileStatus status : tmpStatuses) {
        return statuses.toArray(new FileStatus[statuses.size()]);

Really looking forward to see the a {{if (files == null) continue;}} at line 1277 !
> globStatus doesn't grok groupings with a slash
> ----------------------------------------------
>                 Key: HADOOP-7344
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7344
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs
>    Affects Versions: 0.23.0
>            Reporter: Daryn Sharp
> If a glob contains a grouping with a single item that contains a slash, ex. "{a/b}",
then globStatus throws {{"Illegal file pattern: Unclosed group near index 2"}} -- regardless
of whether the path exists.  However, if the glob set contains more than one item, ex. "{a/b,c}",
then it throws a {{NullPointerException}} from {{FileSystem.java:1277}}.
> {code}
> 1276: FileStatus[] files = globStatusInternal(new Path(filePattern), filter);
> 1277: for (FileStatus file : files) {
> 1278:   results.add(file);
> 1279: }
> {code}
> The method {{globStatusInternal}} can return null, so the iterator fails with the NPE.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message