pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Cheolsoo Park (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (PIG-2856) AvroStorage doesn't load files in the directories when a glob pattern matches both files and directories.
Date Thu, 02 Aug 2012 09:39:02 GMT

     [ https://issues.apache.org/jira/browse/PIG-2856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Cheolsoo Park updated PIG-2856:
-------------------------------

    Status: Patch Available  (was: Open)

Review board:
https://reviews.apache.org/r/6318/
                
> AvroStorage doesn't load files in the directories when a glob pattern matches both files
and directories.
> ---------------------------------------------------------------------------------------------------------
>
>                 Key: PIG-2856
>                 URL: https://issues.apache.org/jira/browse/PIG-2856
>             Project: Pig
>          Issue Type: Bug
>          Components: piggybank
>    Affects Versions: 0.11
>            Reporter: Cheolsoo Park
>            Assignee: Cheolsoo Park
>         Attachments: PIG-2856.patch
>
>
> This is a regression from PIG-2492.
> When a glob pattern such as '*' matches not only files but also directories, AvroStorage
does not load files in the directories. This is a bug in getAllSubDirs() that can be fixed
as follows:
> {code}
> static boolean getAllSubDirs(Path path, Job job, Set<Path> paths)
> ...
> FileStatus[] matchedFiles = fs.globStatus(path, PATH_FILTER);
> ...
> for (FileStatus file : matchedFiles) {
>     if (file.isDir()) {
> -        for (FileStatus sub : fs.listStatus(path)) {
> +        for (FileStatus sub : fs.listStatus(file.getPath())) {
>             getAllSubDirs(sub.getPath(), job, paths);
>         }
>     }
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message