hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tom White (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-3064) Exception with file globbing closures
Date Thu, 27 Mar 2008 11:35:25 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-3064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Tom White updated HADOOP-3064:
------------------------------

    Attachment: inputPath4.patch

> I added some testcases and ended up with rewriting all my code.

Reminds me of Fred Brooks: "Plan to throw one away; you will anyhow." The changes look good,
and it still fixes my original problem.

The new unit tests had the arguments to assertEquals the wrong way round (it's expected value
then actual value), so I've created another patch to fix that.

> Exception with file globbing closures
> -------------------------------------
>
>                 Key: HADOOP-3064
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3064
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.16.1
>            Reporter: Tom White
>            Assignee: Hairong Kuang
>             Fix For: 0.17.0
>
>         Attachments: inputPath.patch, inputPath1.patch, inputPath2.patch, inputPath3.patch,
inputPath4.patch
>
>
> Using file globbing to select various input paths, like so:
> conf.setInputPath(new Path("mr/input/glob/2008/02/{02,08}"));
> gives an exception:
> Exception in thread "main" java.io.IOException: Illegal file pattern:
> Expecting set closure character or end of range, or } for glob {02 at
> 3
>        at org.apache.hadoop.fs.FileSystem$GlobFilter.error(FileSystem.java:1023)
>        at org.apache.hadoop.fs.FileSystem$GlobFilter.setRegex(FileSystem.java:1008)
>        at org.apache.hadoop.fs.FileSystem$GlobFilter.<init>(FileSystem.java:926)
>        at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:826)
>        at org.apache.hadoop.fs.FileSystem.globPaths(FileSystem.java:873)
>        at org.apache.hadoop.mapred.FileInputFormat.validateInput(FileInputFormat.java:131)
>        at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:541)
>        at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:809)
> The code for JobConf.getInputPaths tokenizes using
> a comma as the delimiter, producing two paths
> "mr/input/glob/2008/02/{02" and "08}".

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message