incubator-crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Wills (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CRUNCH-131) Input paths containing globs/wildcards are not accepted
Date Thu, 13 Dec 2012 18:12:12 GMT

    [ https://issues.apache.org/jira/browse/CRUNCH-131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13531258#comment-13531258
] 

Josh Wills commented on CRUNCH-131:
-----------------------------------

@Dave thanks for this, I'll take a look at it now.

Re: the current version, I don't have good answers. It's one of those things where I made
a poor design decision early on in the life of the code that is clearly wrong and sort of
irritating, but not so much so that we have been motivated to think it through properly and
fix it vs. working on other stuff that is both clearly wrong and really irritating.
                
> Input paths containing globs/wildcards are not accepted
> -------------------------------------------------------
>
>                 Key: CRUNCH-131
>                 URL: https://issues.apache.org/jira/browse/CRUNCH-131
>             Project: Crunch
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 0.4.0
>            Reporter: Dave Beech
>            Assignee: Josh Wills
>         Attachments: CRUNCH-131.patch
>
>
> Crunch fails to calculate the size of paths containing wildcards - example error below:
> Exception in thread "main" java.lang.IllegalStateException: Input source SeqFile(/my/path/containing/wildcards*/part*)
does not exist!
>  	at org.apache.crunch.impl.mr.collect.InputCollection.getSizeInternal(InputCollection.java:53)
>  	at org.apache.crunch.impl.mr.collect.PCollectionImpl.getSize(PCollectionImpl.java:253)
>  	at org.apache.crunch.impl.mr.collect.DoCollectionImpl.getSizeInternal(DoCollectionImpl.java:43)
>  	at org.apache.crunch.impl.mr.collect.PCollectionImpl.getSize(PCollectionImpl.java:253)
>  	at org.apache.crunch.impl.mr.collect.DoTableImpl.getSizeInternal(DoTableImpl.java:47)
>  	at org.apache.crunch.impl.mr.collect.PGroupedTableImpl.getSizeInternal(PGroupedTableImpl.java:75)
>  	at org.apache.crunch.impl.mr.collect.PCollectionImpl.getSize(PCollectionImpl.java:253)
>  	at org.apache.crunch.impl.mr.collect.PGroupedTableImpl.configureShuffle(PGroupedTableImpl.java:63)
>  	at org.apache.crunch.impl.mr.plan.JobPrototype.build(JobPrototype.java:162)
>  	at org.apache.crunch.impl.mr.plan.JobPrototype.getCrunchJob(JobPrototype.java:114)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message