crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Wills (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CRUNCH-371) AvroPathPerKeyTarget doesn't work with empty input
Date Mon, 31 Mar 2014 14:14:15 GMT

    [ https://issues.apache.org/jira/browse/CRUNCH-371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13955200#comment-13955200
] 

Josh Wills commented on CRUNCH-371:
-----------------------------------

+1 from me. FWIW, I think it's fine to commit these small fixes immediately w/a review afterwards.

> AvroPathPerKeyTarget doesn't work with empty input
> --------------------------------------------------
>
>                 Key: CRUNCH-371
>                 URL: https://issues.apache.org/jira/browse/CRUNCH-371
>             Project: Crunch
>          Issue Type: Bug
>            Reporter: Gabriel Reid
>         Attachments: CRUNCH-371.patch
>
>
> The AvroPathPerKeyTarget doesn't work properly when there is nothing to be output (for
example, if all records have been filtered out somehow). This situation results in an exception
like the following on HDFS on hadoop-1 
> {code}
> java.io.IOException: java.lang.IllegalArgumentException: Reducer output
> name 'out0' cannot be parsed
> {code}
> and an FileNotFoundException on hadoop-2.
> In both cases, there should be no exception, and no output should be written. More background
on this in the following mailing list discussion:
> http://mail-archives.apache.org/mod_mbox/crunch-user/201403.mbox/%3cCACijwiagOJsKVsDq1_x6tbyFhWEZeCyNMV-jpmwjzhQ8CCSF-g@mail.gmail.com%3e



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message