crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ryan Blue (JIRA)" <j...@apache.org>
Subject [jira] [Created] (CRUNCH-314) Shuffle and Bundle configuration AvroMode settings clobber one another
Date Sat, 21 Dec 2013 01:23:07 GMT
Ryan Blue created CRUNCH-314:
--------------------------------

             Summary: Shuffle and Bundle configuration AvroMode settings clobber one another
                 Key: CRUNCH-314
                 URL: https://issues.apache.org/jira/browse/CRUNCH-314
             Project: Crunch
          Issue Type: Bug
          Components: Core
    Affects Versions: 0.9.0
            Reporter: Ryan Blue
            Assignee: Josh Wills
            Priority: Blocker
             Fix For: 0.10.0


{{AvroMode}} uses the same configuration property for both shuffle and bundle configuration.
If the bundle's AvroMode for a map/reduce round doesn't match the shuffle's AvroMode, then
the bundle wins. In some cases, this prevents the shuffle from being able to write data, e.g.,
because GENERIC is used instead of REFLECT.

A work-around if to set the strict mode to delegate reader and writer construction to the
more permissive one:
{code:java}
AvroMode.GENERIC.override(AvroMode.REFLECT);
{code}



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

Mime
View raw message