avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ey-chih chow (JIRA)" <j...@apache.org>
Subject [jira] Resolved: (AVRO-780) union handling in ReflectDatumWriter
Date Fri, 18 Mar 2011 18:24:29 GMT

     [ https://issues.apache.org/jira/browse/AVRO-780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

ey-chih chow resolved AVRO-780.
-------------------------------

      Resolution: Fixed
    Hadoop Flags: [Reviewed]

> union handling in ReflectDatumWriter
> ------------------------------------
>
>                 Key: AVRO-780
>                 URL: https://issues.apache.org/jira/browse/AVRO-780
>             Project: Avro
>          Issue Type: Bug
>          Components: java
>    Affects Versions: 1.5.0
>         Environment: Mac with VMWare running Linux training-vm 2.6.28-19-server #61-Ubuntu
>            Reporter: ey-chih chow
>            Assignee: Doug Cutting
>             Fix For: 1.5.1
>
>         Attachments: AVRO-780.patch, AVRO-780.patch
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> Our avdl schema definition for the record DeviceRow has a field:
> union {array<DynamicColumn4Games>, null} Games__;
> When we migrated our MR jobs from 1.4.0 to 1.5.0, we got following messages:
> ===================================================================================================
> 11/03/10 11:31:02 INFO mapred.TaskInProgress: Error from attempt_20110310113041953_0001_m_000000_0:
java.lang.NullPointerException: in com.ngmoco.hbase.DeviceRow in union null of union in field
Games__ of com.ngmoco.hbase.DeviceRow
>     at org.apache.avro.reflect.ReflectDatumWriter.write(ReflectDatumWriter.java:104)
>     at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:57)
>     at org.apache.avro.mapred.AvroSerialization$AvroWrapperSerializer.serialize(AvroSerialization.java:131)
>     at org.apache.avro.mapred.AvroSerialization$AvroWrapperSerializer.serialize(AvroSerialization.java:114)
>     at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:900)
>     at org.apache.hadoop.mapred.MapTask$OldOutputCollector.collect(MapTask.java:466)
>     at org.apache.avro.mapred.HadoopMapper$MapCollector.collect(HadoopMapper.java:69)
>     at com.ngmoco.ngpipes.sourcing.NgActivityGatheringMapper.map(NgActivityGatheringMapper.java:91)
>     at com.ngmoco.ngpipes.sourcing.NgActivityGatheringMapper.map(NgActivityGatheringMapper.java:1)
>     at org.apache.avro.mapred.HadoopMapper.map(HadoopMapper.java:80)
>     at org.apache.avro.mapred.HadoopMapper.map(HadoopMapper.java:34)
>     at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
>     at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
>     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
>     at org.apache.hadoop.mapred.Child.main(Child.java:170)
> Caused by: java.lang.NullPointerException: in union null of union in field Games__ of
com.ngmoco.hbase.DeviceRow
>     at org.apache.avro.generic.GenericDatumWriter.npe(GenericDatumWriter.java:92)
>     at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:86)
>     at org.apache.avro.reflect.ReflectDatumWriter.write(ReflectDatumWriter.java:102)
>     ... 14 more
> ===================================================================================================
> After we change definition of the field Games__ to:
> union {null, array<DynamicColumn4Games>} Games__; 
> The system stop complaining.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message