chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Yang (JIRA)" <j...@apache.org>
Subject [jira] Commented: (CHUKWA-452) ChukwaArchive byte[] needs to be wrapped in DataByteArray
Date Tue, 09 Feb 2010 16:27:27 GMT

    [ https://issues.apache.org/jira/browse/CHUKWA-452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12831523#action_12831523
] 

Eric Yang commented on CHUKWA-452:
----------------------------------

This actually changes Chukwa Archive format to be pig specific.  ChukwaArchive should be extensible
to become archiver for pig data format.  It would be better that to leave Chukwa Archive as
it is and create a new archiver class to extend on top of this one.  What do you guys think?

> ChukwaArchive byte[] needs to be wrapped in DataByteArray
> ---------------------------------------------------------
>
>                 Key: CHUKWA-452
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-452
>             Project: Hadoop Chukwa
>          Issue Type: Bug
>    Affects Versions: 0.4.0
>            Reporter: Gerrit Jansen van Vuuren
>             Fix For: 0.4.0
>
>         Attachments: CHUKWA-452.patch
>
>
> I've been trying to read the data inserted into /chukwa/finalArchives using the ChukwaArchive
 Loader in pig. 
> On trying to STORE only the data field using BinaryStore I get:
> FAILED	
> java.lang.ClassCastException: [B cannot be cast to org.apache.pig.data.DataByteArray
> 	at org.apache.pig.builtin.BinaryStorage.putNext(BinaryStorage.java:128)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:200)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:174)
> 	at org.apache.hadoop.mapred.MapTask$DirectMapOutputCollector.collect(MapTask.java:642)
> 	at org.apache.hadoop.mapred.MapTask$OldOutputCollector.collect(MapTask.java:466)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapOnly$Map.collect(PigMapOnly.java:70)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:255)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:244)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapOnly$Map.map(PigMapOnly.java:65)
> 	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
> 	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
> 	at org.apache.hadoop.mapred.Child.main(Child.java:170)
> Looking at the ChukwaArchive class I saw that the data:byte[] is set to the Tuple without
wrapping to DataByteArray.
> The patch I'm submitting applies this change by wrapping the data:byte[] with the DataByteArray
class.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message