avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Kenworthy <adwkenwor...@yahoo.com>
Subject Simple AvroStorage LOAD and STORE with Avro 1.6.0
Date Mon, 09 Jan 2012 09:15:53 GMT
Hallo,

When I run a simple pig script to LOAD and STORE avro data, I get:-

java.lang.ClassCastException: org.apache.pig.data.BinSedesTuple cannot be cast to org.apache.avro.generic.IndexedRecord


Script:

REGISTER /tmp/avro-1.6.0.jar;
--REGISTER /tmp/avro-1.5.4.jar
--REGISTER /tmp/avro-1.4.1.jar;

REGISTER /tmp/piggybank-0.9.1.jar;
REGISTER /tmp/json-simple-1.1.jar;
REGISTER /tmp/jackson-core-asl-1.8.4.jar;
REGISTER /tmp/jackson-mapper-asl-1.8.4.jar;

avroData=LOAD '$DATA_INPUTDIR' USING org.apache.pig.piggybank.storage.avro.AvroStorage();

dataSubset = FOREACH avroData GENERATE myField1, myField2;
describe dataSubset;
-----------------------------------------------
-- shows: 
-- dataSubset : {myField1: int,myField2: int}
----------------------------------------------- 
STORE dataSubset INTO '$OUTPUTDIR' USING org.apache.pig.piggybank.storage.avro.AvroStorage();

If I use the 1.5.4 jar I get the same error, but the script works with the 1.4.1 version.
If I just write one field, then it works with 1.6.0.

I see there's been a related issue fixed here:

https://issues.apache.org/jira/browse/PIG-2202 
https://issues.apache.org/jira/browse/PIG-2195 

Can anyone confirm that this or similar works with avro 1.6.0, and/or point me in the right
direction concering where the problem may lie?

Many thanks,

Andrew
Mime
View raw message