avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Russell Jurney <russell.jur...@gmail.com>
Subject Re: Simple AvroStorage LOAD and STORE with Avro 1.6.0
Date Mon, 09 Jan 2012 20:47:38 GMT
I could only make AvroStorage work with Avro 1.4.1.

Russell Jurney
twitter.com/rjurney
russell.jurney@gmail.com
datasyndrome.com

On Jan 9, 2012, at 1:16 AM, Andrew Kenworthy <adwkenworthy@yahoo.com> wrote:

Hallo,

When I run a simple pig script to LOAD and STORE avro data, I get:-

java.lang.ClassCastException: org.apache.pig.data.BinSedesTuple cannot be
cast to org.apache.avro.generic.IndexedRecord

Script:

REGISTER /tmp/avro-1.6.0.jar;
--REGISTER  /tmp/avro-1.5.4.jar
--REGISTER  /tmp/avro-1.4.1.jar;

REGISTER /tmp/piggybank-0.9.1.jar;
REGISTER /tmp/json-simple-1.1.jar;
REGISTER /tmp/jackson-core-asl-1.8.4.jar;
REGISTER /tmp/jackson-mapper-asl-1.8.4.jar;

avroData=LOAD '$DATA_INPUTDIR' USING
org.apache.pig.piggybank.storage.avro.AvroStorage();

dataSubset = FOREACH avroData GENERATE myField1, myField2;
describe  dataSubset;
-----------------------------------------------
-- shows:
-- dataSubset : { myField1: int, myField2: int}
-----------------------------------------------
STORE dataSubset INTO '$OUTPUTDIR' USING
org.apache.pig.piggybank.storage.avro.AvroStorage();

If I use the 1.5.4 jar I get the same error, but the script works with the
1.4.1 version. If I just write one field, then it works with 1.6.0.

I see there's been a related issue fixed here:

https://issues.apache.org/jira/browse/PIG-2202
https://issues.apache.org/jira/browse/PIG-2195

Can anyone confirm that this or similar works with avro 1.6.0, and/or point
me in the right direction concering where the problem may lie?

Many thanks,

Andrew

Mime
View raw message