Return-Path: X-Original-To: apmail-avro-user-archive@www.apache.org Delivered-To: apmail-avro-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id D9F359955 for ; Mon, 9 Jan 2012 20:48:14 +0000 (UTC) Received: (qmail 44411 invoked by uid 500); 9 Jan 2012 20:48:14 -0000 Delivered-To: apmail-avro-user-archive@avro.apache.org Received: (qmail 44306 invoked by uid 500); 9 Jan 2012 20:48:13 -0000 Mailing-List: contact user-help@avro.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@avro.apache.org Delivered-To: mailing list user@avro.apache.org Received: (qmail 44270 invoked by uid 99); 9 Jan 2012 20:48:12 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 09 Jan 2012 20:48:12 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of russell.jurney@gmail.com designates 74.125.82.49 as permitted sender) Received: from [74.125.82.49] (HELO mail-ww0-f49.google.com) (74.125.82.49) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 09 Jan 2012 20:48:06 +0000 Received: by wgbdt13 with SMTP id dt13so1716748wgb.30 for ; Mon, 09 Jan 2012 12:47:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=references:from:in-reply-to:mime-version:date:message-id:subject:to :cc:content-type; bh=rQBcsCiCVMGPuivazN0OBSZaNQzmJvWH3To4KBLLqo4=; b=gzoZMzHmpiCapKjzN8zEwrgjMEX/mueaj/CK4cXvnaxXRmLMWrLw3cS9keLIUcsUpK qDtkCbsz479eiTo7RKfnwWbZ/TyQws9qHLvR44hPM9+qSusv61BV2UUaspcKDPHyG5Qf l1TBL5kjq+ENLYAEa2YCB8fw+MmNgchJl0uFc= Received: by 10.180.19.130 with SMTP id f2mr7469982wie.12.1326142064592; Mon, 09 Jan 2012 12:47:44 -0800 (PST) References: <1326100553.71422.YahooMailNeo@web120006.mail.ne1.yahoo.com> From: Russell Jurney In-Reply-To: <1326100553.71422.YahooMailNeo@web120006.mail.ne1.yahoo.com> Mime-Version: 1.0 (1.0) Date: Mon, 9 Jan 2012 12:47:38 -0800 Message-ID: <2977167651302216230@unknownmsgid> Subject: Re: Simple AvroStorage LOAD and STORE with Avro 1.6.0 To: "user@avro.apache.org" Cc: "user@pig.apache.org" Content-Type: multipart/alternative; boundary=bcaec53f37edae884004b61e8230 X-Virus-Checked: Checked by ClamAV on apache.org --bcaec53f37edae884004b61e8230 Content-Type: text/plain; charset=ISO-8859-1 I could only make AvroStorage work with Avro 1.4.1. Russell Jurney twitter.com/rjurney russell.jurney@gmail.com datasyndrome.com On Jan 9, 2012, at 1:16 AM, Andrew Kenworthy wrote: Hallo, When I run a simple pig script to LOAD and STORE avro data, I get:- java.lang.ClassCastException: org.apache.pig.data.BinSedesTuple cannot be cast to org.apache.avro.generic.IndexedRecord Script: REGISTER /tmp/avro-1.6.0.jar; --REGISTER /tmp/avro-1.5.4.jar --REGISTER /tmp/avro-1.4.1.jar; REGISTER /tmp/piggybank-0.9.1.jar; REGISTER /tmp/json-simple-1.1.jar; REGISTER /tmp/jackson-core-asl-1.8.4.jar; REGISTER /tmp/jackson-mapper-asl-1.8.4.jar; avroData=LOAD '$DATA_INPUTDIR' USING org.apache.pig.piggybank.storage.avro.AvroStorage(); dataSubset = FOREACH avroData GENERATE myField1, myField2; describe dataSubset; ----------------------------------------------- -- shows: -- dataSubset : { myField1: int, myField2: int} ----------------------------------------------- STORE dataSubset INTO '$OUTPUTDIR' USING org.apache.pig.piggybank.storage.avro.AvroStorage(); If I use the 1.5.4 jar I get the same error, but the script works with the 1.4.1 version. If I just write one field, then it works with 1.6.0. I see there's been a related issue fixed here: https://issues.apache.org/jira/browse/PIG-2202 https://issues.apache.org/jira/browse/PIG-2195 Can anyone confirm that this or similar works with avro 1.6.0, and/or point me in the right direction concering where the problem may lie? Many thanks, Andrew --bcaec53f37edae884004b61e8230 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
I could only make AvroSto= rage work with Avro 1.4.1.

Russell Jurney

On Jan 9, 2012, at 1:16 AM, Andrew Kenworthy <adwkenworthy@yahoo.com> wrote:

<= div>
Hallo,
<= div style=3D"font-family:verdana,helvetica,sans-serif;font-size:10pt">
When I run a simple pig script to LOAD and STORE avro data, I get:-

java.lang.Clas= sCastException: org.apache.pig.data.BinSedesTuple cannot be cast to org.apa= che.avro.generic.IndexedRecord

Script:

REGI= STER /tmp/avro-1.6.0.jar;
--REGISTER=A0 /tmp/avro-1.5.4.jar
--REGISTER=A0 /tmp/avro-1.4.1.jar;

REGISTER /tmp/json-simple-1.1.jar;
REGISTER /tmp/jackson-co= re-asl-1.8.4.jar;
REGISTER /tmp/jackson-mapper-asl-1.8.4.jar;

avroData=3D= LOAD '$DATA_INPUTDIR' USING org.apache.pig.piggybank.storage.avro.A= vroStorage();
dataSubset = =3D FOREACH=A0avroData=A0GENERATE myField1, myField2;
describe=A0 dataSubset;
= -----------------------------------------------
-- shows:=A0
--=A0dataSubset=A0: { myField1: int, myField2: int}
-----------------------------------------------=A0
STORE=A0dataSubset=A0INTO '$O= UTPUTDIR' USING org.apache.pig.piggybank.storage.avro.AvroStorage();

If I use the 1.5.4 jar I get th= e same error, but the script works with the 1.4.1 version. If I just write = one field, then it works with 1.6.0.

I see there's been a relate= d issue fixed here:


Can anyone confirm that this or= similar works with avro 1.6.0, and/or point me in the right direction conc= ering where the problem may lie?

Many thanks,
<= font face=3D"verdana, helvetica, sans-serif">
Andrew


--bcaec53f37edae884004b61e8230--