apex-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Devendra Tagare <devend...@datatorrent.com>
Subject Re: kryo Serealization Exception
Date Fri, 19 Aug 2016 21:53:41 GMT
Hi,

You can set the Locality of the parser and the writer to Container local.

This will ensure that Generic Record from the parser does not get
serialized between containers.

Thanks,
Dev

On Fri, Aug 19, 2016 at 2:21 PM, Mukkamula, Suryavamshivardhan (CWM-NR) <
suryavamshivardhan.mukkamula@rbc.com> wrote:

> Hi,
>
> Can you please help resolve the below issue?
>
> In our project we are using ‘org.apache.avro.generic.GenericRecord’ as
> Tuple writing to a parquet file and we are using avro schema for each
> record. We are getting the below exception, I suppose GenericRecord does
> not have no-arg constructor, and looking for some ideas to solve this
> problem.
>
> ######### Exception ##################################
>
> 2016-08-19 16:29:12,845 [5/silverFileOut:AvroFileOutputOperator] ERROR
> codec.Def
> aultStatefulStreamCodec fromDataStatePair - Catastrophic Error: Execution
> halted
> due to Kryo exception!
> com.esotericsoftware.kryo.KryoException: Class cannot be created (missing
> no-arg
> constructor): org.apache.avro.generic.GenericData$Record
>         at com.esotericsoftware.kryo.Kryo$DefaultInstantiatorStrategy.
> newInstant
> iatorOf(Kryo.java:1228)
>         at com.esotericsoftware.kryo.Kryo.newInstantiator(Kryo.java:1049)
>         at com.esotericsoftware.kryo.Kryo.newInstance(Kryo.java:1058)
>         at com.esotericsoftware.kryo.serializers.FieldSerializer.
> create(FieldSer
> ializer.java:547)
>         at com.esotericsoftware.kryo.serializers.FieldSerializer.
> read(FieldSeria
> lizer.java:523)
>         at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.
> java:761)
>         at com.datatorrent.stram.codec.DefaultStatefulStreamCodec.
> fromDataStateP
> air(DefaultStatefulStreamCodec.java:99)
>         at com.datatorrent.stram.stream.BufferServerSubscriber$
> BufferReservoir.p
> rocessPayload(BufferServerSubscriber.java:364)
>         at com.datatorrent.stram.stream.BufferServerSubscriber$
> BufferReservoir.s
> weep(BufferServerSubscriber.java:316)
>         at com.datatorrent.stram.engine.GenericNode.run(GenericNode.
> java:252)
>         at com.datatorrent.stram.engine.StreamingContainer$2.run(
> StreamingContai
> ner.java:1382)
> 2016-08-19 16:30:09,336 [main] INFO  stram.StreamingContainerManager
> updateCheck
>
> Regards,
> Surya Vamshi
>
>
> _______________________________________________________________________
>
> If you received this email in error, please advise the sender (by return
> email or otherwise) immediately. You have consented to receive the attached
> electronically at the above-noted email address; please retain a copy of
> this confirmation for future reference.
>
> Si vous recevez ce courriel par erreur, veuillez en aviser l'expéditeur
> immédiatement, par retour de courriel ou par un autre moyen. Vous avez
> accepté de recevoir le(s) document(s) ci-joint(s) par voie électronique à
> l'adresse courriel indiquée ci-dessus; veuillez conserver une copie de
> cette confirmation pour les fins de reference future.
>
>

Mime
View raw message