flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Flavio Pompermaier <pomperma...@okkam.it>
Subject Re: Weird Kryo exception (Unable to find class: java.ttil.HashSet)
Date Fri, 20 May 2016 10:48:59 GMT
Hi Ufuk,
my records could be quite large Pojos (I think some MB).
The only thing I do to configure Kryo is:

 env.registerTypeWithKryoSerializer(DateTime.class,
JodaDateTimeSerializer.class );

Best,
Flavio

On Fri, May 20, 2016 at 12:38 PM, Ufuk Celebi <uce@apache.org> wrote:

> @Stefano: the records are serialized anyway for batch jobs. The
> spilling deserializer is only relevant if single records are very
> large. How large are your records? In any case, I don't expect this to
> be the problem.
>
> @Flavio: The class name "typo" errors (Vdhicle instead of Vehicle and
> ttil instead of util) look like some kind of data corruption and would
> need further investigation. The other failure you reported might be
> related to this. As a starting point, how do you configure the Kryo
> serializer?
>
> On Fri, May 20, 2016 at 10:02 AM, Flavio Pompermaier
> <pompermaier@okkam.it> wrote:
> > Today I've got this other strange error.. Obviously I don't have a
> > VdhicleEvent class, but a VehicleEvent class :(
> >
> > java.lang.RuntimeException: Cannot instantiate class.
> >       at
> >
> org.apache.flink.api.java.typeutils.runtime.PojoSerializer.deserialize(PojoSerializer.java:407)
> >       at
> >
> org.apache.flink.runtime.plugable.NonReusingDeserializationDelegate.read(NonReusingDeserializationDelegate.java:55)
> >       at
> >
> org.apache.flink.runtime.io.network.api.serialization.SpillingAdaptiveSpanningRecordDeserializer.getNextRecord(SpillingAdaptiveSpanningRecordDeserializer.java:124)
> >       at
> >
> org.apache.flink.runtime.io.network.api.reader.AbstractRecordReader.getNextRecord(AbstractRecordReader.java:65)
> >       at
> >
> org.apache.flink.runtime.io.network.api.reader.MutableRecordReader.next(MutableRecordReader.java:34)
> >       at
> >
> org.apache.flink.runtime.operators.util.ReaderIterator.next(ReaderIterator.java:73)
> >       at
> >
> org.apache.flink.runtime.operators.FlatMapDriver.run(FlatMapDriver.java:101)
> >       at
> org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:480)
> >       at
> org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:345)
> >       at org.apache.flink.runtime.taskmanager.Task.run(Task.java:559)
> >       at java.lang.Thread.run(Thread.java:745)
> > Caused by: java.lang.ClassNotFoundException:
> > it.okkam.flink.test.model.pojo.VdhicleEvent
> >       at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> >       at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> >       at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> >       at java.lang.Class.forName0(Native Method)
> >       at java.lang.Class.forName(Class.java:348)
> >       at
> >
> org.apache.flink.api.java.typeutils.runtime.PojoSerializer.deserialize(PojoSerializer.java:405)
> >       ... 10 more
> >
> >
> > Thanks in advance,
> > Flavio
> >
> >
> > On Mon, May 16, 2016 at 10:08 AM, Stefano Bortoli <s.bortoli@gmail.com>
> > wrote:
> >>
> >> Hi Flavio, Till,
> >>
> >> do you think this can be possibly related to the serialization problem
> >> caused by 'the management' of Kryo serializer buffer when spilling on
> disk?
> >> We are definitely going beyond what is managed in memory with this task.
> >>
> >> saluti,
> >> Stefano
> >>
> >> 2016-05-16 9:44 GMT+02:00 Flavio Pompermaier <pompermaier@okkam.it>:
> >>>
> >>> That exception showed just once, but the following happens randomly
> (if I
> >>> re-run the job after stopping and restartign the cluster it doesn't
> show up
> >>> usually):
> >>>
> >>> Caused by: java.io.IOException: Serializer consumed more bytes than the
> >>> record had. This indicates broken serialization. If you are using
> custom
> >>> serialization types (Value or Writable), check their serialization
> methods.
> >>> If you are using a Kryo-serialized type, check the corresponding Kryo
> >>> serializer.
> >>>     at
> >>>
> org.apache.flink.runtime.io.network.api.serialization.SpillingAdaptiveSpanningRecordDeserializer.getNextRecord(SpillingAdaptiveSpanningRecordDeserializer.java:142)
> >>>     at
> >>>
> org.apache.flink.runtime.io.network.api.reader.AbstractRecordReader.getNextRecord(AbstractRecordReader.java:65)
> >>>     at
> >>>
> org.apache.flink.runtime.io.network.api.reader.MutableRecordReader.next(MutableRecordReader.java:34)
> >>>     at
> >>>
> org.apache.flink.runtime.operators.util.ReaderIterator.next(ReaderIterator.java:73)
> >>>     at
> >>>
> org.apache.flink.runtime.operators.FlatMapDriver.run(FlatMapDriver.java:101)
> >>>     at
> >>> org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:480)
> >>>     at
> >>> org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:345)
> >>>     at org.apache.flink.runtime.taskmanager.Task.run(Task.java:559)
> >>>     at java.lang.Thread.run(Thread.java:745)
> >>> Caused by: java.lang.ArrayIndexOutOfBoundsException: -2
> >>>     at java.util.ArrayList.elementData(ArrayList.java:418)
> >>>     at java.util.ArrayList.get(ArrayList.java:431)
> >>>     at
> >>>
> com.esotericsoftware.kryo.util.MapReferenceResolver.getReadObject(MapReferenceResolver.java:42)
> >>>     at
> com.esotericsoftware.kryo.Kryo.readReferenceOrNull(Kryo.java:805)
> >>>     at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:759)
> >>>     at
> >>>
> org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.deserialize(KryoSerializer.java:228)
> >>>     at
> >>>
> org.apache.flink.api.java.typeutils.runtime.PojoSerializer.deserialize(PojoSerializer.java:431)
> >>>     at
> >>>
> org.apache.flink.api.java.typeutils.runtime.TupleSerializer.deserialize(TupleSerializer.java:135)
> >>>     at
> >>>
> org.apache.flink.api.java.typeutils.runtime.TupleSerializer.deserialize(TupleSerializer.java:30)
> >>>     at
> >>>
> org.apache.flink.runtime.plugable.NonReusingDeserializationDelegate.read(NonReusingDeserializationDelegate.java:55)
> >>>     at
> >>>
> org.apache.flink.runtime.io.network.api.serialization.SpillingAdaptiveSpanningRecordDeserializer.getNextRecord(SpillingAdaptiveSpanningRecordDeserializer.java:124)
> >>>
> >>>
> >>> On Sun, May 15, 2016 at 10:58 AM, Flavio Pompermaier
> >>> <pompermaier@okkam.it> wrote:
> >>>>
> >>>> Hi to all,
> >>>> in my last run of a job I received this weird Kryo Exception in one
of
> >>>> the TaskManager...obviously this class in not mentioned anywhere,
> neither in
> >>>> my project nor in flink...
> >>>> Any help is appreciated!
> >>>>
> >>>> Best,
> >>>> Flavio
> >>>>
> >>>> INFO  org.apache.flink.runtime.taskmanager.Task - CHAIN GroupReduce
> >>>> (GroupReduce at createResult(IndexMappingExecutor.java:42)) -> Map
> (Map at
> >>>> main(Jsonizer.java:128)) (4/9) switched to FAILED with exception.
> >>>> java.lang.Exception: The data preparation for task 'CHAIN GroupReduce
> >>>> (GroupReduce at createResult(IndexMappingExecutor.java:42)) -> Map
> (Map at
> >>>> main(Jsonizer.java:128))' , caused an error: Error obtaining the
> sorted
> >>>> input: Thread 'SortMerger spilling thread' terminated due to an
> exception:
> >>>> Unable to find class: java.ttil.HashSet
> >>>>         at
> >>>> org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:456)
> >>>>         at
> >>>>
> org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:345)
> >>>>         at
> org.apache.flink.runtime.taskmanager.Task.run(Task.java:559)
> >>>>         at java.lang.Thread.run(Thread.java:745)
> >>>> Caused by: java.lang.RuntimeException: Error obtaining the sorted
> input:
> >>>> Thread 'SortMerger spilling thread' terminated due to an exception:
> Unable
> >>>> to find class: java.ttil.HashSet
> >>>>         at
> >>>>
> org.apache.flink.runtime.operators.sort.UnilateralSortMerger.getIterator(UnilateralSortMerger.java:619)
> >>>>         at
> >>>>
> org.apache.flink.runtime.operators.BatchTask.getInput(BatchTask.java:1079)
> >>>>         at
> >>>>
> org.apache.flink.runtime.operators.GroupReduceDriver.prepare(GroupReduceDriver.java:94)
> >>>>         at
> >>>> org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:450)
> >>>>         ... 3 more
> >>>> Caused by: java.io.IOException: Thread 'SortMerger spilling thread'
> >>>> terminated due to an exception: Unable to find class:
> java.ttil.HashSet
> >>>>         at
> >>>>
> org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:800)
> >>>> Caused by: com.esotericsoftware.kryo.KryoException: Unable to find
> >>>> class: java.ttil.HashSet
> >>>>         at
> >>>>
> com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:138)
> >>>>         at
> >>>>
> com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:115)
> >>>>         at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:641)
> >>>>         at
> >>>> com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:752)
> >>>>         at
> >>>>
> com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:143)
> >>>>         at
> >>>>
> com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:21)
> >>>>         at
> >>>> com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761)
> >>>>         at
> >>>>
> org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.deserialize(KryoSerializer.java:228)
> >>>>         at
> >>>>
> org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.deserialize(KryoSerializer.java:242)
> >>>>         at
> >>>>
> org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.copy(KryoSerializer.java:252)
> >>>>         at
> >>>>
> org.apache.flink.api.java.typeutils.runtime.PojoSerializer.copy(PojoSerializer.java:556)
> >>>>         at
> >>>>
> org.apache.flink.api.java.typeutils.runtime.TupleSerializerBase.copy(TupleSerializerBase.java:75)
> >>>>         at
> >>>>
> org.apache.flink.runtime.operators.sort.NormalizedKeySorter.writeToOutput(NormalizedKeySorter.java:499)
> >>>>         at
> >>>>
> org.apache.flink.runtime.operators.sort.UnilateralSortMerger$SpillingThread.go(UnilateralSortMerger.java:1344)
> >>>>         at
> >>>>
> org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:796)
> >>>> Caused by: java.lang.ClassNotFoundException: java.ttil.HashSet
> >>>>         at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> >>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> >>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> >>>>         at java.lang.Class.forName0(Native Method)
> >>>>         at java.lang.Class.forName(Class.java:348)
> >>>>         at
> >>>>
> com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:136)
> >>>>         ... 14 more
> >>>>
> >>>
> >>
> >
>

Mime
View raw message