flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ufuk Celebi <...@apache.org>
Subject Re: Core Memory Error
Date Tue, 10 Nov 2015 19:01:21 GMT
Thanks for reporting this. Are you using any custom data types?

If you can share your code, it would be very helpful in order to debug this.

– Ufuk

On Tuesday, 10 November 2015, Fabian Hueske <fhueske@gmail.com> wrote:

> I agree with Robert. Looks like a bug in Flink.
> Maybe an off-by-one issue (violating index is 32768 and the default memory
> segment size is 32KB).
>
> Which Flink version are you using?
> In case you are using a custom build, can you share the commit ID (is
> reported in the first lines of the JobManager log file)?
>
> Thanks, Fabian
>
> 2015-11-10 18:29 GMT+01:00 Robert Metzger <rmetzger@apache.org
> <javascript:;>>:
>
> > Hi Ali,
> >
> > this could be a bug in Flink.
> > Can you share the code of your program with us to debug the issue?
> >
> > On Tue, Nov 10, 2015 at 6:25 PM, Kashmar, Ali <Ali.Kashmar@emc.com
> <javascript:;>> wrote:
> >
> > > Hello,
> > >
> > > I’m getting this error while running a streaming module on a cluster
> of 3
> > > nodes:
> > >
> > >
> > > java.lang.ArrayIndexOutOfBoundsException: 32768
> > >
> > > at
> org.apache.flink.core.memory.MemorySegment.get(MemorySegment.java:178)
> > >
> > > at
> > >
> >
> org.apache.flink.runtime.io.network.api.serialization.SpillingAdaptiveSpanningRecordDeserializer$NonSpanningWrapper.readByte(SpillingAdaptiveSpanningRecordDeserializer.java:214)
> > >
> > > at
> > >
> >
> org.apache.flink.runtime.io.network.api.serialization.SpillingAdaptiveSpanningRecordDeserializer$NonSpanningWrapper.readUnsignedByte(SpillingAdaptiveSpanningRecordDeserializer.java:219)
> > >
> > > at org.apache.flink.types.StringValue.readString(StringValue.java:764)
> > >
> > > at
> > >
> >
> org.apache.flink.api.common.typeutils.base.StringSerializer.deserialize(StringSerializer.java:68)
> > >
> > > at
> > >
> >
> org.apache.flink.api.common.typeutils.base.StringSerializer.deserialize(StringSerializer.java:73)
> > >
> > > at
> > >
> >
> org.apache.flink.api.common.typeutils.base.StringSerializer.deserialize(StringSerializer.java:28)
> > >
> > > at
> > >
> >
> org.apache.flink.api.java.typeutils.runtime.PojoSerializer.deserialize(PojoSerializer.java:499)
> > >
> > > at
> > >
> >
> org.apache.flink.streaming.runtime.streamrecord.StreamRecordSerializer.deserialize(StreamRecordSerializer.java:102)
> > >
> > > at
> > >
> >
> org.apache.flink.streaming.runtime.streamrecord.StreamRecordSerializer.deserialize(StreamRecordSerializer.java:29)
> > >
> > > at
> > >
> >
> org.apache.flink.runtime.plugable.ReusingDeserializationDelegate.read(ReusingDeserializationDelegate.java:57)
> > >
> > > at
> > >
> >
> org.apache.flink.runtime.io.network.api.serialization.SpillingAdaptiveSpanningRecordDeserializer.getNextRecord(SpillingAdaptiveSpanningRecordDeserializer.java:110)
> > >
> > > at
> > >
> >
> org.apache.flink.streaming.runtime.io.StreamingAbstractRecordReader.getNextRecord(StreamingAbstractRecordReader.java:80)
> > >
> > > at
> > >
> >
> org.apache.flink.streaming.runtime.io.StreamingMutableRecordReader.next(StreamingMutableRecordReader.java:36)
> > >
> > > at
> > >
> >
> org.apache.flink.runtime.operators.util.ReaderIterator.next(ReaderIterator.java:59)
> > >
> > > at
> > >
> >
> org.apache.flink.streaming.runtime.tasks.OneInputStreamTask.readNext(OneInputStreamTask.java:68)
> > >
> > > at
> > >
> >
> org.apache.flink.streaming.runtime.tasks.OneInputStreamTask.invoke(OneInputStreamTask.java:101)
> > >
> > > at org.apache.flink.runtime.taskmanager.Task.run(Task.java:559)
> > >
> > > at java.lang.Thread.run(Thread.java:745)
> > >
> > >
> > > Here’s the configuration for each node:
> > >
> > >
> > > jobmanager.heap.mb: 2048
> > >
> > > taskmanager.heap.mb: 4096
> > >
> > > taskmanager.numberOfTaskSlots: 5
> > >
> > >
> > > I’m not even sure where to start with this one so any help is
> > appreciated.
> > >
> > >
> > > Thanks,
> > >
> > > Ali
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message