flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephan Ewen <se...@apache.org>
Subject Re: Issues with Java 8 Lambda Expressions
Date Thu, 13 Nov 2014 10:22:19 GMT
Looks to me like the deserialization is not happening properly, leavinf
some unconsumed bytes...

On Thu, Nov 13, 2014 at 11:17 AM, Ufuk Celebi <uce@apache.org> wrote:

> Just my two cents, but the Exception is thrown by the lower layer
> serializers, which write/read IOReadableWriteable types. The respective
> exception is thrown if a partial record has not been fully deserialized and
> you receive an event (channel close event or so). The corresponding writer
> part is the RecordWriter class.
>
> I guess the problem is at the writer side already. I would have a look what
> is pushed down to the RecordWriter for further debugging.
>
> On Thu, Nov 13, 2014 at 11:06 AM, Timo Walther <flink@twalthr.com> wrote:
>
> > I have implemented your idea of an Unkown type which uses the
> > KryoSerializer. Since I don't have type information, I initialize the the
> > serializer with Object.class. Collection execution works fine but when I
> > execute a simple identity mapper job normally I get the following
> > Exception. Is there a way to get this working?
> >
> > 14/11/13 11:01:04 ERROR operators.DataSinkTask: Error in user code:
> > Channel received an event before completing the current partial record.:
> > DataSink(TextOutputFormat (file:/tmp/org.apache.flink.
> > test.javaApiOperators.TypeHintITCase-result) - UTF-8) (1/1)
> > java.lang.IllegalStateException: Channel received an event before
> > completing the current partial record.
> >     at org.apache.flink.runtime.io.network.channels.InputChannel.
> > readRecord(InputChannel.java:158)
> >     at org.apache.flink.runtime.io.network.gates.InputGate.
> > readRecord(InputGate.java:176)
> >     at org.apache.flink.runtime.io.network.api.MutableRecordReader.next(
> > MutableRecordReader.java:51)
> >     at org.apache.flink.runtime.operators.util.ReaderIterator.
> > next(ReaderIterator.java:53)
> >     at org.apache.flink.runtime.operators.DataSinkTask.invoke(
> > DataSinkTask.java:175)
> >     at org.apache.flink.runtime.execution.RuntimeEnvironment.
> > run(RuntimeEnvironment.java:245)
> >     at java.lang.Thread.run(Thread.java:701)
> > 14/11/13 11:01:04 INFO taskmanager.Task: DataSink(TextOutputFormat
> > (file:/tmp/org.apache.flink.test.javaApiOperators.TypeHintITCase-result)
> > - UTF-8) (1/1) switched to FAILED : java.lang.IllegalStateException:
> > Channel received an event before completing the current partial record.
> >     at org.apache.flink.runtime.io.network.channels.InputChannel.
> > readRecord(InputChannel.java:158)
> >     at org.apache.flink.runtime.io.network.gates.InputGate.
> > readRecord(InputGate.java:176)
> >     at org.apache.flink.runtime.io.network.api.MutableRecordReader.next(
> > MutableRecordReader.java:51)
> >     at org.apache.flink.runtime.operators.util.ReaderIterator.
> > next(ReaderIterator.java:53)
> >     at org.apache.flink.runtime.operators.DataSinkTask.invoke(
> > DataSinkTask.java:175)
> >     at org.apache.flink.runtime.execution.RuntimeEnvironment.
> > run(RuntimeEnvironment.java:245)
> >     at java.lang.Thread.run(Thread.java:701)
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message