flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aljoscha Krettek <aljos...@apache.org>
Subject Re: TopSpeedWindowing - in error: Could not forward element to next operator
Date Fri, 21 Oct 2016 15:46:34 GMT
Hi,
the problem is that EvictingWindowOperator uses StreamRecordSerializer to
serialise the contents of the windows. This does not serialise timestamps
so when the objects are deserialised from RocksDB they all have
Long.MIN_VALUE as timestamp. The evictor in the program therefore always
evicts all elements and the window function always sees an empty iterable.

I have a fix for this in a recent PR:
https://github.com/apache/flink/pull/2656

Cheers,
Aljoscha

On Fri, 21 Oct 2016 at 16:57 Ovidiu Cristian Marcu <
ovidiu.cristian.marcu@huawei.com> wrote:

> Hi
>
> I have the latest source code Master.
> I think the refactoring you were doing on State Backend gives this issue,
> unless the example is not supposed to work
> With rocksdb.
> At some point collected records's values are  null, giving the error..
>
> What do you think?
>
> Best,
> Ovidiu
>
> -----Original Message-----
> From: Till Rohrmann [mailto:trohrmann@apache.org]
> Sent: Friday, October 21, 2016 2:09 PM
> To: dev@flink.apache.org
> Subject: Re: TopSpeedWindowing - in error: Could not forward element to
> next operator
>
> Hi Ovidiu,
>
> which version of Flink are you using?
>
> Cheers,
> Till
>
> On Thu, Oct 20, 2016 at 6:38 PM, Ovidiu Cristian Marcu <
> ovidiu.cristian.marcu@huawei.com> wrote:
>
> > Could you check the following issue on master?
> >
> > When running this example org.apache.flink.streaming.examples.windowing.
> > TopSpeedWindowing
> > With default configuration I have no errors.
> >
> > When I change the state backend with RocksDB I receive this error:
> >
> > java.lang.RuntimeException: Could not forward element to next operator
> >                 at
> > org.apache.flink.streaming.runtime.tasks.OperatorChain$
> > CopyingChainingOutput.collect(OperatorChain.java:388)
> >                 at
> > org.apache.flink.streaming.runtime.tasks.OperatorChain$
> > ChainingOutput.collect(OperatorChain.java:1)
> >                 at org.apache.flink.streaming.api.operators.
> >
> AbstractStreamOperator$CountingOutput.collect(AbstractStreamOperator.java:
> > 393)
> >                 at org.apache.flink.streaming.api.operators.
> > AbstractStreamOperator$CountingOutput.collect(
> > AbstractStreamOperator.java:1)
> >                 at org.apache.flink.streaming.api.operators.
> > TimestampedCollector.collect(TimestampedCollector.java:51)
> >                 at org.apache.flink.streaming.api.functions.windowing.
> > PassThroughWindowFunction.apply(PassThroughWindowFunction.java:32)
> >                 at org.apache.flink.streaming.api.functions.windowing.
> > ReduceApplyWindowFunction.apply(ReduceApplyWindowFunction.java:56)
> >                 at
> org.apache.flink.streaming.runtime.operators.windowing.
> > functions.InternalIterableWindowFunction.apply(
> > InternalIterableWindowFunction.java:50)
> >                 at
> org.apache.flink.streaming.runtime.operators.windowing.
> > functions.InternalIterableWindowFunction.apply(
> > InternalIterableWindowFunction.java:1)
> >                 at
> org.apache.flink.streaming.runtime.operators.windowing.
> > EvictingWindowOperator.fire(EvictingWindowOperator.java:334)
> >                 at
> org.apache.flink.streaming.runtime.operators.windowing.
> > EvictingWindowOperator.processElement(EvictingWindowOperator.java:199)
> >                 at org.apache.flink.streaming.runtime.io.
> > StreamInputProcessor.processInput(StreamInputProcessor.java:177)
> >                 at org.apache.flink.streaming.runtime.tasks.
> > OneInputStreamTask.run(OneInputStreamTask.java:66)
> >                 at org.apache.flink.streaming.runtime.tasks.StreamTask.
> > invoke(StreamTask.java:270)
> >                 at org.apache.flink.runtime.taskmanager.Task.run(Task.
> > java:609)
> >                 at java.lang.Thread.run(Thread.java:745)
> > Caused by: java.lang.NullPointerException
> >                 at org.apache.flink.api.java.typeutils.runtime.
> > TupleSerializer.copy(TupleSerializer.java:103)
> >                 at org.apache.flink.api.java.typeutils.runtime.
> > TupleSerializer.copy(TupleSerializer.java:30)
> >                 at
> > org.apache.flink.streaming.runtime.tasks.OperatorChain$
> > CopyingChainingOutput.collect(OperatorChain.java:383)
> >                 ... 15 more
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message