flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yassine MARZOUGUI <y.marzou...@mindlytix.com>
Subject Re: AsyncIO/QueryableStateClient hanging with high parallelism
Date Mon, 06 Mar 2017 16:08:15 GMT
I think I found the reason for what happened. The way I used the
QueryableStateClient is that I wrapped scala.concurrent.Future in a
FlinkFuture and then called FlinkFuture.thenAccept. It turns out
thenAccept doesn't
throw exceptions and when an exception happens (which likely happened once
I inreased the parallelism) the job simply doesn't finish. I solved the
problem by using resultFuture.get()which araised the appropriate exceptions
when they happens and failed the job.

Best,
Yassine

2017-03-06 15:53 GMT+01:00 Yassine MARZOUGUI <y.marzougui@mindlytix.com>:

> Hi all,
>
> I set up a job with simple queryable state sink and tried to query it from
> another job using the new Async I/O API. Everything worked as expected,
> except when I tried to increase the parallelism of the querying job it
> hanged.
> As you can see in the attached image, when the parallism is 5 (even <5)
> the job finishes within 5 seconds, but when it is >5 it hangs. Any Idea of
> what might be causing this behaviour? Thank you.
>
> Best,
> Yassine
>

Mime
View raw message