flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dmitry Golubets <dgolub...@gmail.com>
Subject Re: Can serialization be disabled between chains?
Date Mon, 16 Jan 2017 13:21:36 GMT
Hi Ufuk,

Do you know what's the reason for serialization of data between different
threads?

Also, thanks for the link!

Best regards,
Dmitry

On Mon, Jan 16, 2017 at 1:07 PM, Ufuk Celebi <uce@apache.org> wrote:

> Hey Dmitry,
>
> this is not possible if I'm understanding you correctly.
>
> A task chain is executed by a single task thread and hence it is not
> possible to continue processing before the record "leaves" the thread,
> which only happens when the next task thread or the network stack
> consumes it.
>
> Hand over between chained tasks happens without serialization. Only
> data between different task threads is serialized.
>
> Depending on your use case the newly introduced async I/O feature
> might be worth a look (will be part of the upcoming 1.2 release):
> https://github.com/apache/flink/pull/2629
>

Mime
View raw message