flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shannon Carey <sca...@expedia.com>
Subject Re: [DISCUSS] Proposal for Asynchronous I/O in FLINK
Date Wed, 21 Sep 2016 15:04:00 GMT

I just wanted to say "thanks" for making this proposal! I'm also interested in performing
nonblocking I/O (multiplexing threads/reactive programming) within Flink operators so that
we can, for example, communicate with external web services with Netty/RxNetty without blocking
an entire Flink slot (aka a thread) while we wait for the operation to complete. It looks
like your FLIP will enable that use case.

I'm not sure whether it will be possible to share one Netty EventLoopGroup (or the equivalent
for any other non-blocking framework, connection pool, etc.) among multiple slots in a single
JVM though. Flink supports open/close operation on a RichFunction, but that's on a per-slot
basis. I don't know of a way to open/close objects on a per-job-JVM basis. But I suppose that's
an issue that should be discussed and resolved separately.


View raw message