flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ziyad Muhammed <mmzi...@gmail.com>
Subject Re: Queryable State max number of clients
Date Mon, 14 Aug 2017 20:44:00 GMT
Hi Aljoscha, Ufuk

Thank you for the replies.

I'm using RocksDB state backend. Could you please explain the blocking I/O
calls mentioned?
when will it happen? And what will be the effect? A timeout exception?


Best
Ziyad

On Mon, Aug 14, 2017 at 10:17 PM, Ufuk Celebi <uce@apache.org> wrote:

> This is as Aljoscha describes. Each thread can handle many different
> clients at the same time. You shouldn't need to change the defaults in
> most cases.
>
> The network threads handle the TCP connections and dispatch query
> tasks to the query threads which do the actual querying of the state
> backend. In case of the RocksDB backend for example this might involve
> blocking I/O calls.
>
> – Ufuk
>
> On Mon, Aug 14, 2017 at 12:35 PM, Aljoscha Krettek <aljoscha@apache.org>
> wrote:
> > Hi,
> >
> > I think the number of network treads and number of query threads only
> > roughly correlate with the number of clients that can query in parallel
> > since this is using asynchronous communication via Akka/Netty. Of course,
> > increasing that number means there can be more connections but I think
> even
> > just 1 Thread or each of those should be able to easily handle multiple
> > queries at the same time.
> >
> > I'm cc'ing Ufuk and Kostas who might know more about this.
> >
> > Best,
> > Aljoscha
> >
> > On 9. Aug 2017, at 17:19, Ziyad Muhammed <mmziyad@gmail.com> wrote:
> >
> > Hi all,
> >
> > I'm trying to understand how many parallel clients will be supported by
> the
> > queryable state.
> >
> > query.server.network-threads: number of network (event loop) threads for
> the
> > KvStateServer (0 => #slots)
> > query.server.query-threads: number of asynchronous query threads for the
> > KvStateServerHandler (0 => #slots).
> >
> > so, if I choose 0 for both these parameters, what will be the maximum
> number
> > of parallel clients supported?
> >
> > I tried more parallel clients than number of slots, but all of them were
> > able to query the state in parallel. Can someone help me to understand
> the
> > logic here?
> >
> > Thanks in advance.
> >
> > Best
> > Ziyad
> >
> >
>

Mime
View raw message