kudu-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hao Hao <hao....@cloudera.com>
Subject Re: Changing number of Kudu worker threads
Date Wed, 13 Feb 2019 02:09:21 GMT
Hi Boris,

Sorry for the delay,  --rpc_num_service_threads sets the number of threads
 in RPC service thread pool (the default is 20 for tablet server, 10 for
master).  It should help with processing concurrent incoming RPC requests,
but increasing it more than the number of available CPU cores of the
machines may not bring too much value.

You don't need to set the same value for masters and tablet servers. Most
of the time, tablet servers should have more RPCs where the scans and
writes are taking place.

Best,
Hao

On Tue, Feb 12, 2019 at 5:29 PM Boris Tyukin <boris@boristyukin.com> wrote:

> Can someone point us to documentation or explain what these parameters
> really mean or how they should be set on production cluster?
> I will greatly appreciate it!
>
> Boris
>
> On Fri, Feb 8, 2019 at 3:40 PM Boris Tyukin <boris@boristyukin.com> wrote:
>
>> Hi guys,
>>
>> we need to process 1000s of operations per second and noticed that our
>> Kudu 1.5 cluster was only using 10 threads while our application spins up
>> 50 clients/threads. We observed in the web UI that only 10 threads are
>> working and other 40 waiting in the queue.
>>
>> We found rpc_num_service_threads parameter in the configuration guide but
>> it is still not clear to me what we need to adjust exactly to allow Kudu to
>> handle more concurrent operations.
>>
>> Do we bump this parameter below or we need to consider other rpc related
>> parameters?
>>
>> Also do we need to use the same numbers for Masters and tablets?
>>
>> Is there any good numbers to target based on CPU core count?
>>
>> --rpc_num_service_threads
>> <https://kudu.apache.org/docs/configuration_reference.html#kudu-master_rpc_num_service_threads>
>> <https://kudu.apache.org/docs/configuration_reference.html#kudu-master_rpc_num_service_threads>
>>
>> Number of RPC worker threads to run
>>
>> Type
>>
>> int32
>>
>> Default
>>
>> 10
>>
>> Tags
>>
>> advanced
>>
>

Mime
View raw message