couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Hall <fli...@flimzy.com>
Subject Re: CouchDB and future
Date Thu, 11 Jul 2019 16:13:19 GMT
Does the number and type of disks affect the ideal q value?

On spinning disks, I might imagine a performance gain by putting each 
shard on a separate spindle, in which case the ideal performance might 
be something like q = # of cores = # of spindles. (Although I don't 
believe it's trivial to separate shards in such a way.)

This may not need to figure into the discussion, since I believe the 
goal is to find the ideal for low-end hardware, not every possible tweak 
for high performance.

Jonathan

On 7/11/19 6:07 PM, Joan Touzet wrote:
> Hi Tabeth,
>
> This is a good start. I'll let others comment on the specific advice.
>
> One point raised in this thread is ermouth's request to have this value
> automatically set by the installer. Detecting the number of cores in a
> cross-platform way is pretty challenging, especially when you consider
> things like hyperthreading (which aren't full "cores") on physical machines.
>
> I agree a table would be a good thing for the docs, but we really need a
> single number for q (plus other config settings) that will be good
> defaults for low-performing HW.
>
> -Joan
>
> On 2019-07-11 11:43, Tabeth Nkangoh wrote:
>> Hello all,
>>
>>
>> After reading the documentation <https://docs.couchdb.org/en/2.2.0/cluster/sharding.html>
and this thread I propose the following formula to document officially for determining the
Q and N values.
>>
>>
>> To start, as currently documented, N should be set to the number of nodes. This much
is straightforward and is documented more or less correctly.
>>
>>
>>
>> For Q, it should equal the amount of cores at a minimum. So, for the cheapest Digital
Ocean droplet, which 1GB of RAM and 1vCPU, Q should be set to 1 by the user. In the case that
each core is sufficiently beefy, e.g. Intel Xeon or turbo clocked AMD EPYC 7000 2 should be
added. 2 is an arbitrary value but reflects the reality that from what I can tell a dramatically
higher clock speed benefits CouchDB performance AFAIK (I'm not sure if this benefit is a constant
or a multiplier, but I doubt it is a multiplier simply due to the diminishing returns seen
in increased clock speed).
>>
>>
>> To summarize the following table can be used (for example):
>>
>>
>> CPU Cores       CPU Clock Speed         Q Value
>> 1       Low     1
>> 2       Low     2
>> 2       High    4
>> 3       High    5
>> 4       Low     4
>>
>>
>> Does anyone have any qualms with this as a general guide to give users in determining
the Q value they should set for their CouchDB instance? If not I can submit a PR soon on this
(have to figure out how to create a new reference for one that's currently open first).
>>
>> Tabeth
>>
>> ________________________________
>> From: Joan Touzet <wohali@apache.org>
>> Sent: Thursday, July 11, 2019 11:17 AM
>> To: dev@couchdb.apache.org
>> Subject: Re: CouchDB and future
>>
>> On 2019-07-11 8:05, ermouth wrote:
>>>> to help with the documentation step
>>> Probably. Please give me a hint what you need.
>> What default settings right now are wrong for a system with low RAM, low
>> CPU, and slow disk - such as a RaspberryPi v1, or a $15/mo AWS server?
>>
>> -Joan
>>
>>

Mime
View raw message