couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Filipe David Manana <fdman...@apache.org>
Subject Re: Read request throughput
Date Sat, 11 Dec 2010 17:20:17 GMT
Huw,

I backported that ticket's patch into the branch 1.0.x (from which
1.0.2 will be based).
Just found now that 1.1.x (and trunk) have worst read and write
performance compared to latest 1.0.x. The issue seems to be a major
Mochiweb version upgrade. We're trying to find out what changed.

Therefore I recommend you to use 1.0.x.

regards,

On Thu, Dec 9, 2010 at 11:04 AM, Filipe David Manana
<fdmanana@apache.org> wrote:
> Hi Huw,
>
> Great news!
>
> I don't expect you to see any significant performance differences
> between 1.0.x and 1.1x however.
>
> Thanks for letting us know about your tests.
>
> best regards,
>
> On Thu, Dec 9, 2010 at 10:09 AM, Huw Selley <huw.selley@netdev.co.uk> wrote:
>> Hi,
>>
>> On 8 Dec 2010, at 19:24, Filipe David Manana wrote:
>>
>>> Huw,
>>>
>>> Today trunk was patched to increase both read and write performance
>>> when there are several requests in parallel to the same database/view
>>> index file.
>>
>> Great news :)
>>
>>>
>>> The corresponding ticket is https://issues.apache.org/jira/browse/COUCHDB-980
>>>
>>> Would be much appreciated if you could try the latest trunk and report back :)
>>
>> WOW - I built from svn rev 1043651 (again with Erlang R14B) this morning and have
just performed the same jmeter tests with some good results.
>> I am still seeing the same throughput score from jmeter, ~500 requests/s but what
is interesting is that I can now drive up the threadpool count in jmeter from 25 (the value
I had for my last round of testing) up to 750 with no errors - just increased request latency
(which is to be expected).
>>
>> Processor utilisation also looks more like I would expect:
>>
>> 09:16:43 AM  CPU   %user   %nice    %sys %iowait    %irq   %soft  %steal
  %idle    intr/s
>> 09:16:48 AM  all   19.82    0.00    9.19    0.00    0.04    0.25    0.00
  70.70   6136.20
>> 09:16:48 AM    0   43.80    0.00    5.00    0.00    0.00    0.00  
 0.00   51.20   1000.20
>> 09:16:48 AM    1   38.40    0.00   21.00    0.00    0.00    0.40    0.00
  40.20    460.80
>> 09:16:48 AM    2   44.49    0.00   14.43    0.00    0.00    0.00    0.00
  41.08      8.80
>> 09:16:48 AM    3   36.87    0.00   24.05    0.00    0.00    0.40    0.00
  38.68    449.80
>> 09:16:48 AM    4   45.29    0.00   11.42    0.00    0.00    0.00    0.00
  43.29      0.00
>> 09:16:48 AM    5   40.68    0.00   24.65    0.00    0.00    0.00    0.00
  34.67      1.60
>> 09:16:48 AM    6   47.20    0.00   14.40    0.00    0.00    0.00    0.00
  38.40      0.00
>> 09:16:48 AM    7   13.00    0.00   10.40    0.00    0.00    0.00    0.00
  76.60      0.00
>> 09:16:48 AM    8    5.01    0.00   10.82    0.00    0.00    0.00  
 0.00   84.17      0.00
>> 09:16:48 AM    9    0.00    0.00    0.00    0.00    0.00    0.40  
 0.00   99.60    455.40
>> 09:16:48 AM   10    0.00    0.00    0.00    0.00    0.00    0.00  
 0.00  100.00      0.00
>> 09:16:48 AM   11    0.00    0.00    0.00    0.00    0.20    0.40  
 0.00   99.40    482.60
>> 09:16:48 AM   12    2.00    0.00    8.58    0.00    0.00    0.00  
 0.00   89.42     10.40
>> 09:16:48 AM   13    0.00    0.00    0.00    0.00    0.00    0.40  
 0.00   99.60    488.80
>> 09:16:48 AM   14    0.20    0.00    1.40    0.00    0.00    0.40  
 0.00   98.00    447.20
>> 09:16:48 AM   15    0.60    0.00    0.80    0.00    0.20    1.20  
 0.00   97.20   2330.60
>>
>> The box has HyperThreading enabled but the spread of load over cores looks a lot
more like what I would expect to see.
>> I retried this same test against 1.0.1 with a threadpool count of 750 and it manages
~126 requests/s with a high error % (again, to be expected), Am yet to retry against the 1.1.x
branch build I was using as it was on the same box the rev 1043651 build it on.
>>
>> I have also experimented with increasing the threadpool count against the rev 1043651
build and I can go upto 950 threads with no errors (throughput drops a little at that point).
This is with a default couch configuration (only change is delayed_commits=false).
>>
>> Also, I noticed your relaximation tool (which looks pretty awesome btw) from reading
COUCHDB-980 so if I get some time I might give that a run on this hardware and see what pretty
graphs it can generate :)
>>
>> So to summarise: All in all a massive performance win!
>>
>> Many thanks to both Adam and Filipe for all your help and advice. It's really nice
to see such that couch has such a vibrant community and I hope to be able to contribute more
in the future.
>>
>> Regards
>> Huw
>
>
>
> --
> Filipe David Manana,
> fdmanana@gmail.com, fdmanana@apache.org
>
> "Reasonable men adapt themselves to the world.
>  Unreasonable men adapt the world to themselves.
>  That's why all progress depends on unreasonable men."
>



-- 
Filipe David Manana,
fdmanana@gmail.com, fdmanana@apache.org

"Reasonable men adapt themselves to the world.
 Unreasonable men adapt the world to themselves.
 That's why all progress depends on unreasonable men."

Mime
View raw message