couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dave Amies <damie...@gmail.com>
Subject Re: CouchDB / NoSQL Benchmarking
Date Wed, 04 Nov 2015 13:41:07 GMT
>    4. Couch DB crashed

Unfortunately I lost the crash logs for this test, I tried to reproduce the
crash but was not successful, instead I got the best result to date.

In this test the benchmark script and CouchDB were not running on the same
machine. The benchmark script  was running on my Macbook Pro (detailed in
earlier email) and CouchDB was running on my iMac (also detailed in earlier
email). They are both connected via a wifi network with the wifi router
being a FritzBox 7390 ADSL Modem Router, which reported the following
details about their connections:

DavesMBP 192.168.xx.xx WLAN 300 Mbit/sMbit/s 5 GHz / n / 40 MHz WPA2
iMac 192.168.xx.xx WLAN 300 Mbit/sMbit/s 5 GHz / n / 40 MHz WPA2

I suspect that the wifi network may have played a part in the crash but
without the logs I don't really know. If i'm successful in reproducing the
crash I will reply to this thread.


the benchmark test output from the retry:
------------------------------------------------------------------
------------------------------------------------------------------
----------------------
Started DataPrime(100) 11/04/15 21:30:48 AEST
Finished DataPrime(100) 11/04/15 21:38:18 AEST
DataPrime took 449.787853003 Seconds
Started Phase 1(50) 11/04/15 21:38:18 AEST
.....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
Finished Phase 1(50) 11/04/15 21:51:06 AEST
Phase 1 took 768.545020103 Seconds
Started Phase 2(50) 11/04/15 21:51:06 AEST
.....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
Finished Phase 2(50) 11/04/15 22:20:21 AEST
Phase 2 took 1754.77258897 Seconds
Started Phase 3(50) 11/04/15 22:20:21 AEST
.....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
Finished Phase 3(50) 11/04/15 22:31:11 AEST
Phase 3 took 649.708209038 Seconds
Phase 1
All : 3.47095863737
Query 3.1 : 18.4526163564
Query 3.2 : 41.4287324734
Query 3.3 : 0.593025727749
Read : 0.132324189504
Write : 0.133825695554
Phase 2
All : 18.4827108563
Query 3.1 : 8.42682806325
Query 3.2 : 49.6830593367
Query 3.3 : 0.128207126141
Read : 0.0981377413273
Write : 0.117080961227
Phase 3
All : 3.76686505096
Query 3.1 : 7.71395163965
Query 3.2 : 43.3974933605
Query 3.3 : 0.186422455788
Read : 0.127953700352
Write : 0.158706254005
KV Bench v1.0.3 score: 8.57351151487
------------------------------------------------------------------
------------------------------------------------------------------
----------------------




On Tue, Oct 27, 2015 at 8:18 PM, Garren Smith <garren@apache.org> wrote:

> Hi Dave,
>
> This is very cool. Do you have the results and the scripts you used to
> benchmarch CouchDB?
>
> Cheers
> Garren
>
> On Thu, Oct 22, 2015 at 3:13 PM, Dave Amies <damies13@gmail.com> wrote:
>
> > Hi All,
> >
> > I'm sure by now most of you will have read at least some parts of this
> > guide:
> >
> > http://guide.couchdb.org/draft/performance.html
> >
> > I was reading it the other day and noticed the "Call to Arms" section at
> > the bottom of the page. I don't know if there are already any
> benchmarking
> > tools out there, but I decided to try writing one. Hopefully the one I
> have
> > written will be useful.
> >
> > About my background, for my day job i am a performance tester, usually
> > specialising in Loadrunner, so this project was something to keep my mind
> > occupied while waiting for my test system to be rebuilt. Given this I
> have
> > only spent a few hours on it and so there is probably still room for
> > improvement, this email is about finding out if there is interest or if
> > this will be useful to the CouchDB community, so really should I continue
> > developing this tool, or am I wasting my time?
> >
> > In designing this benchmarking utility I reflected on all the systems I
> > have tested and tried to come up with some common areas where database
> > systems suffer in performance. Then bearing in mind the fundamental
> > differences between traditional databases and NoSQL databases
> (particularly
> > CouchDB) I tried to construct some some common database usage scenarios.
> >
> > The 3 scenarios I came up with are:
> >
> >    1. Write heavy (each user performs 12 writes, 6 reads and 3 searches /
> >    index queries)
> >    2. Index / Query / Search heavy (each user performs 1 write, 2 reads
> and
> >    6 searches / index queries)
> >    3. Read Heavy (each user performs 1 writes, 10 reads and 3 searches /
> >    index queries)
> >
> > I have tried out my benchmarking tool on a couple of machines so far, in
> > these tests I managed to cause CouchDB to encounter the following
> > situations:
> >
> >    1. Performance degradation due to being Disk IO bound
> >    2. Performance degradation due to being Memory bound
> >    3. Performance degradation due to being CPU bound
> >    4. Couch DB crashed
> >    5. Benchmarking completed successfully and produce a performance score
> >
> > Based on these results I believe I have created an effective tool for
> > benchmarking, so I decided the best next step was to release the tool as
> an
> > open source project, so I created a github project which can be found
> here:
> > https://github.com/damies13/kvbench. Here you will the readme file
> > describes the 3 scenarios in more detail, the benchmark definition or
> > design and also the pre benchmark data priming. You will also find here
> the
> > python script that is the benchmarking tool and some instructions for
> > setting up a couch db database for the benchmarking process.
> >
> > As this is getting long i'll wrap up by noting that I deliberately did
> not
> > use the python couchdb libraries but instead I used the requests library
> > (standard http) and json library because I wanted to keep the code as
> > generic as possible, the intention is that this benchmarking tool should
> be
> > able to be used to benchmarking any key / value store, whether that be a
> > document based NoSQL, and Key Value based NoSQL database or some other
> Rest
> > API / engine (e.g. backed by a traditional database).
> >
> > I look forward to some feed back, hopefully I have created something
> > useful.
> >
> > Sincerely,
> >
> > Dave.
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message