couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dave Amies <damie...@gmail.com>
Subject CouchDB / NoSQL Benchmarking
Date Thu, 22 Oct 2015 13:13:06 GMT
Hi All,

I'm sure by now most of you will have read at least some parts of this
guide:

http://guide.couchdb.org/draft/performance.html

I was reading it the other day and noticed the "Call to Arms" section at
the bottom of the page. I don't know if there are already any benchmarking
tools out there, but I decided to try writing one. Hopefully the one I have
written will be useful.

About my background, for my day job i am a performance tester, usually
specialising in Loadrunner, so this project was something to keep my mind
occupied while waiting for my test system to be rebuilt. Given this I have
only spent a few hours on it and so there is probably still room for
improvement, this email is about finding out if there is interest or if
this will be useful to the CouchDB community, so really should I continue
developing this tool, or am I wasting my time?

In designing this benchmarking utility I reflected on all the systems I
have tested and tried to come up with some common areas where database
systems suffer in performance. Then bearing in mind the fundamental
differences between traditional databases and NoSQL databases (particularly
CouchDB) I tried to construct some some common database usage scenarios.

The 3 scenarios I came up with are:

   1. Write heavy (each user performs 12 writes, 6 reads and 3 searches /
   index queries)
   2. Index / Query / Search heavy (each user performs 1 write, 2 reads and
   6 searches / index queries)
   3. Read Heavy (each user performs 1 writes, 10 reads and 3 searches /
   index queries)

I have tried out my benchmarking tool on a couple of machines so far, in
these tests I managed to cause CouchDB to encounter the following
situations:

   1. Performance degradation due to being Disk IO bound
   2. Performance degradation due to being Memory bound
   3. Performance degradation due to being CPU bound
   4. Couch DB crashed
   5. Benchmarking completed successfully and produce a performance score

Based on these results I believe I have created an effective tool for
benchmarking, so I decided the best next step was to release the tool as an
open source project, so I created a github project which can be found here:
https://github.com/damies13/kvbench. Here you will the readme file
describes the 3 scenarios in more detail, the benchmark definition or
design and also the pre benchmark data priming. You will also find here the
python script that is the benchmarking tool and some instructions for
setting up a couch db database for the benchmarking process.

As this is getting long i'll wrap up by noting that I deliberately did not
use the python couchdb libraries but instead I used the requests library
(standard http) and json library because I wanted to keep the code as
generic as possible, the intention is that this benchmarking tool should be
able to be used to benchmarking any key / value store, whether that be a
document based NoSQL, and Key Value based NoSQL database or some other Rest
API / engine (e.g. backed by a traditional database).

I look forward to some feed back, hopefully I have created something
useful.

Sincerely,

Dave.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message