couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Garren Smith <gar...@apache.org>
Subject Re: CouchDB / NoSQL Benchmarking
Date Tue, 27 Oct 2015 10:18:02 GMT
Hi Dave,

This is very cool. Do you have the results and the scripts you used to
benchmarch CouchDB?

Cheers
Garren

On Thu, Oct 22, 2015 at 3:13 PM, Dave Amies <damies13@gmail.com> wrote:

> Hi All,
>
> I'm sure by now most of you will have read at least some parts of this
> guide:
>
> http://guide.couchdb.org/draft/performance.html
>
> I was reading it the other day and noticed the "Call to Arms" section at
> the bottom of the page. I don't know if there are already any benchmarking
> tools out there, but I decided to try writing one. Hopefully the one I have
> written will be useful.
>
> About my background, for my day job i am a performance tester, usually
> specialising in Loadrunner, so this project was something to keep my mind
> occupied while waiting for my test system to be rebuilt. Given this I have
> only spent a few hours on it and so there is probably still room for
> improvement, this email is about finding out if there is interest or if
> this will be useful to the CouchDB community, so really should I continue
> developing this tool, or am I wasting my time?
>
> In designing this benchmarking utility I reflected on all the systems I
> have tested and tried to come up with some common areas where database
> systems suffer in performance. Then bearing in mind the fundamental
> differences between traditional databases and NoSQL databases (particularly
> CouchDB) I tried to construct some some common database usage scenarios.
>
> The 3 scenarios I came up with are:
>
>    1. Write heavy (each user performs 12 writes, 6 reads and 3 searches /
>    index queries)
>    2. Index / Query / Search heavy (each user performs 1 write, 2 reads and
>    6 searches / index queries)
>    3. Read Heavy (each user performs 1 writes, 10 reads and 3 searches /
>    index queries)
>
> I have tried out my benchmarking tool on a couple of machines so far, in
> these tests I managed to cause CouchDB to encounter the following
> situations:
>
>    1. Performance degradation due to being Disk IO bound
>    2. Performance degradation due to being Memory bound
>    3. Performance degradation due to being CPU bound
>    4. Couch DB crashed
>    5. Benchmarking completed successfully and produce a performance score
>
> Based on these results I believe I have created an effective tool for
> benchmarking, so I decided the best next step was to release the tool as an
> open source project, so I created a github project which can be found here:
> https://github.com/damies13/kvbench. Here you will the readme file
> describes the 3 scenarios in more detail, the benchmark definition or
> design and also the pre benchmark data priming. You will also find here the
> python script that is the benchmarking tool and some instructions for
> setting up a couch db database for the benchmarking process.
>
> As this is getting long i'll wrap up by noting that I deliberately did not
> use the python couchdb libraries but instead I used the requests library
> (standard http) and json library because I wanted to keep the code as
> generic as possible, the intention is that this benchmarking tool should be
> able to be used to benchmarking any key / value store, whether that be a
> document based NoSQL, and Key Value based NoSQL database or some other Rest
> API / engine (e.g. backed by a traditional database).
>
> I look forward to some feed back, hopefully I have created something
> useful.
>
> Sincerely,
>
> Dave.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message