Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (athena.apache.org: domain of
 sebastiancohnen@googlemail.com designates 209.85.214.48 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=googlemail.com; s=gamma;
        h=content-type:mime-version:subject:from:in-reply-to:date
         :content-transfer-encoding:message-id:references:to:x-mailer;
        b=lwr2039g5E0Sk5KXzbkY5IKXQkO4Vp0ztWufW1V1qmTwcGNsIb+XTuwff06RRvmE8a
         +sq9SNy2WMYnAqyFFBqW6iRbug+7SIbutjHrUYvHd+FndMEX7UgM902NbFn+u/dZnjMa
         0rn+m7IeteKg58mDzN7XN0EVI47oNgQCSjeRU=
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Apple Message framework v1082)
Subject: Re: Comparison of MongoDB & CouchDB: MongoDB seems better on insert
From: Sebastian Cohnen <sebastiancohnen@googlemail.com>
In-Reply-To: <AANLkTimHk26NU2sMW-d1M4aXERpvTFXNxv7hy=8dqLiU@mail.gmail.com>
Date: Tue, 21 Dec 2010 00:01:10 +0100
Content-Transfer-Encoding: quoted-printable
Message-Id: <3F821DE2-2DC5-47A8-B7F6-CE8B0B131693@googlemail.com>
References: 
 <E0A3C299B996D840B5E96ECD9EEDC5B501361AAF@GP2K0084V3.GEICO.corp.net>
 <0D367803-5B0E-4C84-9517-5EDCD96DB114@apache.org>
 <D38B72C1-8D98-4798-80BB-264AD29A49B2@googlemail.com>
 <AANLkTimHk26NU2sMW-d1M4aXERpvTFXNxv7hy=8dqLiU@mail.gmail.com>
To: user@couchdb.apache.org


On 20.12.2010, at 23:24, Paul Davis wrote:

> On Mon, Dec 20, 2010 at 5:20 PM, Sebastian Cohnen
> <sebastiancohnen@googlemail.com> wrote:
>> question inside :)
>>=20
>> On 20.12.2010, at 23:02, Jan Lehnardt wrote:
>>=20
>>> Hi,
>>>=20
>>> On 20 Dec 2010, at 22:32, Chenini, Mohamed wrote:
>>>=20
>>>> Hi,
>>>>=20
>>>> I found this info on the net at =
http://www.slideshare.net/danglbl/schemaless-databases
>>>> [...]
>>>> Does anyone knows if this was verified?
>>>=20
>>> I think the author's comment on slide 35 sums it up pretty nicely:
>>>=20
>>> "Of course this is just one (lame) test."
>>>=20
>>> Coming up good numbers is hard which means that people with easy =
ways to make them come up with bad ones.
>>>=20
>>> I've written about the difficulties on benchmarks databases on my =
blog:
>>>=20
>>>  =
http://jan.prima.de/~jan/plok/archives/175-Benchmarks-You-are-Doing-it-Wro=
ng.html
>>>  =
http://jan.prima.de/~jan/plok/archives/176-Caveats-of-Evaluating-Databases=
.html
>>>=20
>>> They should give you a few pointers on why this is hard.
>>>=20
>>> --
>>>=20
>>> To the point: CouchDB generally performs best with concurrent load. =
In the case of loading data into CouchDB, bulk requests* will speed up =
things again. To push CouchDB to a write limit, you want to use =
concurrent bulk requests (specific numbers will depend on your data and =
hardware).
>>=20
>> Does this really speed up things? I've tried this approach =
(concurrent bulk inserts) with small/big docs and small/big bulk chunk =
sizes: the difference was not significant. I thought this was =
reasonable, since writes are serialized anyways. The setup was one box =
generating documents, creating bulks and keep them in memory and bulk =
insert batches of complete docs (incl. simple monotonic increasing ints =
as doc ids) to another node. delayed commit was off.
>>=20
>=20
> I think delayed commit would need to be on there otherwise you'll be
> hitting fsync barriers for every bulk docs call which are serialized
> by the updater. Theoretically the speedups would come from letting the
> kernel manage the file buffers and what not.

delayed_commit was off because I needed to test insertion of lots of =
data (more than what would fit nicely into memory). I wanted to figure =
out, if normal bulk vs concurrent bulks does have an impact on insert =
performance. the difference was, as I said, not significant better or =
worse. btw: I didn't saturated the disks (mid-classed SSDs), since couch =
was eating up the CPU (3GHz Core 2 Duo). This was some time ago, maybe =
this is more disk bound now.

>=20
>>>=20
>>> * http://wiki.apache.org/couchdb/HTTP_Bulk_Document_API
>>>=20
>>> Unfortunately this means that these one-off benchmarks don't show =
any good numbers for CouchDB, yet fortunately this shows easily that =
these one-off benchmarks don't really reflect common real-world usage =
and should be discouraged.
>>>=20
>>> Hope that helps, let us know if you have any more questions :)
>>>=20
>>> Cheers
>>> Jan
>>> --
>>>=20
>>=20
>>=20