couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Benoit Chesneau <bchesn...@gmail.com>
Subject Re: Performance Regression for view generation in 0.11
Date Sat, 13 Mar 2010 19:44:56 GMT
On Thu, Mar 11, 2010 at 12:51 PM, Henrik Thostrup Jensen
<thostrup@gmail.com> wrote:
> Hi
>
> I recently did some performance test of different _id sizes [1], including a
> comparison between stock 0.10 and a 0.11 snapshot. Unexpectedly, 0.11 turned
> out to be slower for view generation, despite the work done with COUCHDB-495.
> I've poked a bit at the problem, and I think the reason (or a part of it), is
> the increased number of fsync call during view generation for checkpointing. In
> 0.11, checkpointing is done every second, where it is done around 10-15 seconds
> in 0.10. This increases the disk load and creates large B-trees due to
> extensive shadowing (as least that is what I observe).
>
> Is it possible to configure this interval? Delayed commits is not what i want,
> as i want insertions to fsync. I tried changing the three 1000 constants in the
> send_after delays in couch_view_group.erl, but checkpointing was still done
> each second (I'm not to strong on Erlang, so I had a bit of problem grokking
> the code). In the case of a crash this will only cause a certain part of the db
> to be scanned for the view, so IMHO the checkpointing interval could easily be
> set to 60 seconds or so.
>
> There are a lot of guessing in the above, and I'm not sure of everything.
> However as we are accumulating more data the time view generation is starting
> to hurt when creating new views, so I'd like to see CouchDB become faster at
> this (we expect to do some _id field reducement which should help a bit as
> well, but it won't buy us that much).
>
> Are there any other explanations for the performance regressions?
>
> Please CC any replies, as I am not subscribed.
>
>
>  best regards, Henrik
>
>

delayed_commits = true ; set this to false to ensure an fsync before
201 Created is returned
batch_save_size = 1000 ; number of docs at which to save a batch
batch_save_interval = 1000 ; milliseconds after which to save batches

are the settings you may want to change in ini.

- benoît

Mime
View raw message