couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Henrik Thostrup Jensen <thost...@gmail.com>
Subject Re: Performance Regression for view generation in 0.11
Date Mon, 29 Mar 2010 09:02:23 GMT
Hi Adam, thanks for taking an interest in this.

Btw., I also created this as a jira issue at:
https://issues.apache.org/jira/browse/COUCHDB-700

> Henrik, thanks for this cogent and detailed analysis.  I was aware that the 0.11 work
queues introduced significantly more checkpoints and thus larger uncompacted view index files
in the normal case where writes to the view indices are fast.  I was not aware that this
slowed down overall view generation time; I had always sort of assumed that the queues would
make optimal use of resources.  How many cores do you have on your benchmark rig?

Well, these things should really be measured :-). Actually 0.11 writes
A LOT to more disk. More I/O almost always means slower. And
unfortunately keeping resources busy is not the same as optimal usage.
The scheme also has the unintentional effect that faster I/O systems
can do more writes thereby using more space.

I've used my laptop for benchmarking. It has 2 cores.

> It's probably too late to add this simple patch to 0.11, but I'm sure something like
it will be in 1.0 (which should follow shortly on the heels of 0.11).  Best,

The patch I submitted is really only a simple stopgap; though it works
fairly well. A proper fix it to better control the disk-writeouts.
0.10 had this fairly right with the batch_save_size and
batch_save_interval options (which no longer affects views in 0.11).


   best regards, Henrik

Mime
View raw message