couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Henrik Thostrup Jensen (JIRA)" <j...@apache.org>
Subject [jira] Updated: (COUCHDB-700) Performance Regression for view generation in 0.11
Date Tue, 30 Mar 2010 11:52:27 GMT

     [ https://issues.apache.org/jira/browse/COUCHDB-700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Henrik Thostrup Jensen updated COUCHDB-700:
-------------------------------------------

        Fix Version/s: 0.12
    Affects Version/s:     (was: 0.10)
                       0.11

> Performance Regression for view generation in 0.11
> --------------------------------------------------
>
>                 Key: COUCHDB-700
>                 URL: https://issues.apache.org/jira/browse/COUCHDB-700
>             Project: CouchDB
>          Issue Type: Improvement
>          Components: Database Core
>    Affects Versions: 0.11
>         Environment: Ubuntu 9.04: Stock CouchDB 0.10 / Own build of CoucDB 0.11 branch.
>            Reporter: Henrik Thostrup Jensen
>             Fix For: 0.12
>
>         Attachments: couchdb_011_view_speedup.diff
>
>
> Copied from mail to dev at March 15, 2010:
> I have a synthetic benchmark for view generation over 70K documents. In stock CouchDB
0.10, the view will be checkpointed about 15-17
> times. Around 9 times with the batch_save_size and batch_save_interval set to 10000.
CouchDB 0.11 on the other hand performs a whopping
> 108/109 checkpoints of the view. Due to shadow B-trees this generates significantly larger
view files (2-3x much) and more time is spend
> writing to disk. Generating the view takes roughly twice as long in 0.11 as it does in
0.10.
> I've tracked down the problem to the new view generation architecture; particularly the
small sizes of the work queues defined in couch_view_updater.erl. The attached patch decreases
the number of checkpoints to around 15, and makes view generation slightly faster
> than 0.10. It basically increases the size of the write queue. Inserting a 500 ms sleep
in do_writes increased the performance a bit
> more, but that is not a nice or right solution.
> I suspect the patch is not "the completely right solution (tm)", as a lot checkpoints
are performed initially whereafter it backs off and
> starts to take longer time/revisions between the checkpoints. I suspect that the code
is just writing repeatedly and as writes start
> to take longer time, more revisions are added per checkpoint. Though I am not really
sure of this.
> Still, it is a 2 line patch, and it significantly increases view generation performance.
I'd very much like to see this in 0.11, to
> avoid a rather large performance regression between 0.10 and 0.11. If 0.11 comes out
as it is, we would either have to stick with 0.10 or build our own patched 0.11.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message