Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@couchdb.apache.org
Message-ID: <838097018.286291268731107266.JavaMail.jira@brutus.apache.org>
Date: Tue, 16 Mar 2010 09:18:27 +0000 (UTC)
From: "Henrik Thostrup Jensen (JIRA)" <jira@apache.org>
To: dev@couchdb.apache.org
Subject: [jira] Updated: (COUCHDB-700) Performance Regression for view
 generation in 0.11
In-Reply-To: <1732606902.286241268731107224.JavaMail.jira@brutus.apache.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/COUCHDB-700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Henrik Thostrup Jensen updated COUCHDB-700:
-------------------------------------------

    Attachment: couchdb_011_view_speedup.diff

Increase write work queue in Couchdb. Lowers the number of writes in view generation, which leads to greater throughput / performance.

> Performance Regression for view generation in 0.11
> --------------------------------------------------
>
>                 Key: COUCHDB-700
>                 URL: https://issues.apache.org/jira/browse/COUCHDB-700
>             Project: CouchDB
>          Issue Type: Improvement
>          Components: Database Core
>    Affects Versions: 0.10
>         Environment: Ubuntu 9.04: Stock CouchDB 0.10 / Own build of CoucDB 0.11 branch.
>            Reporter: Henrik Thostrup Jensen
>         Attachments: couchdb_011_view_speedup.diff
>
>
> Copied from mail to dev at March 15, 2010:
> I have a synthetic benchmark for view generation over 70K documents. In stock CouchDB 0.10, the view will be checkpointed about 15-17
> times. Around 9 times with the batch_save_size and batch_save_interval set to 10000. CouchDB 0.11 on the other hand performs a whopping
> 108/109 checkpoints of the view. Due to shadow B-trees this generates significantly larger view files (2-3x much) and more time is spend
> writing to disk. Generating the view takes roughly twice as long in 0.11 as it does in 0.10.
> I've tracked down the problem to the new view generation architecture; particularly the small sizes of the work queues defined in couch_view_updater.erl. The attached patch decreases the number of checkpoints to around 15, and makes view generation slightly faster
> than 0.10. It basically increases the size of the write queue. Inserting a 500 ms sleep in do_writes increased the performance a bit
> more, but that is not a nice or right solution.
> I suspect the patch is not "the completely right solution (tm)", as a lot checkpoints are performed initially whereafter it backs off and
> starts to take longer time/revisions between the checkpoints. I suspect that the code is just writing repeatedly and as writes start
> to take longer time, more revisions are added per checkpoint. Though I am not really sure of this.
> Still, it is a 2 line patch, and it significantly increases view generation performance. I'd very much like to see this in 0.11, to
> avoid a rather large performance regression between 0.10 and 0.11. If 0.11 comes out as it is, we would either have to stick with 0.10 or build our own patched 0.11.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.