incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jan Lehnardt <...@apache.org>
Subject Re: view building batch sizes
Date Thu, 09 Dec 2010 12:51:52 GMT
Hi Huw,


On 9 Dec 2010, at 13:32, Huw Selley wrote:

> Hi,
> 
> I read on http://guide.couchdb.org/draft/performance.html that 
> 
> "Views load a batch of updates from disk, pass them through the view engine, and then
write the view rows out. Each batch is a few hundred documents, so the writer can take advantage
of the bulk efficiencies we see in the next section."
> 
> Is there a method to change the batch size? I would like to try measure the impact of
using smaller and larger batches.

Thanks for helping to profile things. You may want to take this to
dev@couchdb.apache.org as it is the development-related mailing list.

For tuning these values, see src/couchdb/couch_view_updater.erl

The `update()` function has these lines:

    {ok, MapQueue} = couch_work_queue:new(100000, 500),
    {ok, WriteQueue} = couch_work_queue:new(100000, 500),

They set up a queue for mapping and writing each. The parameters are

    couch_work_queue:new(MaxSize, MaxItems)

If either maximum is hit, the queue is deemed full.

Note: This is from about 30 seconds of looking at the source, so I 
might miss a subtlety or three.

Cheers
Jan
-- 



Mime
View raw message