couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jan Lehnardt <>
Subject Re: view building batch sizes
Date Thu, 09 Dec 2010 12:51:52 GMT
Hi Huw,

On 9 Dec 2010, at 13:32, Huw Selley wrote:

> Hi,
> I read on that 
> "Views load a batch of updates from disk, pass them through the view engine, and then
write the view rows out. Each batch is a few hundred documents, so the writer can take advantage
of the bulk efficiencies we see in the next section."
> Is there a method to change the batch size? I would like to try measure the impact of
using smaller and larger batches.

Thanks for helping to profile things. You may want to take this to as it is the development-related mailing list.

For tuning these values, see src/couchdb/couch_view_updater.erl

The `update()` function has these lines:

    {ok, MapQueue} = couch_work_queue:new(100000, 500),
    {ok, WriteQueue} = couch_work_queue:new(100000, 500),

They set up a queue for mapping and writing each. The parameters are

    couch_work_queue:new(MaxSize, MaxItems)

If either maximum is hit, the queue is deemed full.

Note: This is from about 30 seconds of looking at the source, so I 
might miss a subtlety or three.


View raw message