Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: neutral (nike.apache.org: local policy)
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Apple Message framework v1082)
Subject: Re: view building batch sizes
From: Jan Lehnardt <jan@apache.org>
In-Reply-To: <062561F4-3C4A-4824-95C6-F26DDAB004FC@netdev.co.uk>
Date: Thu, 9 Dec 2010 13:51:52 +0100
Content-Transfer-Encoding: quoted-printable
Message-Id: <2C192E38-C044-4E8E-B907-2549A80FB11C@apache.org>
References: <062561F4-3C4A-4824-95C6-F26DDAB004FC@netdev.co.uk>
To: user@couchdb.apache.org

Hi Huw,


On 9 Dec 2010, at 13:32, Huw Selley wrote:

> Hi,
>=20
> I read on http://guide.couchdb.org/draft/performance.html that=20
>=20
> "Views load a batch of updates from disk, pass them through the view =
engine, and then write the view rows out. Each batch is a few hundred =
documents, so the writer can take advantage of the bulk efficiencies we =
see in the next section."
>=20
> Is there a method to change the batch size? I would like to try =
measure the impact of using smaller and larger batches.

Thanks for helping to profile things. You may want to take this to
dev@couchdb.apache.org as it is the development-related mailing list.

For tuning these values, see src/couchdb/couch_view_updater.erl

The `update()` function has these lines:

    {ok, MapQueue} =3D couch_work_queue:new(100000, 500),
    {ok, WriteQueue} =3D couch_work_queue:new(100000, 500),

They set up a queue for mapping and writing each. The parameters are

    couch_work_queue:new(MaxSize, MaxItems)

If either maximum is hit, the queue is deemed full.

Note: This is from about 30 seconds of looking at the source, so I=20
might miss a subtlety or three.

Cheers
Jan
--=20