Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@couchdb.apache.org
Received-SPF: pass (athena.apache.org: domain of b.candler@pobox.com
 designates 208.72.237.25 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws; d=pobox.com; h=date:from:to:cc
	:subject:message-id:references:mime-version:content-type
	:in-reply-to; q=dns; s=sasl; b=UoADgv6TOrW76Et2VyQMmZzZThV/1PDek
	IhOA8ncn2UtwWJKaCGo47JaEelHaEkO20sU3AXvcz4MUL0A7o1x5tfoqrTZW+Ppq
	8g9g0ikuFOcN00GYpWE6xMRfXn/4+KqeVbDxmRw3sp9unr/bKcnv3ptezYVDj/Eo
	eSXhm7FFVU=
Date: Wed, 25 Nov 2009 09:25:55 +0000
From: Brian Candler <B.Candler@pobox.com>
To: Dirkjan Ochtman <dirkjan@ochtman.nl>
Cc: dev@couchdb.apache.org
Subject: Re: Improving pagination for views
Message-ID: <20091125092555.GD7275@uk.tiscali.com>
Mail-Followup-To: Dirkjan Ochtman <dirkjan@ochtman.nl>,
 dev@couchdb.apache.org
References: <ea2499da0911241133y6645ec07l6ab853f541cd6e77@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <ea2499da0911241133y6645ec07l6ab853f541cd6e77@mail.gmail.com>
User-Agent: Mutt/1.5.17+20080114 (2008-01-14)

On Tue, Nov 24, 2009 at 08:33:43PM +0100, Dirkjan Ochtman wrote:
> On Couch, as far as I can see, pre-issuing a count query means
> actually retrieving all the row data, which doesn't seem to scale
> nicely.

Not true: you have a count reduce function. Because each node of the Btree
contains the already-calculated reduce value, this is very efficient. At
best, it just reads the value out of the root node. At worst, for a key
range, it will combine (re-reduce) count values for the subtrees within the
key range, and count a few loose values at each end of the range.

Where it's inefficient is when retrieving: if I understand it correctly,
using offset=50000 *will* require walking through 50,000 rows in the index.

That's unless each Btree node has a calculated 'number of rows' value which
would allow chunks of tree to be skipped over quickly. I haven't heard
anyone say that it does, but I could be wrong.

For now, the best advice seems to be use 'next' and 'prev' links, using
descending=true to get the values for the previous page.