Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (nike.apache.org: domain of b.candler@pobox.com designates
 208.72.237.25 as permitted sender)
Date: Sat, 31 Jan 2009 21:00:48 +0000
From: Brian Candler <B.Candler@pobox.com>
To: user@couchdb.apache.org
Subject: Re: Incremental map/reduce
Message-ID: <20090131210048.GA12190@uk.tiscali.com>
References: <20090130101859.GA7348@uk.tiscali.com>
 <e282921e0901301032k27be6d67u2301418fb2798825@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <e282921e0901301032k27be6d67u2301418fb2798825@mail.gmail.com>
User-Agent: Mutt/1.5.17+20080114 (2008-01-14)

On Fri, Jan 30, 2009 at 10:32:15AM -0800, Chris Anderson wrote:
> Once you understand how normal reduce queries (with group=false) work,
> eg: those that return a single reduction value for whatever key-range
> you specify, group_level queries are not more complex. Group_level
> queries are essentially a macro, which run one normal (group=false)
> reduce query automatically for each interval on a set of intervals as
> defined by the level.

Ah - it was new to me that map/reduce queries with group=false could run
over arbitary key intervals:

$ kurl 'http://localhost:5984/maptest/_view/test/sum'
{"rows":[{"key":null,"value":45}]}
$ kurl 'http://localhost:5984/maptest/_view/test/sum?startkey="doc3"'
{"rows":[{"key":null,"value":42}]}
$ kurl 'http://localhost:5984/maptest/_view/test/sum?startkey="doc3"&endkey="doc5"'
{"rows":[{"key":null,"value":33}]}

This means that couchdb *must* be performing the reduce part of the query
on-demand, as opposed to keeping precomputed values stored like the map
part.

In SQL terms, this is like "count(*)" doing an index scan, rather than
having the answer precomputed in a materialised view. And suddenly the
various forms of reduce make much more sense.

However at http://damienkatz.net/2008/02/incremental_map.html it says:

"This requirement of reduce functions allows CouchDB to store off
intermediated reductions directly into inner nodes of btree indexes, and the
view index updates and retrievals will have logarithmic cost. It also allows
the indexes to be spread across machines and reduced at query time with
logarithmic cost."

Is storing the reductions a planned future feature, rather than describing
how it works today?

Sorry to keep asking, but I'm going to try to update the wiki in as accurate
a way as I can.

Thanks,

Brian.