Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (nike.apache.org: domain of b.candler@pobox.com designates
 208.72.237.25 as permitted sender)
Date: Thu, 25 Jun 2009 10:08:39 +0100
From: Brian Candler <B.Candler@pobox.com>
To: hhsuper <hhsuper@gmail.com>
Cc: user@couchdb.apache.org
Subject: Re: 'Grouping' documents so that a set of documents is passed to the
 view function
Message-ID: <20090625090839.GA7316@uk.tiscali.com>
References: <69a992ac0906220649w539baba5hff01b0ab8b841da2@mail.gmail.com>
 <20090622190530.GA18055@uk.tiscali.com>
 <69a992ac0906221815h1b61e0e2r9f3ef1fe247727a9@mail.gmail.com>
 <20090623075940.GA6643@uk.tiscali.com>
 <69a992ac0906240335n1a6ec400we8e7831f0b46947d@mail.gmail.com>
 <20090624154301.GA27351@uk.tiscali.com>
 <69a992ac0906241824n32d1f6a5r334346db02d4a065@mail.gmail.com>
 <20090625083431.GA7022@uk.tiscali.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20090625083431.GA7022@uk.tiscali.com>
User-Agent: Mutt/1.5.17+20080114 (2008-01-14)

On Thu, Jun 25, 2009 at 09:34:31AM +0100, Brian Candler wrote:
> Perhaps it will help you to understand this if you consider the limiting
> case where exactly one document is fed into the 'reduce' function at a time,
> and then the outputs of the reduce functions are combined with a large
> re-reduce phase.

Incidentally, this is a partly realistic scenario. It's quite possible given
N documents that couchdb will reduce the first N-1, then reduce the last 1,
then re-reduce those two values. This might be because of how the documents
are split between Btree nodes, or there may be a limit on the number of
documents passed to the reduce function in one go. This is entirely an
implementation issue which you have no control over, so you must write your
reduce/rereduce to give the same answer for *any* partitioning of documents.

More info at http://wiki.apache.org/couchdb/Introduction_to_CouchDB_views

"To make incremental Map/Reduce possible, the Reduce function has the
requirement that not only must it be referentially transparent, but it must
also be commutative and associative for the array value input, to be able
reduce on its own output and get the same answer, like this:

f(Key, Values) == f(Key, [ f(Key, Values) ] )"

Now, at first glance your re-reduce function appears to satisfy that
condition, so perhaps there should be another one: namely, that for any
partitioning of Values into subsets Values1, Values2, ... then

  f(Key, Values) == f(Key, [ f(Key,Values1), f(Key,Values2), ... ] )

But I am not a mathematician so I'm not sure if this condition is actually
stronger.

Regards,

Brian.