couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam Kocoloski <kocol...@apache.org>
Subject Re: some thoughts on view
Date Wed, 18 Nov 2009 16:46:09 GMT
Hi,

On Nov 18, 2009, at 4:17 AM, Li Zhengji wrote:

> Hi all,
> 
> I am planning to use CouchDB now, and coming into some thoughts today,
> which may be silly.
> 
> 1. One map, multiple reduces.
> 
> There are possibilities that multiple views share the same map
> function. Then, I think these views could be combined, sharing the
> same map function. The outcome is obvious: duplicated operations on
> the same doc are eliminated.

We already do this -- if two views in a design document have byte-identical map functions,
the map is only run once, the index is only saved once, and the different reduce functions
share that index.

> 2. Why "reduce", let's "fold"
> 
> The reduce function is a little ugly due to the existance of parameter
> "rereduce".
> 
> The reduce process is much like the "fold" function in functional
> programming languages, except that the function to be reduced is
> required to be assocative and commutative (then, the reduce process
> could be solved/implemented in map/reduce manner, in parallel, and
> incrementally).
> 
> It's better to introduce an "Acc" parameter and remove the "rereduce"
> flag (like fold), then the reduce function would be simpler:
> 
> fun (keys, values, Acc) -> updated Acc.
> 
> We just need to attach an "Acc" field for reduce in view doc.

I agree, reduce is a bit warty.  I'm seeing an awful lot of people write reduce functions
that don't properly account for the possibility of rereduce.  Their views work fine on small
datasets, but they've got this hidden bug that may only manifest itself in production.

On the face of it, I'm not sure the "fold" approach is a good match for the btree-based index
storage that CouchDB uses.  But maybe I'm just not thinking about it hard enough.

Thanks for posting these thoughts -- always nice to get a fresh look at things.  Best,

Adam





Mime
View raw message