couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Anderson" <jch...@apache.org>
Subject Re: Map/Reduce takes lots of time every request
Date Sat, 22 Nov 2008 20:53:28 GMT
On Sat, Nov 22, 2008 at 12:25 PM, maddiin <maddiin@googlemail.com> wrote:
>
> Do you have any advice what I am doing wrong and how I could speed this up?


I'm curious how long it takes with reduce=false (should be limited
basically by IO).

I'm almost certain (please correct me if I'm wrong) that reduce
requests must call the JavaScript interpreter at least once per
request, to rereduce the btree inner-nodes that fit in that request
range. This means for group=true requests, the rereduce function must
run once per unique key (at minimum). That would be the source of your
slowness. It sounds like you are building a tag-cloud. The smart money
would be on caching the results of that operation, which is standard
practice with SQL based tag clouds as well.

If you're not doing a tag cloud, maybe there's a way you can get the
needed results using map only?

Also, I'm not sure, but perhaps it would be possible for CouchDB to
cache final reduce values in the btree as well, so that group=true
queries can save the cost of the final rereduce (and make subsequent
queries fast...)

Chris


-- 
Chris Anderson
http://jchris.mfdz.com

Mime
View raw message