incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nuno Job" <>
Subject Re: Map/Reduce takes lots of time every request
Date Sat, 22 Nov 2008 20:56:53 GMT
Slightly offtopic: Anyone saw Simon Peyton Jones talking about making a
generic mapreduce-like in haskell?

On Sat, Nov 22, 2008 at 3:53 PM, Chris Anderson <> wrote:

> On Sat, Nov 22, 2008 at 12:25 PM, maddiin <> wrote:
> >
> > Do you have any advice what I am doing wrong and how I could speed this
> up?
> I'm curious how long it takes with reduce=false (should be limited
> basically by IO).
> I'm almost certain (please correct me if I'm wrong) that reduce
> requests must call the JavaScript interpreter at least once per
> request, to rereduce the btree inner-nodes that fit in that request
> range. This means for group=true requests, the rereduce function must
> run once per unique key (at minimum). That would be the source of your
> slowness. It sounds like you are building a tag-cloud. The smart money
> would be on caching the results of that operation, which is standard
> practice with SQL based tag clouds as well.
> If you're not doing a tag cloud, maybe there's a way you can get the
> needed results using map only?
> Also, I'm not sure, but perhaps it would be possible for CouchDB to
> cache final reduce values in the btree as well, so that group=true
> queries can save the cost of the final rereduce (and make subsequent
> queries fast...)
> Chris
> --
> Chris Anderson

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message