couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From maddiin <madd...@googlemail.com>
Subject Re: Map/Reduce takes lots of time every request
Date Sat, 22 Nov 2008 21:20:15 GMT
 > It sounds like you are building a tag-cloud.

That is right.


 > I'm curious how long it takes with reduce=false

With reduce=False it takes around 50 seconds to load all tags.
1000 tags are loading in 2-3 seconds with reduce=False.

With reduce=True it takes around 80 seconds to load all tags.
1000 tags are loading in 8-9 seconds with reduce=True.


 > The smart money would be on caching the results of that operation, 
which is standard practice with SQL based tag clouds as well.

I guess I will have to unless..

 > Also, I'm not sure, but perhaps it would be possible for CouchDB to 
cache final reduce values in the btree as well, so that group=true 
queries can save the cost of the final rereduce (and make subsequent 
queries fast...).

That'd be nice.


Thanks for the response,
maddiin


Chris Anderson schrieb:
> On Sat, Nov 22, 2008 at 12:25 PM, maddiin <maddiin@googlemail.com> wrote:
>   
>> Do you have any advice what I am doing wrong and how I could speed this up?
>>     
>
>
> I'm curious how long it takes with reduce=false (should be limited
> basically by IO).
>
> I'm almost certain (please correct me if I'm wrong) that reduce
> requests must call the JavaScript interpreter at least once per
> request, to rereduce the btree inner-nodes that fit in that request
> range. This means for group=true requests, the rereduce function must
> run once per unique key (at minimum). That would be the source of your
> slowness. It sounds like you are building a tag-cloud. The smart money
> would be on caching the results of that operation, which is standard
> practice with SQL based tag clouds as well.
>
> If you're not doing a tag cloud, maybe there's a way you can get the
> needed results using map only?
>
> Also, I'm not sure, but perhaps it would be possible for CouchDB to
> cache final reduce values in the btree as well, so that group=true
> queries can save the cost of the final rereduce (and make subsequent
> queries fast...)
>
> Chris
>
>
>   


Mime
View raw message