couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From maddiin <madd...@googlemail.com>
Subject Re: Map/Reduce takes lots of time every request
Date Sun, 23 Nov 2008 17:55:08 GMT
I am using a permanent view in my app, to make sure I tried with curl -X 
GET "http://localhost:5984/testdb/_view/posts/all_tags?group=true" with 
same results.

I recently installed otp_src_R12B-5, maybe it is misconfigured?
Erlang (BEAM) emulator version 5.6.5 [source] [async-threads:0] [hipe] 
[kernel-poll:false]


Damien Katz schrieb:
> It sounds like ever you are using a temp view, or an older version of 
> Erlang (R11 I think) that has performance problems.
>
> -Damien
>
> On Nov 22, 2008, at 4:20 PM, maddiin wrote:
>
>> > It sounds like you are building a tag-cloud.
>>
>> That is right.
>>
>>
>> > I'm curious how long it takes with reduce=false
>>
>> With reduce=False it takes around 50 seconds to load all tags.
>> 1000 tags are loading in 2-3 seconds with reduce=False.
>>
>> With reduce=True it takes around 80 seconds to load all tags.
>> 1000 tags are loading in 8-9 seconds with reduce=True.
>>
>>
>> > The smart money would be on caching the results of that operation, 
>> which is standard practice with SQL based tag clouds as well.
>>
>> I guess I will have to unless..
>>
>> > Also, I'm not sure, but perhaps it would be possible for CouchDB to 
>> cache final reduce values in the btree as well, so that group=true 
>> queries can save the cost of the final rereduce (and make subsequent 
>> queries fast...).
>>
>> That'd be nice.
>>
>>
>> Thanks for the response,
>> maddiin
>>
>>
>> Chris Anderson schrieb:
>>> On Sat, Nov 22, 2008 at 12:25 PM, maddiin <maddiin@googlemail.com> 
>>> wrote:
>>>
>>>> Do you have any advice what I am doing wrong and how I could speed 
>>>> this up?
>>>>
>>>
>>>
>>> I'm curious how long it takes with reduce=false (should be limited
>>> basically by IO).
>>>
>>> I'm almost certain (please correct me if I'm wrong) that reduce
>>> requests must call the JavaScript interpreter at least once per
>>> request, to rereduce the btree inner-nodes that fit in that request
>>> range. This means for group=true requests, the rereduce function must
>>> run once per unique key (at minimum). That would be the source of your
>>> slowness. It sounds like you are building a tag-cloud. The smart money
>>> would be on caching the results of that operation, which is standard
>>> practice with SQL based tag clouds as well.
>>>
>>> If you're not doing a tag cloud, maybe there's a way you can get the
>>> needed results using map only?
>>>
>>> Also, I'm not sure, but perhaps it would be possible for CouchDB to
>>> cache final reduce values in the btree as well, so that group=true
>>> queries can save the cost of the final rereduce (and make subsequent
>>> queries fast...)
>>>
>>> Chris
>>>
>>>
>>>
>>
>
>


Mime
View raw message