incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Damien Katz <dam...@apache.org>
Subject Re: Reduce is Really Slow!
Date Tue, 19 Aug 2008 23:21:02 GMT
I think the problem with your reduce is that it looks like its not  
actually reducing to a single value, but instead using reduce for  
grouping data. That will cause severe performance problems.

For reduce to work properly, you should end up with a fixed size data  
structure regardless of the number of values being reduced (not  
stricty true, but that's the general rule).

-Damien

On Aug 19, 2008, at 6:55 PM, Nicholas Retallack wrote:

> Okay, I got it built on gentoo instead, but I'm still having  
> performance
> issues with reduce.
>
> Erlang (BEAM) emulator version 5.6.3 [source] [64-bit] [async- 
> threads:0]
> couchdb - Apache CouchDB 0.8.1-incubating
>
> Here's a query I tried to do:
>
> I freshly imported about 191MB of data in 155399 documents.  29090  
> are not
> discarded by map.  Map produces one row with 5 fields for each of  
> these
> documents.  After grouping, each group should have four rows.   
> Reduce is a
> simple function(keys,values){return values}.
>
> Here's the query call:
> time curl -X GET '
> http://localhost:5984/clickfund/_view/offers/index?count=1&group=true&group_level=1
> '
>
> This is running on a 512MB slicehost account.  http://www.slicehost.com/
>
> I'd love to give you this command's execution time, since I ran it  
> last
> night before I went to bed, but it must have taken over an hour  
> because my
> laptop went to sleep and severed the connection.  Trying it again.
>
> Considering it's blazing fast without the reduce function, I can  
> only assume
> what's taking all this time is overhead setting up and tearing down  
> the
> simple function(keys,values){return values}.
>
> I can give you guys the python source to set up this database so you  
> can try
> it yourself if you like.


Mime
View raw message