incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nicholas Retallack" <nickretall...@gmail.com>
Subject Reduce is Really Slow!
Date Tue, 19 Aug 2008 06:27:12 GMT
You may have noticed my old post talking about reduce's weird behavior.
Well, I just tried out a reduce technique on a relatively large database,
and it seems quirks are the least of our problems.

This is the view I used: http://www.friendpaste.com/2AHz3ahr

My database is 78.3MB and contains 77604 documents, about 34000 of which
satisfy doc.Type == 'offer'.  I will do some queries with &count=1 to
minimize download time as a factor.

A normal query on this view takes 4.5 seconds.  That's pretty bad, since I
wanted to use this for a web interface.  If I add group=true&group_level=1
to my query string, it jumps up to 25 seconds!  If I remove the reduce
method from my view, queries take only 44 milliseconds.

I could not have predicted this big a performance problem just from using
reduce.  This makes it pretty unusable as a web interface.  In fact, I might
be better off doing the reduce operation myself in the application layer,
which is not very awesome at all.

Poking further, I see a huge performance problem even with the simplest of
reduce functions, like function(keys,values){return values} for example.  I
couldn't get a benchmark on this one because it took so long to run that
couchdb timed out and killed it.
> curl -X GET http://localhost:5984/clickfund/_view/offers/index
{"error":"error","reason":"{{nocatch,{map_process_error,\"map function timed
out\"}},\n [{couch_query_servers,readline,2},\n
 {couch_query_servers,read_json,1},\n  {couch_query_servers,prompt,2},\n
 {couch_query_servers,'-rereduce\/3-fun-0-',3},\n  {lists,zipwith,3},\n
 {couch_query_servers,rereduce,3},\n
 {couch_view,'-init_group\/4-fun-0-',4},\n
 {couch_btree,'-write_node\/3-lc$^0\/1-0-',3}]}"}

I am using couchdbx 0.8.0  I have been told that 0.8.1 improves on
javascript views, but I was unable to get it working on my mac.  It compiles
and runs, but when I try to connect to it I am refused.  This is strange,
because Couchdbx works fine when I run it on the same port.

> couchdb
Apache CouchDB 0.9.0a686685-incubating (LogLevel=info)
Apache CouchDB is starting.

Config Info /usr/local/etc/couchdb/couch.ini:
CurrentWorkingDir=/Users/nick/hooraycouch
DbRootDir=/usr/local/var/lib/couchdb
BindAddress="127.0.0.1"
Port="5984"
DocumentRoot=/usr/local/share/couchdb/www
LogFile=/usr/local/var/log/couchdb/couch.log
UtilDriverDir=/usr/local/lib/couchdb/erlang/lib/couch-0.9.0a686685-incubating/priv/lib
DbUpdateNotificationProcesses=
FullTextSearchQueryServer=
javascript=/usr/local/bin/couchjs /usr/local/share/couchdb/server/main.js

(on another terminal)
> curl -X GET http://localhost:5984/_all_dbs
curl: (7) couldn't connect to host

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message