couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jan Lehnardt <...@apache.org>
Subject Re: map-reduce question: map done on many servers
Date Thu, 01 May 2008 13:01:00 GMT
Hello Krzysztof,
On May 1, 2008, at 00:52, Krzysztof Kulewski wrote:
> Hello,
> I have a question for which I couldn't find solution on docs  
> available online. Please help:
> Lets suppose that I have one couchdb server storing text documents.  
> I want to do some map reduce on it. Map part take very long time per  
> one doc, so I want to distribute maps between 20 map servers (btw.  
> maybe unreliable). And then somebody will do the reduce part.
> As I can see, "CouchDB delegates computation of Views to external  
> query servers." But I want to use 20 separate machines for doing map  
> part to obtain 20x speed-up.
> How to distribute this map part? Is there any pretty solution out of  
> the box?

This is not yet possible out of the box. You could probably
achieve this by writing your own view server component
that distributes the map requests that CouchDB sends in.

For a reference view server see:
http://svn.apache.org/viewvc/incubator/couchdb/trunk/share/server/main.js?revision=645661&view=markup

You don't need to write it in JavaScript (the couch.ini lets
you specify a daemon written in any language. The only
thing you need to make sure is that the interface to
CouchDB behaves the same as the original view server.

Cheers
Jan
--

Mime
View raw message