incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Krzysztof Kulewski" <kkulew...@student.uw.edu.pl>
Subject Re: map-reduce question: map done on many servers
Date Thu, 01 May 2008 15:54:05 GMT
couch_query_servers.erl is working in such a way: 

% send command and get a response.
prompt(Port, Json) ->
    writeline(Port, cjson:encode(Json)),
    read_json(Port). 

so to obtain many tasks at once in this javascript (or whatever) view server 
I have also to pass more than one request before actualy reading the output. 
Then I can distribute the load in view server. 

Am I right? 

 

Jan Lehnardt napisaƂ(a): 

> Hello Krzysztof,
> On May 1, 2008, at 00:52, Krzysztof Kulewski wrote:
>> Hello,
>> I have a question for which I couldn't find solution on docs  available 
>> online. Please help:
>> Lets suppose that I have one couchdb server storing text documents.  I 
>> want to do some map reduce on it. Map part take very long time per  one 
>> doc, so I want to distribute maps between 20 map servers (btw.  maybe 
>> unreliable). And then somebody will do the reduce part.
>> As I can see, "CouchDB delegates computation of Views to external  query 
>> servers." But I want to use 20 separate machines for doing map  part to 
>> obtain 20x speed-up.
>> How to distribute this map part? Is there any pretty solution out of  the 
>> box?
> 
> This is not yet possible out of the box. You could probably
> achieve this by writing your own view server component
> that distributes the map requests that CouchDB sends in. 
> 
> For a reference view server see:
> http://svn.apache.org/viewvc/incubator/couchdb/trunk/share/server/main.js? 
> revision=645661&view=markup 
> 
> You don't need to write it in JavaScript (the couch.ini lets
> you specify a daemon written in any language. The only
> thing you need to make sure is that the interface to
> CouchDB behaves the same as the original view server. 
> 
> Cheers
> Jan
> --

Mime
View raw message