Mailing-List: contact couchdb-user-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: couchdb-user@incubator.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
Message-Id: <55F937E1-4A5D-4FFD-A7D4-4DF2AD8719F9@apache.org>
From: Jan Lehnardt <jan@apache.org>
To: couchdb-user@incubator.apache.org
In-Reply-To: <courier.000000004818F83D.00005635@apollo2.poczta.uw.edu.pl>
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
Content-Transfer-Encoding: 7bit
Mime-Version: 1.0 (Apple Message framework v919.2)
Subject: Re: map-reduce question: map done on many servers
Date: Thu, 1 May 2008 15:01:00 +0200
References: <courier.000000004818F83D.00005635@apollo2.poczta.uw.edu.pl>

Hello Krzysztof,
On May 1, 2008, at 00:52, Krzysztof Kulewski wrote:
> Hello,
> I have a question for which I couldn't find solution on docs  
> available online. Please help:
> Lets suppose that I have one couchdb server storing text documents.  
> I want to do some map reduce on it. Map part take very long time per  
> one doc, so I want to distribute maps between 20 map servers (btw.  
> maybe unreliable). And then somebody will do the reduce part.
> As I can see, "CouchDB delegates computation of Views to external  
> query servers." But I want to use 20 separate machines for doing map  
> part to obtain 20x speed-up.
> How to distribute this map part? Is there any pretty solution out of  
> the box?

This is not yet possible out of the box. You could probably
achieve this by writing your own view server component
that distributes the map requests that CouchDB sends in.

For a reference view server see:
http://svn.apache.org/viewvc/incubator/couchdb/trunk/share/server/main.js?revision=645661&view=markup

You don't need to write it in JavaScript (the couch.ini lets
you specify a daemon written in any language. The only
thing you need to make sure is that the interface to
CouchDB behaves the same as the original view server.

Cheers
Jan
--