couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Cortland Klein ...@pixelcort.com>
Subject Re: map-reduce question: map done on many servers
Date Thu, 01 May 2008 05:52:34 GMT
I'm also curious how this will play out over time.

Will we have clusters of peer CouchDB servers that have one View  
Controller each or one CouchDB server with many mapreduce controllers?

I'm thinking in big picture there are two scalability issues:

1. Huge datasets. The concept of sharding would apply here.
2. Complex maps. In some cases it might be faster to do these on  
multiple machines with one CouchDB server.

I'm not one to suggest any architectural solutions at this point, just  
thinking out loud at this point.

Cortland Klein <me@pixelcort.com> +1 408 506 9791
http://pixelcort.com/

Sent from my iPhone

On Apr 30, 2008, at 3:52 PM, "Krzysztof Kulewski" <kkulewski@student.uw.edu.pl 
 > wrote:

> Hello,
> I have a question for which I couldn't find solution on docs  
> available online. Please help:
> Lets suppose that I have one couchdb server storing text documents.  
> I want to do some map reduce on it. Map part take very long time per  
> one doc, so I want to distribute maps between 20 map servers (btw.  
> maybe unreliable). And then somebody will do the reduce part.
> As I can see, "CouchDB delegates computation of Views to external  
> query servers." But I want to use 20 separate machines for doing map  
> part to obtain 20x speed-up.
> How to distribute this map part? Is there any pretty solution out of  
> the box?
> BR,
> Krzysztof

Mime
View raw message