incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dean Landolt" <d...@deanlandolt.com>
Subject Re: Getting only updates from a view
Date Mon, 27 Oct 2008 17:37:59 GMT
On Sun, Oct 26, 2008 at 11:54 PM, kowsik <kowsik@gmail.com> wrote:

> The view server (couchjs) today is primarily a local process
> interacting with couch through stdin/stdout. This is the one that does
> all the map/reduce. I can envision at some point (not sure if and when
> it's planned), the view servers executing on remote machines through
> the exact same line-based JSON protocol (though using TCP) to heavily
> parallelize the map/reduce process. The incremental set of documents
> to be indexed can be easily chunked and distributed across this
> cluster. Until that's possible, the single machine solution is going
> to require some serious compute power (and lots of memory).
>
> IMHO, this [potential] parallelization (in addition to all the
> benefits of document-centric storage) is what makes couch incredibly
> attractive from a scaling perspective.


>From what I've gleaned from blog posts and other reading, distributed
map/reduce is in the roadmap, but implemented in a more couchy manner (over
http). Partial replication is one approach I've heard bantied about -- and
once that lands, depending on how the API looks, it should be easy to shard
documents based on some partition function (or some such) and assemble the
results. There may even be a more explicit distributed solution in the
works, but I doubt it would use pipes for anything other than local
communication.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message