couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Dionne <dio...@dionne-associates.com>
Subject Re: Has Erlang's promise of parallelism been realised in CouchDB?
Date Thu, 30 Jun 2011 10:38:55 GMT
Yes, that sounds possible.

 I think one thing that really impacts performance with the view indexer is the storing of
the reductions
in the internal nodes of the btree.



On Jun 30, 2011, at 5:26 AM, Dirkjan Ochtman wrote:

> On Thu, Jun 30, 2011 at 11:21, Robert Newson <rnewson@apache.org> wrote:
>> Individual view building is sequential and it's hard to see how it
>> could be otherwise, given then append-only nature of view files today.
> 
> IIRC JSON encoding/decoding and the process of running the actual view
> functions is a non-trivial part of view building, which could be
> somewhat trivially parallellized at least for the map functions. I.e.,
> if there are 10000 new documents to index, why not start 4 view
> servers and let each of them process 25% of the updated documents? The
> writes will be serialized again, of course, but I didn't think the
> disk writes were the bottleneck for the view indexer?
> 
> Cheers,
> 
> Dirkjan


Mime
View raw message