couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Newson <rnew...@apache.org>
Subject Re: Has Erlang's promise of parallelism been realised in CouchDB?
Date Thu, 30 Jun 2011 09:21:26 GMT
Individual view building is sequential and it's hard to see how it
could be otherwise, given then append-only nature of view files today.

To be honest, this constraint is only of note in a single database,
single view scenario, which I don't think is likely, or even
interesting, and certainly not worth optimizing for.

I don't believe reads are parallelized either, since the file
descriptor is held by a gen_server and all calls to a gen_server are
serialized.

Where Erlang shines is the ability to have many requests processed
sanely at the same time, it just happens to need to be to different
databases and views in CouchDB. The reason reads seem fast, and why
you might think they are parallel, is because of disk caching effects.

B.

On 30 June 2011 10:16, Andrew Stuart (SuperCoders)
<andrew.stuart@supercoders.com.au> wrote:
> I imagine that if anything needs the performance it is the view indexer?
>
> On 30/06/2011, at 6:06 PM, Dirkjan Ochtman wrote:
>
> On Thu, Jun 30, 2011 at 09:53, Randall Leeds <randall.leeds@gmail.com>
> wrote:
>>
>> But beyond the parallelization of request handling there's concurrency
>> in the more general sense. The neat thing about Erlang, and why it has
>> its reputation, is that the CouchDB code can be liberal about its use
>> of concurrency at the code level without suffering from deadlocks or
>> other headaches that often plague programmers of complex,
>> multi-threaded shared memory systems. The Erlang team has taken care
>> of all the hard parts about sharing data in a concurrent environment.
>> As I understand it, the Erlang runtime's use of chipsets with many
>> hardware threads is only improving, and those benefits will be
>> automatically conferred upon CouchDB.
>
> One thing that, as far as I know, has not been parallellized is the
> view indexer. While it should be possible to execute at least the map
> part of map/reduce concurrently, CouchDB doesn't do that yet. IIRC
> there were reasons for that? But at least it's good to be aware of.
>
> Cheers,
>
> Dirkjan
> --
> Message  protected by MailGuard: e-mail anti-virus, anti-spam and content
> filtering.http://www.mailguard.com.au/mg
> Click here to report this message as spam:
> https://login.mailguard.com.au/report/1CCaMoXsmg/2UfOi5VJGXiukwyBDiHnm2/1
>

Mime
View raw message