incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Davis <paul.joseph.da...@gmail.com>
Subject Re: couchdb on smp?
Date Mon, 04 May 2009 16:08:13 GMT
Also a point I forgot to make explicit is that the erlview engine
doesn't require any JSON serialization in either direction obviously.
So in the end it'll probably be what you want if you're going for pure
speed anyway.

Paul

On Mon, May 4, 2009 at 12:07 PM, Paul Davis <paul.joseph.davis@gmail.com> wrote:
> On Mon, May 4, 2009 at 11:58 AM, Chris Anderson <jchris@apache.org> wrote:
>> On Mon, May 4, 2009 at 7:23 AM, Zachary Zolton <zachary.zolton@gmail.com> wrote:
>>> @janl
>>>
>>> Perhaps he's asking why there's no activity on the other processor?
>>>
>>> I think his expectation here is that Map-Reduce would be parallelized.
>>> Correct me if I'm wrong, but CouchDB does not yet exploit parallelism
>>> in view indexing yet, right?
>>>
>>
>> Correct. And the JavaScript view server doesn't make it easy to do so
>> without starting multiple OS processes per map-function. I can see
>> this being nifty down the road, but it's an optimization.
>>
>> However, interest is growing in an Erlang view server, and
>> http://github.com/mmcdanie/erlview points out that the view process
>> architecture could use some changes to make Erlang views parallel.
>> Those changes will in turn make it more trivial to parallelize JS view
>> computation, in the cases that need it. So I can see us getting to
>> parallel JS views, but I think the cleaner way to get there is to
>> start with proper Erlang views.
>>
>
> I'd also throw in that the biggest bottleneck for JS views is probably
> the JSON serialization. I've been asked to try getting eep0018 in to a
> copy of trunk as a configuration parameter. I'll get a branch on
> github up by the end of the week to have a reference for that. My gut
> feeling is that once we get that sorted out, we'll probably see that
> disk I/O becomes the bottleneck. I could see maybe eeking out some
> extra performance by streaming data through the map servers as opposed
> to the serialized delegation method we're using now, but then it
> becomes a cost/benfit of complexity. We'll see how it goes.
>
> Paul Davis
>
>>> —zdzolton
>>>
>>> On Mon, May 4, 2009 at 7:46 AM, Jan Lehnardt <jan@apache.org> wrote:
>>>> Beam and couchjs pipe data back and  forth during view generation. While
the
>>>> one works, the other waits. The scheduler is smart enough to keep the
>>>> processes local to a single CPU. Otherwise it's be even more expensive.
>>>>
>>>> Cheers
>>>> Jan
>>>> --
>>>>
>>>> On 04.05.2009, at 12:50, Elf <elf2001@gmail.com> wrote:
>>>>
>>>>> Hello.
>>>>> I'm using couchdb-0.9 (erlang 13.2) on my linux server with 4 CPUs
>>>>> (Core 2 Quad).
>>>>> Every time couchdb needs to reindex views, i see 2 serious processes
>>>>> in htop - beam and couchjs. Each of them eats < 100% of 1 cpu, and
sum
>>>>> of their usage is about (but not greater) 100% of 1 cpu (80/20, 70/30
>>>>> and so on).
>>>>> Can somebody explain, why that 2 different processes (they have
>>>>> differend PIDs) doesn't used different cpus - (one process for cpu)?
>>>>>
>>>>>
>>>>> --
>>>>> ----------------
>>>>> Best regards
>>>>> Elf
>>>>> mailto:elf2001@gmail.com
>>>>>
>>>>
>>>
>>
>>
>>
>> --
>> Chris Anderson
>> http://jchrisa.net
>> http://couch.io
>>
>

Mime
View raw message