Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (nike.apache.org: domain of paul.joseph.davis@gmail.com
 designates 74.125.46.31 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type:content-transfer-encoding;
        b=MXbBEI1Sve0Cr3A/HHTCTCzmUjo71EB/HlC3eRYGzdqWuNlEn/fb5A72FzPTQ1tvlI
         pwb2TIuWDpfj9gfJ4HZB1P4S9tqxROPxlS0A188FEIpsiIqPSi6ZW2Lq05eGQozH/oPz
         ZrzO9qyVskJ0qOBjXmy9t6/XB9ixKAaxykm/A=
MIME-Version: 1.0
In-Reply-To: <e2111bbb0905040907y7bb6f455h29097b4e443b6380@mail.gmail.com>
References: <4a29064f0905040450j13b9ffedq505d0fc3e7ac1e5c@mail.gmail.com>
	 <24961DEF-EAFC-4765-AFBB-AF15AA67629D@apache.org>
	 <f728979f0905040723p27eddce5t28c8766a3279594c@mail.gmail.com>
	 <e282921e0905040858m41ca6a67k88eafbf78128bb19@mail.gmail.com>
	 <e2111bbb0905040907y7bb6f455h29097b4e443b6380@mail.gmail.com>
Date: Mon, 4 May 2009 12:08:13 -0400
Message-ID: <e2111bbb0905040908u565fabf6t58e791e2602d37e3@mail.gmail.com>
Subject: Re: couchdb on smp?
From: Paul Davis <paul.joseph.davis@gmail.com>
To: user@couchdb.apache.org
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

Also a point I forgot to make explicit is that the erlview engine
doesn't require any JSON serialization in either direction obviously.
So in the end it'll probably be what you want if you're going for pure
speed anyway.

Paul

On Mon, May 4, 2009 at 12:07 PM, Paul Davis <paul.joseph.davis@gmail.com> w=
rote:
> On Mon, May 4, 2009 at 11:58 AM, Chris Anderson <jchris@apache.org> wrote=
:
>> On Mon, May 4, 2009 at 7:23 AM, Zachary Zolton <zachary.zolton@gmail.com=
> wrote:
>>> @janl
>>>
>>> Perhaps he's asking why there's no activity on the other processor?
>>>
>>> I think his expectation here is that Map-Reduce would be parallelized.
>>> Correct me if I'm wrong, but CouchDB does not yet exploit parallelism
>>> in view indexing yet, right?
>>>
>>
>> Correct. And the JavaScript view server doesn't make it easy to do so
>> without starting multiple OS processes per map-function. I can see
>> this being nifty down the road, but it's an optimization.
>>
>> However, interest is growing in an Erlang view server, and
>> http://github.com/mmcdanie/erlview points out that the view process
>> architecture could use some changes to make Erlang views parallel.
>> Those changes will in turn make it more trivial to parallelize JS view
>> computation, in the cases that need it. So I can see us getting to
>> parallel JS views, but I think the cleaner way to get there is to
>> start with proper Erlang views.
>>
>
> I'd also throw in that the biggest bottleneck for JS views is probably
> the JSON serialization. I've been asked to try getting eep0018 in to a
> copy of trunk as a configuration parameter. I'll get a branch on
> github up by the end of the week to have a reference for that. My gut
> feeling is that once we get that sorted out, we'll probably see that
> disk I/O becomes the bottleneck. I could see maybe eeking out some
> extra performance by streaming data through the map servers as opposed
> to the serialized delegation method we're using now, but then it
> becomes a cost/benfit of complexity. We'll see how it goes.
>
> Paul Davis
>
>>> =97zdzolton
>>>
>>> On Mon, May 4, 2009 at 7:46 AM, Jan Lehnardt <jan@apache.org> wrote:
>>>> Beam and couchjs pipe data back and =A0forth during view generation. W=
hile the
>>>> one works, the other waits. The scheduler is smart enough to keep the
>>>> processes local to a single CPU. Otherwise it's be even more expensive=
.
>>>>
>>>> Cheers
>>>> Jan
>>>> --
>>>>
>>>> On 04.05.2009, at 12:50, Elf <elf2001@gmail.com> wrote:
>>>>
>>>>> Hello.
>>>>> I'm using couchdb-0.9 (erlang 13.2) on my linux server with 4 CPUs
>>>>> (Core 2 Quad).
>>>>> Every time couchdb needs to reindex views, i see 2 serious processes
>>>>> in htop - beam and couchjs. Each of them eats < 100% of 1 cpu, and su=
m
>>>>> of their usage is about (but not greater) 100% of 1 cpu (80/20, 70/30
>>>>> and so on).
>>>>> Can somebody explain, why that 2 different processes (they have
>>>>> differend PIDs) doesn't used different cpus - (one process for cpu)?
>>>>>
>>>>>
>>>>> --
>>>>> ----------------
>>>>> Best regards
>>>>> Elf
>>>>> mailto:elf2001@gmail.com
>>>>>
>>>>
>>>
>>
>>
>>
>> --
>> Chris Anderson
>> http://jchrisa.net
>> http://couch.io
>>
>