Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
MIME-Version: 1.0
In-Reply-To: 
 <CACLy94XC=6QFk=FxE+c9Wh41G9C_KosMk8j-tn9-_gia8outRQ@mail.gmail.com>
References: 
 <CAPNuaCjQ3wgAd1BMyNUtip0rztK6n3Ex3rnq-hc=NXwL4Wt+pg@mail.gmail.com>
	<CAG+HO1ysanHLAYTh5XXuN=WO3RMsjxsY9c5nXjGiHy1n_m74mA@mail.gmail.com>
	<CAPNuaCjTS1+GAqx=WG0qjiP+3Rc06bXbSFDc9jdvHOOZwWY+3g@mail.gmail.com>
	<CAG+HO1x+L2rKTXaVcuu2_n0FQW_5CWZ7Jk5qo96-8OK-sRU7XA@mail.gmail.com>
	<CAPNuaCh5qK2YxS0txBicvrc4fnQCgrj5pVz=_M0xqXaKfQyYjg@mail.gmail.com>
	<CAG+HO1w+N_TxeroFpKwY+U=G2iC6eEU2QMDbV9aGgueDsZ5Bcg@mail.gmail.com>
	<CACLy94Uq9eafd_Q-LXV8b3aCipnHgMnaWdn2C1LjnvuMQ-csiA@mail.gmail.com>
	<CAG+HO1zySzJyQMnSj9hfeHVR-XNB2Hpqr97A5ugabvo-wGKdXg@mail.gmail.com>
	<CACLy94XLwHcDyP12igsvCPJ5p46=3=HdoW3xAz_+2RgXH6s=pA@mail.gmail.com>
	<CACLy94Vf6TOz80cM_gWn26nFvMe8A4uQwQNNZQSBRBgYzXB9zg@mail.gmail.com>
	<CAPNuaCiQV2xoDj2O5vg+a-pDzj5ywCwmc+To16ybLeUh4Uc4eA@mail.gmail.com>
	<CACLy94UZtdV+nUUQzoYTiuUP3L=cLy-OjTUC_+6FdBNyAp2fhA@mail.gmail.com>
	<CAPNuaCjavr_udkm_XzO3SMU9gKMfdNnCM1cC71MUQhmtmP5scQ@mail.gmail.com>
	<CACLy94XC=6QFk=FxE+c9Wh41G9C_KosMk8j-tn9-_gia8outRQ@mail.gmail.com>
Date: Thu, 15 Aug 2013 09:09:55 +0100
Message-ID: 
 <CABvT1DHPsTi=4WwwhTysmZkOda-R6FqsA10NXFyYTUQJR5wGJQ@mail.gmail.com>
Subject: Re: Erlang vs JavaScript
From: Robert Newson <rnewson@apache.org>
To: "user@couchdb.apache.org" <user@couchdb.apache.org>
Content-Type: text/plain; charset=ISO-8859-1

A big +1 to Jason's clarification of "erlang" vs "native". CouchDB
could have shipped an erlang view server that worked in a separate
process and had the stdio overhead, to combine the slowness of the
protocol with the obtuseness of erlang. ;)

Evaluating Javascript within the erlang VM process intrigues me, Jens,
how is that done in your case? I've not previously found the assertion
that V8 would be faster than SpiderMonkey for a view server compelling
since the bottleneck is almost never in the code evaluation, but I do
support CouchDB switching to it for the synergy effects of a closer
binding with node.js, but if it's running in the same process, that
would change (though I don't immediately see why the same couldn't be
done for SpiderMonkey). Off the top of my head, I don't know a safe
way to evaluate JS in the VM. A NIF-based approach would either be
quite elaborate or would trip all the scheduling problems that
long-running NIF's are now notorious for.

At a step removed, the view server protocol itself seems like the
thing to improve on, it feels like that's the principal bottleneck.

B.


On 15 August 2013 08:22, Jason Smith <jhs@apache.org> wrote:
> Yes, to a first approximation, with a native view, CouchDB is basically
> running eval() on your code. In my example, I took advantage of this to
> build a nonstandard response to satisfy an application. (Instead of a 404,
> we sent a designated fallback document body.)
>
> But, if you accumulate the list in a native view, a JavaScript view, or a
> hypothetical Erlang view (i.e. a subprocess), from the operating system's
> perspective, the memory for that list will be allocated somewhere. Either
> the CouchDB process asks for X KB more memory, or its subprocess will ask
> for it. So I think the total system impact is probably low in practice.
>
> So I guess my point is not that native views are wrong, just they have a
> cost so you should weigh the cost/benefit for your own project. In the case
> of manage_couchdb, I wrote a JavaScript implementation; but since sometimes
> I have an emergency and I must find conflicts ASAP, I made an Erlang
> version because it is worth it.
>
>
> On Thu, Aug 15, 2013 at 2:05 PM, Stanley Iriele <siriele2x3@gmail.com>wrote:
>
>> Whoa...OK...that I had no idea about...thanks for taking the time to go to
>> that granularity, by the way.
>>
>> So does this mean that the process memory is shared? As apposed to living
>> in its own space?.so if someone accumulates a large json object in a list
>> function its chewing up couchdb's memory?... I guess I'm a little confused
>> about what's in the same process and what isn't now
>> On Aug 14, 2013 11:57 PM, "Jason Smith" <jhs@apache.org> wrote:
>>
>> > To me, an Erlang view is a view server which supports map, reduce, show,
>> > update, list, etc. functions in the Erlang language. (Basically it is
>> > implemented in Erlang.)
>> >
>> > A view server is a subprocess that runs beneath CouchDB which
>> communicates
>> > with it over standard i/o. It is a different process in the operating
>> > system and only interfaces with the main server using the view server
>> > protocol (basically a bunch of JSON messages going back and forth).
>> >
>> > I do not know of an Erlang view server which works well and is currently
>> > maintained.
>> >
>> > A native view (shipped by CouchDB but disabled by default) is some
>> > corner-cutting. Code is evaluated directly by the primary CouchDB server.
>> > Since CouchDB is Erlang, the native query server is necessarily Erlang.
>> The
>> > key difference is, your code is right there in the eye of the storm. You
>> > can call couch_server:open("some_db") and completely circumvent security
>> > and other invariants which CouchDB enforces. You can leak memory until
>> the
>> > kernel OOM killer terminates CouchDB. It's not about the language, it's
>> > that is is running inside the CouchDB process.
>> >
>> >
>> >
>> > On Thu, Aug 15, 2013 at 1:36 PM, Stanley Iriele <siriele2x3@gmail.com
>> > >wrote:
>> >
>> > > Wait....I'm a tad confused here..Jason what is the difference between
>> > > native views and Erlang views?...
>> > > On Aug 14, 2013 11:16 PM, "Jason Smith" <jhs@apache.org> wrote:
>> > >
>> > > > Oh, also:
>> > > >
>> > > > They are **not** Erlang views. They are **native** views. We should
>> > > > emphasize the latter to remind ourselves about the security and
>> > > reliability
>> > > > risks which Bob identifies.
>> > > >
>> > > > They are very powerful, but it is a trade-off. Once I had a customer
>> > who
>> > > > had a basic "class" document describing common values. All other
>> > > documents
>> > > > were for modifications to the "base class" so to speak. He needed to
>> > > query
>> > > > by document ID, but if no such document existed, return the "base
>> > class"
>> > > > document instead. The product was already in the field and so the
>> code
>> > > > could not change. We had to change it in CouchDB.
>> > > >
>> > > > The fix was very simple: a _rewrite rule to a native _show function.
>> In
>> > > the
>> > > > show function, if the Doc was null, then we used the internal CouchDB
>> > API
>> > > > to fetch the default document. Voila.
>> > > >
>> > > >
>> > > > On Thu, Aug 15, 2013 at 1:08 PM, Jason Smith <jhs@apache.org> wrote:
>> > > >
>> > > > > On Thursday, August 15, 2013, Andrey Kuprianov wrote:
>> > > > >
>> > > > >> Doesnt server performance downgrade, while views are being
>> rebuilt?
>> > So
>> > > > the
>> > > > >> faster they are rebuilt, the better for you.
>> > > > >
>> > > > >
>> > > > > If my view build would degrade total performance to cross an
>> > > unacceptable
>> > > > > threshold, then I am really riding the line! What about an
>> unplanned
>> > > > > compaction? What if one day the clients have a bug and load
>> > increases?
>> > > > What
>> > > > > if an unplanned disaster happens and a backup must be performed
>> > > urgently?
>> > > > >
>> > > > > I would evaluate view performance in the larger context of the
>> entire
>> > > > > application life cycle.
>> > > > >
>> > > > > Men seem to want to date beautiful women. It is a very high
>> priority
>> > at
>> > > > > the pub or whatever. But long-married men do not even think about
>> > their
>> > > > > wife's attractiveness because that is a small, superficial part of
>> a
>> > > much
>> > > > > larger story.
>> > > > >
>> > > > >
>> > > > >>
>> > > > >> Besides, looks like it's possible to do the same 3 steps with
>> design
>> > > doc
>> > > > >> views created in Erlang? Or is it just about using require() in
>> > > Node.js?
>> > > > >>
>> > > > >
>> > > > > Actually, yes that is a fine point. I myself prefer Node.js but
>> > anyone
>> > > > can
>> > > > > choose the best fit for them.
>> > > > >
>> > > > > And speaking more broadly, CouchDB is a very flexible platform so
>> it
>> > is
>> > > > > quite likely that my own policies do not apply to every use case.
>> In
>> > > fact
>> > > > > if I'm honest I use native views myself, usually for unplanned
>> > > > > troubleshooting, I want to find conflicts so I use manage_couchdb:
>> > > > > http://github.com/iriscouch/manage_couchdb
>> > > > >
>> > > > > My main point is, anybody time somebody says "performance" ask
>> > yourself
>> > > > if
>> > > > > it is really a "performance siren." Earlier in this thread, Jens
>> > raises
>> > > > > some examples of plausible true performance requirements, not just
>> > > siren
>> > > > > songs.
>> > > > >
>> > > >
>> > >
>> >
>>