Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@couchdb.apache.org
Received-SPF: pass (nike.apache.org: domain of paul.joseph.davis@gmail.com
 designates 209.85.214.180 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:from:date:message-id:subject:to
         :content-type;
        b=D7GyULRUNX3q7l7EBCQSX1eZbYGO6bvsB1xqhvZzxAKRKD4knwTd2Fl7mG8zft4hs8
         bdlXXFqfY9DRvTyowSgSQmipeZykpD3IVVDJfrcNvlTYbj3uARqUIANaY7gHeIiLlpGr
         6FWEj80io+Xt4dLG+vGQCnsMQA5+fxUD37tsw=
MIME-Version: 1.0
In-Reply-To: <AANLkTimujNeUyEvBKxvQnchUNf_Tgxmjc5c6Ew62AHQo@mail.gmail.com>
References: <AANLkTin=PhtkbQX7SJSoDAgcs3EJ+VZ_BBsKb2fukwnr@mail.gmail.com>
	<AANLkTinte65oBBY1iDhjaV2e4QOrHAs60d0ZBqPb+Hst@mail.gmail.com>
	<AANLkTik1E-NzesS3hxO7zdtSdtM2eVscpcSULOud11Jj@mail.gmail.com>
	<AANLkTimfXM9YM1+TLRuF_SgD_LRhw+Z20vC9bmfg7ikv@mail.gmail.com>
	<AANLkTinF3Ypzk5ujTwD5mr=8v7sJcKkXbTjBAJ_GEDMC@mail.gmail.com>
	<AANLkTinbuVM+bK3=KgqcckgoJM9n1MjRfO3eUW0mmNWg@mail.gmail.com>
	<AANLkTinobdPn+k8fWFmS3OxDBQLALfn=x+9GkLaicsvP@mail.gmail.com>
	<AANLkTintbE3qZkK4c3Xz---fU35tnEh22r2Hib+9+MoJ@mail.gmail.com>
	<AANLkTi=kazQfTsYicZbiKTcqovy9d4Fg5=SswTdAW627@mail.gmail.com>
	<AANLkTinVafxTLYjvyEPRgRPgrwG1a2hzOuBRuo-xGOzM@mail.gmail.com>
	<AANLkTikZ0aYZim01gEFaQOeXgZYcxST074sa5yvF6D_q@mail.gmail.com>
	<AANLkTimDRfYfV3ZhOrD69VoWom0tCbr+c93zVwWkQ2yV@mail.gmail.com>
	<AANLkTimwq+0=CZG_YdJpX_=2V9kVQymkOhQBub7wYPet@mail.gmail.com>
	<AANLkTimujNeUyEvBKxvQnchUNf_Tgxmjc5c6Ew62AHQo@mail.gmail.com>
From: Paul Davis <paul.joseph.davis@gmail.com>
Date: Mon, 2 Aug 2010 16:11:22 -0400
Message-ID: <AANLkTincb6PdwHtPYC7i2ZX25A+7qf2f7tvh3gEUytnr@mail.gmail.com>
Subject: Re: Proposal for changes in view server/protocol
To: dev@couchdb.apache.org
Content-Type: text/plain; charset=ISO-8859-1

On Mon, Aug 2, 2010 at 2:49 PM, Mikeal Rogers <mikeal.rogers@gmail.com> wrote:
> Back up a few steps.
>
> The view server line protocol while being terribly defined and taking
> breaking changes in almost every release has still maintained a fair amount
> of alternate implementations in languages other than JavaScript. I seriously
> doubt that we would see this kind of diverse additions to CouchDB if the
> barrier to integrating was writing erlang. We shouldn't fix fundamental
> problems with the view server protocol and architecture solely by moving it
> in to erlang.

Even if we moved to a JS NIF runtime, we should still support the line
protocol so that other languages can be used. Internally it would
require some refactoring, but I think this is completely reasonable.
And the refactoring probably needs to happen regardless.

> I'm also *highly* skeptical of the perceived performance benefits of moving
> the js in to erlang compared to moving away from full JSON parsing and just
> doing selective parsing/validation and storing the JSON string as opposed to
> the native erlang representation. I can write this idea up in greater detail
> in another thread but the native JSON parser in *any* language is likely to
> be faster than something that parses the native erlang terms so it's only
> faster on the erlang side to not hit the serialization again which we would
> get if we stored the string+parsed underscore properties instead of the full
> native erlang terms.

I'm also *highly* skeptical that storing the string and selective
parsing is going to do anything but complicate things even further. :D

I'm not sure what you're comparing the native JSON parser too, but I
can say that parsing Erlang terms is much easier and faster than JSON.
Although, what's even *more* faster, is not using JSON at all. In the
code that runs the NIF stuff I translate directly from Erlang terms to
Spidermonkey objects programattically, so there's zero
serialization/deserialization overhead. I haven't timed this code
against couchjs yet, but I did a similar thing in passing Erlang terms
to the view server and going from the external format to JavaScript
objects directly which gave a noticeable speed up IIRC. It just
happened to add about 1K lines of code that didn't seem like something
I could require of other view implementations.

> Not to mention the security and durability considerations of moving the view
> server in to erlang. There is a reason erlang views aren't enable by
> default.
>

The reason that Erlang isn't enabled by default is because its not
sandbox-able. Or, more specifically, I couldn't figure out how to
sandbox it. JS as a NIF is so sandboxed I still can't figure out how
to make it communicate at all with Erlang without returning to the top
of the stack.

> -Mikeal
>
>
> On Mon, Aug 2, 2010 at 11:39 AM, Jason Smith <jhs@couch.io> wrote:
>
>> On Tue, Aug 3, 2010 at 01:35, Randall Leeds <randall.leeds@gmail.com>
>> wrote:
>>
>> > Either way we're waiting for a response from couch to call back into
>> > JS, it's just whether we're calling the original function or the
>> > continuation.
>> > I don't have a strong preference either way, but I do slightly favor
>> > the callback continuation as a mild style preference.
>> >
>>
>> I have to say, callback style is way nicer. I have not thought about how
>> that would look from non-Javascript query servers though.
>>
>>
>> > In either case, do we have to add some new response format so the
>> > query server can say "Hey call me again and tell me what to do with
>> > this result?"
>> > In other words, both of our suggestions can work on top of the same
>> > change that needs to happen to the view server line protocol, so you
>> > can code up either one and we can settle on what it should look like
>> > to people writing update handlers later, no?
>> >
>>
>> I suppose so. But I am unclear because earlier in the thread I posted
>> working code that allows me to handle save conflicts (or validation
>> failures) from within javascript, with no change to the line protocol.
>> (That
>> is about all it has going for it in fact!)
>>
>> --
>> Jason Smith
>> Couchio Hosting
>>
>