couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Davis <>
Subject Re: Proposal for changes in view server/protocol
Date Mon, 02 Aug 2010 20:11:22 GMT
On Mon, Aug 2, 2010 at 2:49 PM, Mikeal Rogers <> wrote:
> Back up a few steps.
> The view server line protocol while being terribly defined and taking
> breaking changes in almost every release has still maintained a fair amount
> of alternate implementations in languages other than JavaScript. I seriously
> doubt that we would see this kind of diverse additions to CouchDB if the
> barrier to integrating was writing erlang. We shouldn't fix fundamental
> problems with the view server protocol and architecture solely by moving it
> in to erlang.

Even if we moved to a JS NIF runtime, we should still support the line
protocol so that other languages can be used. Internally it would
require some refactoring, but I think this is completely reasonable.
And the refactoring probably needs to happen regardless.

> I'm also *highly* skeptical of the perceived performance benefits of moving
> the js in to erlang compared to moving away from full JSON parsing and just
> doing selective parsing/validation and storing the JSON string as opposed to
> the native erlang representation. I can write this idea up in greater detail
> in another thread but the native JSON parser in *any* language is likely to
> be faster than something that parses the native erlang terms so it's only
> faster on the erlang side to not hit the serialization again which we would
> get if we stored the string+parsed underscore properties instead of the full
> native erlang terms.

I'm also *highly* skeptical that storing the string and selective
parsing is going to do anything but complicate things even further. :D

I'm not sure what you're comparing the native JSON parser too, but I
can say that parsing Erlang terms is much easier and faster than JSON.
Although, what's even *more* faster, is not using JSON at all. In the
code that runs the NIF stuff I translate directly from Erlang terms to
Spidermonkey objects programattically, so there's zero
serialization/deserialization overhead. I haven't timed this code
against couchjs yet, but I did a similar thing in passing Erlang terms
to the view server and going from the external format to JavaScript
objects directly which gave a noticeable speed up IIRC. It just
happened to add about 1K lines of code that didn't seem like something
I could require of other view implementations.

> Not to mention the security and durability considerations of moving the view
> server in to erlang. There is a reason erlang views aren't enable by
> default.

The reason that Erlang isn't enabled by default is because its not
sandbox-able. Or, more specifically, I couldn't figure out how to
sandbox it. JS as a NIF is so sandboxed I still can't figure out how
to make it communicate at all with Erlang without returning to the top
of the stack.

> -Mikeal
> On Mon, Aug 2, 2010 at 11:39 AM, Jason Smith <> wrote:
>> On Tue, Aug 3, 2010 at 01:35, Randall Leeds <>
>> wrote:
>> > Either way we're waiting for a response from couch to call back into
>> > JS, it's just whether we're calling the original function or the
>> > continuation.
>> > I don't have a strong preference either way, but I do slightly favor
>> > the callback continuation as a mild style preference.
>> >
>> I have to say, callback style is way nicer. I have not thought about how
>> that would look from non-Javascript query servers though.
>> > In either case, do we have to add some new response format so the
>> > query server can say "Hey call me again and tell me what to do with
>> > this result?"
>> > In other words, both of our suggestions can work on top of the same
>> > change that needs to happen to the view server line protocol, so you
>> > can code up either one and we can settle on what it should look like
>> > to people writing update handlers later, no?
>> >
>> I suppose so. But I am unclear because earlier in the thread I posted
>> working code that allows me to handle save conflicts (or validation
>> failures) from within javascript, with no change to the line protocol.
>> (That
>> is about all it has going for it in fact!)
>> --
>> Jason Smith
>> Couchio Hosting

View raw message