couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jan Lehnardt <...@apache.org>
Subject Re: Branch to switch from SpiderMonkey to Node.js
Date Thu, 31 Jan 2013 15:36:26 GMT
Here is where this becomes *really* interesting to me. I have touched on
this before, but let me expand.

Over the past year I thought hard about what is the best way to spend my
time on Apache CouchDB. I repeatedly came to the conclusion that the
single best thing I can do is encourage contributions. There is no amount
of code I could produce that couldn’t be out-coded by three people I get
to contribute and take ownership. See Fauxton as an example.

Of course that is just me, anyone here can decide how to spend their time,
and I will not refrain from working on the code, but I know what is most
effective from a bang for the buck point of view.

Now, here is the situation:

 - we don’t have anyone caring much for the viewserver. It works as-is,
   could be improved, but nobody is going there (not to diminish the work
   people have put into SpiderMonkey version compat which was badly needed),
   but we don’t see much innovation going on.

 - CouchDB ships with JS-by-default as a view server. I would absolutely
   love if we could support multiple view server languages out of the box
   but we currently don’t have anyone on the project that would take a lead
   and ownership of this (which brings me back to the contribution
   encouragement). But as it stands today, we support Erlang and JS as
   view engine languages.

None of us have to *personally* be invested in Node or JS, but it is hard
for anyone looking, to see that the Node folks come at things with great
energy, tenacity, and willingness todo whatever it takes to make a good
thing happen. I am not saying other languages don’t do that, but that is
completely irrelevant as we don’t support other languages by default 
today. Bringing up that nginx is faster at HTTP than Node has also nothing
to do with this discussion.

What I want for CouchDB is to gain many more contributors and keep it a
lively project. I think having the opportunity of tapping into the Node
community is a once-in-a-lifetime chance. We are on the verge of being
the MySQL to the next PHP. We must not miss this chance. For the sake
of everything we keep dear with CouchDB.

Sorry if that sounds drastic, but that’s how I feel. Not that CouchDB
would perish without this, but it’d be a massive missed opportunity. And 
we don’t have $100M marketing department to make up for it.

More importantly, none of that means that we don’t support other languages,
tools, frameworks or whatever. It is simply opening up a part of CouchDB
to *many* *enthusiastic* people. I’d love to see embedded (as in emonk)
Python, Ruby, Smalltalk, Lisp, Haskell etc. view servers, as well as their
out-of-process counterparts, all installable with a simple command. That
is a future I want to work towards. But I think it is absolutely necessary
to pioneer this with our JS view server and moving that to Node gives us
the chance to reach exponentially more contributions than we’ve ever had.

And only if we get many more people excited about contributing to CouchDB
we get the dev-time to do all the other crazy cool things that we want to
do.

Except for solving a few relatively small technical issues, this isn’t
actually too much work.

We have nothing to loose and we can win the world.

Let’s do that.

Jan
-- 












On Jan 31, 2013, at 16:06 , Benoit Chesneau <bchesneau@gmail.com> wrote:

> Here are some notes following your *enthousiast* mail. There is not
> intention to diminish the work or things like it. It is intended to temper
> a little this enthousiast and trying to find the right approach on the
> problems we are trying to solve.
> 
> 
> On Thu, Jan 31, 2013 at 3:46 PM, Jason Smith <jhs@iriscouch.com> wrote:
> 
>> On Thu, Jan 31, 2013 at 1:51 PM, Randall Leeds <randall.leeds@gmail.com
>>> wrote:
>> 
>>> On Thu, Jan 31, 2013 at 5:23 AM, Jason Smith <jhs@iriscouch.com> wrote:
>>>> On Thu, Jan 31, 2013 at 12:55 PM, Paul Davis <
>>> paul.joseph.davis@gmail.com>wrote:
>>>> 
>>>>> 
>>>>> That whole process sounds like not a lot of fun.
>>>>> 
>>>> 
>>>> Right. That is kind of my point. CouchDB is a JavaScript thing, and
>>>> nowadays people have a very well-adopted and well-understood JavaScript
>>>> engine on their computers. Maybe it should just use that.
>>> 
>>> (Some) developers have node installed (or can install it easily). End
>>> users are a totally different story. They may be able to install it,
>>> but we're talking about adding a runtime dependency unless we bundle
>>> node.
>>> 
>> 
>> Quite right. This branch answers half that question: what do you get?
>> 
>> So far, this is my list of good things I've seen:
>> 
>> 1. Better code.
>> 1a. Cut almost 3,000 lines of code
>> 1b. Exchanged SM build dependency for Node runtime dependency. This right
>> here--this summarizes the whole exercise.
>> 
>> 2. Very encouraging degree of compatibility. Consider, the 1,500 lines of
>> view server JS code: none of it was ever intended for Node.js. But the test
>> suite shows, the two are virtually identical.
>> 
>> 3. Apparently this is already easier to use than homebrew. Homebrew pins
>> SM apparently to support Mongo (unsure if the latter is true).
>> 
>> 4. Got a lot of enthusiasm. (A lot of people tested it and emailed to ask
>> "why isn't it faster?"). This thread got a lot of feedback about new
>> protocols, and async APIs, and app-building features. Why? I think when you
>> say Node.js and CouchDB everybody says "Yes!"
>> 
> 
> 
> I say "maybe". nodejs is quite trendy. but also quite new and didn't really
> prove anything right now. It is quite surpassed by a pure C thing like
> nginx/uwsgi when it's about http, and when it's about stability by erlang
> or some. I also never had any need of nodejs when it was about doing things
> with couchdb. Differerent approach I guess. Not saying the nodejs is a bad
> one. If it solves your problems or at least is easier for you to handle
> then that's perfectly fine.  One true thing is that javascript is really
> user friendly and this thing is the one that count.
> 
> About the number of lines. Well you don't count all the lines in nodejs+v8
> as well ... but the number of loc isn't really an argument I guess.
> 
> 
>> Imagine you have CouchDB installed and then a future node version
>>> breaks compatibility for some API used by the node-couchjs. You now
>>> have to decide whether to upgrade node or try to have multiple node
>>> versions so couchjs can continue to work.
>>> 
>>> In short I think this is my issue: we're pushing problems down from
>>> maintainers and packagers to users.
>>> 
>> 
>> If you want API stability, then you'll like Node.js. The whole principle of
>> the project is to be "finished" one day.
>> 
>> Node.js is less likely than Python, say, to break a simple, 300-line repl
>> program. (My point is: not likely.) But yes, you've put your finger on it.
>> This is a runtime dependency.
>> 
> 
> This has nothing with the langage. Considering that a lot of big system are
> running under python. You can also write a view server in python in one day
> as fast as the nodejs server using libuv for example.  or other eventloops
> that didn't wait nodejs to exist.
> 
> These has nothing to do with python.
> 
>> Thinking about this all a bit more, maybe this is a viable option:
>>> 1) Figure out how to refactor the JS code in couchjs to be engine
>>> agnostic. It likely mostly is already, but we may want to make the I/O
>>> a bit more abstracted and certainly give it an async API
>>> 2) Make couchjs an optional configure flag, which builds a couchjs
>>> that wraps step (1)
>>> 3) Have node-couchjs included, as pure JS code that also wraps (1),
>>> but don't bundle any dependency like node or v8
>> 
>> 
>> The JS code is already engine-agnostic. My couchjs implementation runs all
>> that code.
>> 
>> I can also test couchjs directly with a couchjs client I wrote:
>> https://github.com/iriscouch/couchjs-client. That tool can read log files
>> and play them against any couchjs-compatible program. I log the
>> interactions messages during the test suite. And then I play them back to
>> both C and Node.js couchjs. They are identical.
>> 
>> I did some work on an async API.
>> 
>> 1. https://github.com/apache/couchdb/commit/a614678c
>> 2. https://github.com/apache/couchdb/commit/e0278596
>> 
>> I tried this. But the problem is, list functions provide basically a
>> blocking getRow() function. Once I realized that, I backed it out.
>> 
>> I think this would be a good solution unless we can demonstrate
>>> confidently that we can sandbox node appropriately for a view server.
>>> It lets your average developer get by without the spidermonkey hassle,
>>> but a maintainer of a downstream package can decide whether they want
>>> a sandboxed view server and don't mind dealing with spidermonkey
>>> packaging or to punt and add a runtime dependency on node. We could
>>> also run the JS test suite from the CLI using node-couchjs. Is there
>>> anyone who isn't served by this path, or whose problems are worsened
>>> by it?
>> 
>> 
>> The word "sandbox" is vague. There is no clear definition. (There is a
>> mundane historical reason for that: the "sandbox" was whatever the C
>> program did.)
>> 
> 
> There have been in previous mails that you ignored. Sandboxing in couchdb
> is about not sharing anything between view and not allowing external I/O
> meaning access to disk or the net. This is why the erlang view engine is
> disabled by default btw.
> 
> 
> 
>> Prediction: as quickly as we identify sandbox features, somebody can build
>> a Node.js implementation to reasonable satisfaction. But we'll see.
>> 
> 
> removing I/O to the only thing that make nodejs different from v8 doesn't
> mean anything. It should be possible but it seems wrong at the first hand.
> But we can try.
> 
>> 
>> --
>> Iris Couch
>> 


Mime
View raw message