Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@couchdb.apache.org
Date: Fri, 27 Sep 2013 21:24:07 +0000 (UTC)
From: "Jan Lehnardt (JIRA)" <jira@apache.org>
To: dev@couchdb.apache.org
Message-ID: <JIRA.12670467.1380108613301.32827.1380317047193@arcas>
In-Reply-To: <JIRA.12670467.1380108613301@arcas>
References: <JIRA.12670467.1380108613301@arcas>
Subject: [jira] [Commented] (COUCHDB-1894) Add experimental NodeJS query
 server
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


    [ https://issues.apache.org/jira/browse/COUCHDB-1894?page=3Dcom.atlassi=
an.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D13=
780430#comment-13780430 ]=20

Jan Lehnardt commented on COUCHDB-1894:
---------------------------------------

Sorry for not addressing security concerns upfront, I knew they would be im=
portant and I rushed this ticket instead of taking the time to lay it all o=
ut.

But first, this is a ticket for a branch for an experiment. This is not a s=
uggestion that we should stop and change everything we do now. If we can=E2=
=80=99t experiment, we can=E2=80=99t progress, so I=E2=80=99d ask to take t=
his with an experimental state of mind.


This ticket is for review and iteration on an idea and some code.

This code is an addition to the existing ways of doing things. It is off by=
 default and opt-in and comes=20
with large disclaimers.

This view server runtime is not meant to replace couchjs today. It might ne=
ver replace couchjs.

If this is to ever replace couchjs, we need to start somewhere and this is =
as good a start as any. If there are other starting points, we should captu=
re them in JIRA & branches as well.

Getting this into more people=E2=80=99s hands via an experimental feature w=
ill allow us to make this good sooner.

Dave brought up CORS as a good example of getting something experimental ou=
t that we can improve with user feedback once it is out. I hope we can do t=
he same here and I really hope we can use this model a lot more in the futu=
re. This project has long suffered from trying to ship perfection.

Finally, we already ship an off-by-default and totally dangerous view serve=
r that has access to all of CouchDB=E2=80=99s internals and we don=E2=80=99=
t sweat much about that. Let=E2=80=99s not start now.


couchjs works but it is in dire need of improving. Mikeal=E2=80=99s origina=
l proposal for make it pipelined is now three or four years old. The fact t=
hat one has to juggle slightly scary JavaScript in an unfamiliar environmen=
t (SpiderMonkey on the cli is no really common), some scary C and some Erla=
ng to iterate on the query server implementation lead to de-facto zero iter=
ation and improvement. Sure we tacked on a few things and refactored the Er=
lang side of things, but the fundamental operation of couchjs & main.js has=
 not changed in half a decade.

I believe we can open this up to more experimenting and eventually to bette=
r software if we have a Node.js version of this.

I have outlined the reasons why we want the default query server to be Java=
Script in a recent mailing list post[1] so I won=E2=80=99t reiterate this h=
ere.

What I would love to see here is the following:

 - definition of an acceptable secure code execution environment for view f=
unctions.
 - an improved communication method and protocol between the view server an=
d CouchDB.
 - the ripping out of anything that isn=E2=80=99t necessary for views=20
 - the moving of features like _show/_list/_update etc. to a separate execu=
tion environment that is better suited for these kinds of access models. (t=
he way we run _show & _list is really not ideal).
 - and finally and most importantly: other language implementations of the =
improved protocol and better separated features that we can then promote mo=
re prominently.

This is a lot of work and we need to ship working software along the way. T=
hat=E2=80=99s why I propose to get this experiment going early and start fr=
om a point of minimal differences to the current model so we all have a cha=
nce of going on the journey of iterating and improving the view server and =
ultimately a core feature of CouchDB.

Now, as for the security scenario in some more detail. I agree with Jason t=
hat we need to be very clear about what we mean with various terms and what=
 we compare couchjs-nodejs too, especially because wrecking havoc with couc=
hjs is not too hard today.

That said, despite the state of the vm module in Node (and its ongoing rewr=
ite which we should watch closely), it should give us exactly what we need:=
 a pure execution environment that has whitelisted access to outside resour=
ces. There is some more legwork required today which is where sandbox.js co=
mes in. We haven=E2=80=99t solved all the problems yet (e.g. sandbox isn=E2=
=80=99t even used today), but cursory trying to break out of the current im=
plementation wasn=E2=80=99t trivial. I plan to get the node security projec=
t involved so we get a bit of a better understanding and maybe even a prope=
r security review.

I hope this addresses all concerns for making this a priority in CouchDB la=
nd. I=E2=80=99m looking forward to hack with you.

[1]: http://mail-archives.apache.org/mod_mbox/couchdb-dev/201309.mbox/%3c22=
2E40F7-99E5-4C2C-8EE1-756F0535263C@apache.org%3e
               =20
> Add experimental NodeJS query server
> ------------------------------------
>
>                 Key: COUCHDB-1894
>                 URL: https://issues.apache.org/jira/browse/COUCHDB-1894
>             Project: CouchDB
>          Issue Type: New Feature
>          Components: JavaScript View Server
>            Reporter: Jan Lehnardt
>
> Let=E2=80=99s clean up and merge Jason Smith=E2=80=99s Node.js query serv=
er into ASF land and ship it as opt-in and experimental.
> I=E2=80=99ve prepared a branch that does the following:
>  - remove fancy extra features like app server handlers and the visual de=
bugger support for now
>  - make it a drop-in replacement for couchjs
>  - bundle the code in src/couchjs-node
>  - add a new query server language =E2=80=9Cnodejs=E2=80=9D that people c=
an use
>  - include sandbox.js from https://github.com/KlausTrainer/sandbox.js (no=
t hooked up yet)
> The query server is not installed by default and users can install them i=
n two ways:
> 1. from source:
>     $ cd src/couchjs-node
>     $ npm link
> 2. from NPM:
>     $ npm install couchjs # add @1.x.x for once the module mirrors CouchD=
B version numbers for forward compat)
> And then they can uncomment and update the [query_server] line in local.i=
ni.
> * * *
> Open work items on the view server:
>  - make it work with CLI tests
>  - fix remaining test cases in web test runner
>  - hook up sandbox.js from https://github.com/KlausTrainer/sandbox.js

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrato=
rs
For more information on JIRA, see: http://www.atlassian.com/software/jira