couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jan Lehnardt (JIRA)" <>
Subject [jira] [Commented] (COUCHDB-1894) Add experimental NodeJS query server
Date Fri, 27 Sep 2013 21:24:07 GMT


Jan Lehnardt commented on COUCHDB-1894:

Sorry for not addressing security concerns upfront, I knew they would be important and I rushed
this ticket instead of taking the time to lay it all out.

But first, this is a ticket for a branch for an experiment. This is not a suggestion that
we should stop and change everything we do now. If we can’t experiment, we can’t progress,
so I’d ask to take this with an experimental state of mind.

This ticket is for review and iteration on an idea and some code.

This code is an addition to the existing ways of doing things. It is off by default and opt-in
and comes 
with large disclaimers.

This view server runtime is not meant to replace couchjs today. It might never replace couchjs.

If this is to ever replace couchjs, we need to start somewhere and this is as good a start
as any. If there are other starting points, we should capture them in JIRA & branches
as well.

Getting this into more people’s hands via an experimental feature will allow us to make
this good sooner.

Dave brought up CORS as a good example of getting something experimental out that we can improve
with user feedback once it is out. I hope we can do the same here and I really hope we can
use this model a lot more in the future. This project has long suffered from trying to ship

Finally, we already ship an off-by-default and totally dangerous view server that has access
to all of CouchDB’s internals and we don’t sweat much about that. Let’s not start now.

couchjs works but it is in dire need of improving. Mikeal’s original proposal for make it
pipelined is now three or four years old. The fact that one has to juggle slightly scary JavaScript
in an unfamiliar environment (SpiderMonkey on the cli is no really common), some scary C and
some Erlang to iterate on the query server implementation lead to de-facto zero iteration
and improvement. Sure we tacked on a few things and refactored the Erlang side of things,
but the fundamental operation of couchjs & main.js has not changed in half a decade.

I believe we can open this up to more experimenting and eventually to better software if we
have a Node.js version of this.

I have outlined the reasons why we want the default query server to be JavaScript in a recent
mailing list post[1] so I won’t reiterate this here.

What I would love to see here is the following:

 - definition of an acceptable secure code execution environment for view functions.
 - an improved communication method and protocol between the view server and CouchDB.
 - the ripping out of anything that isn’t necessary for views 
 - the moving of features like _show/_list/_update etc. to a separate execution environment
that is better suited for these kinds of access models. (the way we run _show & _list
is really not ideal).
 - and finally and most importantly: other language implementations of the improved protocol
and better separated features that we can then promote more prominently.

This is a lot of work and we need to ship working software along the way. That’s why I propose
to get this experiment going early and start from a point of minimal differences to the current
model so we all have a chance of going on the journey of iterating and improving the view
server and ultimately a core feature of CouchDB.

Now, as for the security scenario in some more detail. I agree with Jason that we need to
be very clear about what we mean with various terms and what we compare couchjs-nodejs too,
especially because wrecking havoc with couchjs is not too hard today.

That said, despite the state of the vm module in Node (and its ongoing rewrite which we should
watch closely), it should give us exactly what we need: a pure execution environment that
has whitelisted access to outside resources. There is some more legwork required today which
is where sandbox.js comes in. We haven’t solved all the problems yet (e.g. sandbox isn’t
even used today), but cursory trying to break out of the current implementation wasn’t trivial.
I plan to get the node security project involved so we get a bit of a better understanding
and maybe even a proper security review.

I hope this addresses all concerns for making this a priority in CouchDB land. I’m looking
forward to hack with you.

> Add experimental NodeJS query server
> ------------------------------------
>                 Key: COUCHDB-1894
>                 URL:
>             Project: CouchDB
>          Issue Type: New Feature
>          Components: JavaScript View Server
>            Reporter: Jan Lehnardt
> Let’s clean up and merge Jason Smith’s Node.js query server into ASF land and ship
it as opt-in and experimental.
> I’ve prepared a branch that does the following:
>  - remove fancy extra features like app server handlers and the visual debugger support
for now
>  - make it a drop-in replacement for couchjs
>  - bundle the code in src/couchjs-node
>  - add a new query server language “nodejs” that people can use
>  - include sandbox.js from (not hooked up
> The query server is not installed by default and users can install them in two ways:
> 1. from source:
>     $ cd src/couchjs-node
>     $ npm link
> 2. from NPM:
>     $ npm install couchjs # add @1.x.x for once the module mirrors CouchDB version numbers
for forward compat)
> And then they can uncomment and update the [query_server] line in local.ini.
> * * *
> Open work items on the view server:
>  - make it work with CLI tests
>  - fix remaining test cases in web test runner
>  - hook up sandbox.js from

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message