incubator-couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Randall Leeds <randall.le...@gmail.com>
Subject Re: On Alternative View Engines
Date Tue, 02 Jul 2013 21:25:12 GMT
Very cool experiments there.

Have you taken the time to understand JSONiq well enough to say
whether that would be a good fit?

http://www.jsoniq.org/

On Sun, Jun 30, 2013 at 2:37 PM, Russell Branca <chewbranca@apache.org> wrote:
> The discussion of alternative approaches to view engines is one that
> bubbles up semi regularly, with the latest addition for a Lua native query
> server described in COUCHDB-1842 by Alexander. Lua is a great language for
> embedding into systems and provides powerful sandboxing facilities.
>
> I'm very intrigued by optimizing for the standard use case, where a user
> wants to build a simple secondary index on their data, and then use a built
> in reduce function. I think we can find a solution that allows a user to
> define a doc level transformation in a DSL or query language or some other
> approach that allows us to keep the view generation functionality within
> the Erlang VM and avoid the overhead costs of using an external engine.
>
> I do think it makes sense to have an external engine for flexibility, and
> allowing us to focus on the simple cases while providing a fallback for
> more complex user defined functions.
>
> To experiment with different approaches, I built a Lisp interpreter on top
> of Erlang with the premise of white listing the entire language, allowing
> explicit control over what the user can and cannot do in view functions.
> You can see it here: [Lispenport](https://github.com/chewbranca/lispenport).
> It's by no means a full solution, but it has some interesting properties
> such as really just being syntactic sugar on top of Erlang and all
> constructs are direct Erlang terms, even lambdas are just Erlang funs.
>
> Now, while I would be intrigued by a Lisp DSL for user defined functions in
> CouchDB, I didn't expect that to be well received by everyone, so I've
> considered this just an experiment. If we were going to take this approach,
> I would rather take Lisp Flavored Erlang (LFE, another project by Robert
> Virding along with Luerl, and also erlog, a Prolog interpreter in Erlang),
> and rip out all the pieces we would not want a user to access and use LFE
> as a base starting point. LFE compiles down to intermediate Erlang bytecode
> and is designed to follow Erlang functionality, making it a nice option for
> building a view engine to execute in the Erlang VM.
>
> I've also toyed around with the idea of building a NIF around [JQ](
> http://stedolan.github.io/jq/) which is a great application for slicing and
> dicing json data structures written in C.
>
> So my general proposal for discussion is that we build a minimal DSL of
> some sort, providing fast and simple doc manipulations that executes
> securely in the Erlang VM, and then we abstract out all functionality for a
> "full" view engine, list functions, show functions, etc to a separate
> engine that is easily swappable and not required for standard functionality.
>
> Thoughts?
>
>
> -Russell

Mime
View raw message