incubator-couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Russell Branca <chewbra...@gmail.com>
Subject Re: On Alternative View Engines
Date Wed, 03 Jul 2013 20:20:13 GMT
On Tue, Jul 2, 2013 at 2:25 PM, Randall Leeds <randall.leeds@gmail.com>wrote:

> Very cool experiments there.
>
> Have you taken the time to understand JSONiq well enough to say
> whether that would be a good fit?
>
> http://www.jsoniq.org/
>
> On Sun, Jun 30, 2013 at 2:37 PM, Russell Branca <chewbranca@apache.org>
> wrote:
> > The discussion of alternative approaches to view engines is one that
> > bubbles up semi regularly, with the latest addition for a Lua native
> query
> > server described in COUCHDB-1842 by Alexander. Lua is a great language
> for
> > embedding into systems and provides powerful sandboxing facilities.
> >
> > I'm very intrigued by optimizing for the standard use case, where a user
> > wants to build a simple secondary index on their data, and then use a
> built
> > in reduce function. I think we can find a solution that allows a user to
> > define a doc level transformation in a DSL or query language or some
> other
> > approach that allows us to keep the view generation functionality within
> > the Erlang VM and avoid the overhead costs of using an external engine.
> >
> > I do think it makes sense to have an external engine for flexibility, and
> > allowing us to focus on the simple cases while providing a fallback for
> > more complex user defined functions.
> >
> > To experiment with different approaches, I built a Lisp interpreter on
> top
> > of Erlang with the premise of white listing the entire language, allowing
> > explicit control over what the user can and cannot do in view functions.
> > You can see it here: [Lispenport](
> https://github.com/chewbranca/lispenport).
> > It's by no means a full solution, but it has some interesting properties
> > such as really just being syntactic sugar on top of Erlang and all
> > constructs are direct Erlang terms, even lambdas are just Erlang funs.
> >
> > Now, while I would be intrigued by a Lisp DSL for user defined functions
> in
> > CouchDB, I didn't expect that to be well received by everyone, so I've
> > considered this just an experiment. If we were going to take this
> approach,
> > I would rather take Lisp Flavored Erlang (LFE, another project by Robert
> > Virding along with Luerl, and also erlog, a Prolog interpreter in
> Erlang),
> > and rip out all the pieces we would not want a user to access and use LFE
> > as a base starting point. LFE compiles down to intermediate Erlang
> bytecode
> > and is designed to follow Erlang functionality, making it a nice option
> for
> > building a view engine to execute in the Erlang VM.
> >
> > I've also toyed around with the idea of building a NIF around [JQ](
> > http://stedolan.github.io/jq/) which is a great application for slicing
> and
> > dicing json data structures written in C.
> >
> > So my general proposal for discussion is that we build a minimal DSL of
> > some sort, providing fast and simple doc manipulations that executes
> > securely in the Erlang VM, and then we abstract out all functionality
> for a
> > "full" view engine, list functions, show functions, etc to a separate
> > engine that is easily swappable and not required for standard
> functionality.
> >
> > Thoughts?
> >
> >
> > -Russell
>

Wow JSONiq looks cool! I hadn't seen that before, and it definitely seems
like an intriguing option. Would be a solid chunk of development to make a
new implementation of it for Erlang, but that could work. We would also
want to take a subset of the spec, and not include the pieces of JSONiq
that provide IO, but it definitely looks like it's on the right track for
what we want.


-Russell

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message