couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McDaniel <>
Subject Re: Erlang API
Date Wed, 11 Feb 2009 22:36:41 GMT
On Tue, Feb 10, 2009 at 12:29:01PM -0800, Chris Anderson wrote:
> Just bumping this over to a new thread for now, so we don't derail
> "Roadmap discussion"
> On Tue, Feb 10, 2009 at 11:34 AM, Kerr Rainey <> wrote:
> > 2009/2/10 Michael McDaniel <>:
> >>
> >>  ... also, an Erlang API that skips the
> >>
> >>                   JSON <-convert-> native Erlang terms
> >>
> >>  translation overhead.  Being as term translation is not necessary
> >>  when talking 'directly' with the CDB engine
> >>  (e.g. couch_query_servers:map_docs/2 could skip the JSON <-> term()
> >>  translation if the view engine reads/writes native Erlang terms)
> >
> > Interesting.  I'd certainly consider this another level further than
> > what I was thinking of, or indeed would be thinking of using.  There
> > is probably a few levels where couch functionality could be exposed
> > natively.
> >
> > I wonder how much doing this kind of bypassing for a native erlang
> > view engine would complicate the code?  Or would it give another clean
> > layer?
> >
> I skip the JSON step in my native interfacing. I may be able to make
> some of that code available, once it is cleaned up.
> I've noticed that I tend to be jumping in and out of the current
> CouchDB stack, depending on what I need to do. Putting everything I'm
> using into a baseline interface layer might clean CouchDB up, or it
> might needlessly complicate it. I guess we'd have to see.
> -- 
> Chris Anderson

 I am working on an Erlang view server (EVS).  It currently works
 in a rudimentary fashion.  I have been testing from Futon.  I do
 not think it is currently as flexible as the default javascript map
 funs, though do not know all the map fun capabilities available with
 the default. 

 The only CDB changes I have needed are to add the following two
 functions to couch_query_servers.erl  (valid as of CDB v740870)
 before their respective existing funs.  

 The start_doc_map/2 fun is different only in that it provides EVS
 Pid rather than using get_os_process(Lang).  Because the EVS is
 started as a daemon and not like main.js is started, 
 get_os_process(Lang) does not work as written. 

 The map_docs/2 fun is different in that no JSON <-> Erlang term()
 conversion is done (not needed when talking Erlang <-> Erlang).

start_doc_map(<<"erlang">>, Functions) ->
  Pid = erlang:whereis(erlview) ,   % need to parameterize erlview
                                    % Pid is fed to map_docs
    lists:foreach(fun(FunctionSource) ->
                      true = couch_os_process:prompt(Pid, 
                  end, Functions),
    {ok, {<<"erlang">>, Pid}}

% Pid comes from start_doc_map/2 (see call in couch_view_updater)
map_docs({<<"erlang">>, Pid}, Docs) ->
    Results = lists:map( fun( Doc ) ->
                           couch_os_process:prompt(Pid, [<<"map_doc">>, Doc])
                         Docs) ,
    {ok, Results}

NOTE that the additional start_doc_map/2 fun can be eliminated if you
want to change get_os_process/1 to ...

get_os_process(Lang) ->
   case Lang 
     of <<"erlang">> -> erlang:whereis(erlview) ;
     _               -> gen_server:call(couch_query_servers,
                                       {get_proc, Lang})

 Also note that, obviously, the EVS name 'erlview' is hard-coded and
 should be a local.ini parameter.

 Above is a first-cut; possibly a message passing protocol to the
 EVS would be faster though that would require more CDB hacking.

 I'll post some code when it does a bit more than gurgle bubbles.

 Please, if there is some activity to change CDB internals to 
 simplify a native Erlang view server or create a more direct
 interface, let me know so I don't go too far down this road.

 No sense replicating effort (only databases!).


Michael McDaniel
Portland, Oregon, USA

View raw message