couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephen Prater <>
Subject Re: Replacing the _external API
Date Fri, 24 Sep 2010 19:49:25 GMT
PICK story alert!

We have that exact implementation for communication with our PICK  
servers.  Works pretty well, and since PICK does
really horrifying things when you feed it bad data, it provides us  
another level of input sanity checking.  (No, we will not accept your  
^Y, thank you.)

On Sep 24, 2010, at 2:12 PM, Paul Davis wrote:

> Dave,
> My knee-jerk solution would be to write a small Python script that
> translates HTTP traffic to the current protocol. Ie, present your
> scripts with the same interface, but not from inside the Erlang VM.
> I'm not sure if I'm missing any details that would prevent that from
> working though.
> HTH,
> Paul Davis
> On Fri, Sep 24, 2010 at 3:03 PM, David Hardtke <>  
> wrote:
>> Hi Robert,
>> Assume you've got a working program that communicates over stdio  
>> and want to
>> use it as an external.  Or if you're using some antiquated programing
>> language without good http libraries (octave and matlab, for  
>> instance).
>> stdio is very easy to use.
>> Dave
>> On 09/24/10 11:46, Robert Newson wrote:
>>> Who would use stdio when a concurrent and HTTP-based alternative was
>>> available?
>>> On Fri, Sep 24, 2010 at 7:43 PM, David  
>>> Hardtke<>  wrote:
>>>> Hi Paul,
>>>> Is it possible to support both?  The precedence is query servers,  
>>>> where
>>>> you
>>>> can define either a stdio based query_server or a erlang/otp based
>>>> native_query_server.  Depending on what you are trying to do,   
>>>> these
>>>> multiple options come in handy.  In fact, it would be really nice  
>>>> to have
>>>> 3
>>>> options for externals -- stdio, reverse proxy, and erlang/otp.
>>>> Dave
>>>> On 09/24/10 11:10, Paul Davis wrote:
>>>>> At CouchCamp there was a bit of discussion on replacing the  
>>>>> _external
>>>>> API with something a bit more modern to give _external processes  
>>>>> more
>>>>> control over their environment.
>>>>> The idea was born out of a discussion with Robert Newson who  
>>>>> mentioned
>>>>> that couchdb-lucene really only needs a reverse proxy to put  
>>>>> itself in
>>>>> the same URL namespace. It occurred to us that having a reverse  
>>>>> proxy
>>>>> instead of the current _external stdio protocol would allow lots  
>>>>> of
>>>>> other interesting features like node.js integration, as well as  
>>>>> allow
>>>>> implementors to handle requests in parallel and so on and such  
>>>>> forth.
>>>>> The major drawback that was identified was that if we switched  
>>>>> to just
>>>>> a reverse proxy, people would then be responsible for handling the
>>>>> process management of their _external handlers. Ie, they'd have to
>>>>> configure daemon monitoring to make sure the processes stayed up  
>>>>> and
>>>>> what not. The solution we came up with was to include another  
>>>>> feature
>>>>> that did process management. Ie, something that would bring up  
>>>>> an OS
>>>>> process when the server booted, and respawn it if it crashed.  
>>>>> There'd
>>>>> be no connection to the _externals. Other than the basic "just  
>>>>> keep a
>>>>> process up" sort of behaviour, the only other thing I could see  
>>>>> adding
>>>>> is a simple stdio protocol to get configuration values from  
>>>>> CouchDB.
>>>>> Other people have expressed interest in just the process  
>>>>> management
>>>>> functionality as well which makes me think that having the two new
>>>>> features to replace the _external API would be both easier on
>>>>> developers as well as providing more functionality.
>>>>> So now I'm looking for feedback on what other people might think  
>>>>> of
>>>>> this. I'll start working on this fairly soon if I don't hear any  
>>>>> major
>>>>> objections.
>>>>> HTH,
>>>>> Paul Davis

View raw message