couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Davis <>
Subject Re: Replacing the _external API
Date Fri, 24 Sep 2010 20:40:37 GMT
"I met a real, live Pick user/admin/developer..."

On Fri, Sep 24, 2010 at 3:49 PM, Stephen Prater <> wrote:
> PICK story alert!
> We have that exact implementation for communication with our PICK servers.
>  Works pretty well, and since PICK does
> really horrifying things when you feed it bad data, it provides us another
> level of input sanity checking.  (No, we will not accept your ^Y, thank
> you.)
> On Sep 24, 2010, at 2:12 PM, Paul Davis wrote:
>> Dave,
>> My knee-jerk solution would be to write a small Python script that
>> translates HTTP traffic to the current protocol. Ie, present your
>> scripts with the same interface, but not from inside the Erlang VM.
>> I'm not sure if I'm missing any details that would prevent that from
>> working though.
>> HTH,
>> Paul Davis
>> On Fri, Sep 24, 2010 at 3:03 PM, David Hardtke <> wrote:
>>> Hi Robert,
>>> Assume you've got a working program that communicates over stdio and want
>>> to
>>> use it as an external.  Or if you're using some antiquated programing
>>> language without good http libraries (octave and matlab, for instance).
>>> stdio is very easy to use.
>>> Dave
>>> On 09/24/10 11:46, Robert Newson wrote:
>>>> Who would use stdio when a concurrent and HTTP-based alternative was
>>>> available?
>>>> On Fri, Sep 24, 2010 at 7:43 PM, David Hardtke<>
>>>>  wrote:
>>>>> Hi Paul,
>>>>> Is it possible to support both?  The precedence is query servers, where
>>>>> you
>>>>> can define either a stdio based query_server or a erlang/otp based
>>>>> native_query_server.  Depending on what you are trying to do,  these
>>>>> multiple options come in handy.  In fact, it would be really nice to
>>>>> have
>>>>> 3
>>>>> options for externals -- stdio, reverse proxy, and erlang/otp.
>>>>> Dave
>>>>> On 09/24/10 11:10, Paul Davis wrote:
>>>>>> At CouchCamp there was a bit of discussion on replacing the _external
>>>>>> API with something a bit more modern to give _external processes
>>>>>> control over their environment.
>>>>>> The idea was born out of a discussion with Robert Newson who mentioned
>>>>>> that couchdb-lucene really only needs a reverse proxy to put itself
>>>>>> the same URL namespace. It occurred to us that having a reverse proxy
>>>>>> instead of the current _external stdio protocol would allow lots
>>>>>> other interesting features like node.js integration, as well as allow
>>>>>> implementors to handle requests in parallel and so on and such forth.
>>>>>> The major drawback that was identified was that if we switched to
>>>>>> a reverse proxy, people would then be responsible for handling the
>>>>>> process management of their _external handlers. Ie, they'd have to
>>>>>> configure daemon monitoring to make sure the processes stayed up
>>>>>> what not. The solution we came up with was to include another feature
>>>>>> that did process management. Ie, something that would bring up an
>>>>>> process when the server booted, and respawn it if it crashed. There'd
>>>>>> be no connection to the _externals. Other than the basic "just keep
>>>>>> process up" sort of behaviour, the only other thing I could see adding
>>>>>> is a simple stdio protocol to get configuration values from CouchDB.
>>>>>> Other people have expressed interest in just the process management
>>>>>> functionality as well which makes me think that having the two new
>>>>>> features to replace the _external API would be both easier on
>>>>>> developers as well as providing more functionality.
>>>>>> So now I'm looking for feedback on what other people might think
>>>>>> this. I'll start working on this fairly soon if I don't hear any
>>>>>> objections.
>>>>>> HTH,
>>>>>> Paul Davis

View raw message