Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@couchdb.apache.org
Received-SPF: pass (athena.apache.org: domain of paul.joseph.davis@gmail.com
 designates 209.85.219.52 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <A1838CF9-CCBF-4709-8013-C8E221A839B7@apache.org>
References: <96FBCC7A-F696-4BD8-9B7B-4126F3B5362A@apache.org>
 <CACLy94VaTGpWqrfyBDen5py3tSi1bTVjVyR2dStRmp7q8fy=9g@mail.gmail.com>
 <001BB5DC-23F7-4BEB-9553-09DCF4F9F5CB@apache.org>
 <CACLy94Xtctb+zNJWwzqTnb1sKWJF51f3xKSNYusj+DD-c_9J3A@mail.gmail.com>
 <617A17F9-7349-4D07-A689-95CD6CE2D6D3@apache.org>
 <CACLy94VPu0trm0eN6Qq1oJxESS9jO-SufJNMD=M5aKo6MfvR8g@mail.gmail.com>
 <A1838CF9-CCBF-4709-8013-C8E221A839B7@apache.org>
From: Paul Davis <paul.joseph.davis@gmail.com>
Date: Thu, 8 Aug 2013 07:45:09 -0500
Message-ID: 
 <CAJ_m3YDVj6KMtSJp2AXRO6vCZB90upd6O_wt1Q7MUJiG3Ni_mw@mail.gmail.com>
Subject: Re: [PLUGINS] Plugin Hooks
To: dev@couchdb.apache.org
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

A behavior isn't quite right because as Jan points out, you may not
want to implement every hook. Thinking briefly on the idea I think the
general idea is good but I'd implement it via message passing rather
than callbacks. While in the happy case callbacks are usually faster
than message passing its fairly easy to get into a situation where a
particular callback ends up being slow which will slow down the
calling process (it sounds obvious and it kind of is, but its easy to
miss places where a function that's usually quick suddenly starts
taking milliseconds to run which causes the mailbox to explode if it
can't receive messages fast enough).

A good example of this is how couch_db_update_notifier works with
gen_event. When a node gets overloaded due to changes feed listeners
or other random slowdown in this function it can lead to the entire
system halting because its waiting for events to be processed through
this function. After having dealt with that its become fairly obvious
to me that gen_event is only useful when the number of callbacks is a
constant (or rather, not dependent on user input).

On the one hand plugins using hooks should be a relatively small
number of well known functions that we can patch if they end up
causing problems. On the other hand if we ever plan on supporting
closed source plugins then I'd vote for the safer message passing
approach as a preventative measure against misbehaving code.

On Thu, Aug 8, 2013 at 7:25 AM, Jan Lehnardt <jan@apache.org> wrote:
>
> On Aug 8, 2013, at 14:05 , Jason Smith <jhs@apache.org> wrote:
>
>> Note, I intentially began this discussion as a question, not a statement=
.
>> Totally thinking "aloud."
>
> Same here, thanks for exploring this. I do like `on` :)
>
> Jan
> --
>
>>
>> I think it is like gen_server. For gen_server all you need is a
>> handle_call/3 and handle_cast/2. The real API is inside that.
>>
>> For example, maybe all that's required for the behaviour is on/1 and we =
can
>> make a flexible hook system from that. (Now why would "on" be a meaningf=
ul
>> or useful function name?)
>>
>> on({log, Severity, Message}) -> ok
>>    , io:format("Couch says ~s\n", [Message])
>>    ;
>>
>> % Suppose I want a database that stays synced to my config
>> % (something I personally need at the moment).
>>
>> on({startup}) -> ok
>>    , io:format("CouchDB started!\n")
>>    , init_my_config_db(couch_config:get("*/*")) % Pseudocode
>>    ;
>>
>> on({config, Section, Key, Value}) -> ok
>>    , io:format("Config update: ~s/~s =3D ~p\n", [Section, Key, Value])
>>    , update_my_config_db(Section, Key, Value)
>>    ;
>>
>> on(Unknown) -> ok
>>    , io:format("Did you know there is a ~p hook?\n", [element(1, Unknown=
)])
>>    .
>>
>>
>>
>> On Thu, Aug 8, 2013 at 5:42 PM, Jan Lehnardt <jan@apache.org> wrote:
>>
>>>
>>> On Aug 8, 2013, at 12:39 , Jason Smith <jhs@apache.org> wrote:
>>>
>>>> Well, I just googled it. Basically there is a couchdb_plugin.erl which
>>>> tells Erlang what a behavior looks like. And all that does is define t=
he
>>>> functions and arity which a couchdb_plugin would have to export.
>>>>
>>>> Probably there are some better Erlangers on the list who might chime i=
n.
>>> It
>>>> looks like okay bang-for-buck; only not much bang or much buck.
>>>
>>> how would this work if a plugin is only interested in handling a single
>>> hook,
>>> would it have to implement mock funs for all hooks then?
>>>
>>>> On Thu, Aug 8, 2013 at 5:26 PM, Jan Lehnardt <jan@apache.org> wrote:
>>>>
>>>>> how would this look in code?
>>>>>
>>>>> On Aug 8, 2013, at 12:21 , Jason Smith <jhs@apache.org> wrote:
>>>>>
>>>>>> Perhaps a custom behaviour to help catch API problems at compile tim=
e?
>>>>>>
>>>>>>  -behaviour(couchdb_plugin).
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Aug 8, 2013 at 3:47 PM, Jan Lehnardt <jan@apache.org> wrote:
>>>>>>
>>>>>>> Heya,
>>>>>>>
>>>>>>> I=92m toying with the idea of moving some of my experimental into
>>>>>>> bona-fide plugins. One of them is my log_to_db branch that on top o=
f
>>>>>>> writing log messages to a text file also writes a document to a log
>>>>>>> database.
>>>>>>>
>>>>>>> Conceptually, this is the perfect use of a plugin: the feature is n=
ot
>>>>>>> useful in the general case, because *any* activity creates write lo=
ad
>>>>>>> on a single database, but for certain low-volume installations, thi=
s
>>>>>>> might be a useful feature (I wouldn=92t have written it, if I hadn=
=92t
>>>>>>> needed it at some point) so allowing users to enable it as a plugin
>>>>>>> would be nice.
>>>>>>>
>>>>>>> But regardless of whether my plugin is useful, it illustrates an
>>>>>>> interesting point:
>>>>>>>
>>>>>>> A log_to_db plugin would need to register for logging events or, if=
 it
>>>>>>> doesn=92t want to duplicate all the logging-level logic in couch_lo=
g, it
>>>>>>> would need some way of injecting a function call into
>>>>>>> `couch_log:log().`. We could of course try and find a way where a
>>>>>>> plugin would be able to provide an API compatible version of a Couc=
hDB
>>>>>>> module and swap it out for it=92s custom one, but that=92s hardly a=
 great
>>>>>>> idea.
>>>>>>>
>>>>>>> Other software has the notion of =93hooks=94 (some may call it some=
thing
>>>>>>> else) where at well defined points in the main code base, external
>>>>>>> functions get called with certain parameters. To make things dynami=
c,
>>>>>>> there might be a way for plugins to register to be called by those
>>>>>>> hooks and the main code then asks the registry whether there are an=
y
>>>>>>> plugin functions to call.
>>>>>>>
>>>>>>> In the log_to_db example, we=92d have something like this:
>>>>>>>
>>>>>>> couch_log_to_db.erl:
>>>>>>>
>>>>>>>  init() ->
>>>>>>>      couch_hooks:register(couch_log_hook, log_hook_fun/1),
>>>>>>>      ok.
>>>>>>>
>>>>>>>  log_hook_fun(Log) ->
>>>>>>>      % do the log_to_db magic
>>>>>>>      ok.
>>>>>>>
>>>>>>>
>>>>>>> couch_hooks.erl:
>>>>>>>
>>>>>>>  register(Hook, Fun) ->
>>>>>>>      % store the Fun with the Hook somewhere
>>>>>>>      ok.
>>>>>>>
>>>>>>>  call(Hook, Args) ->
>>>>>>>       % retrieve Fun for Hook from somewhere
>>>>>>>      Fun(Args).
>>>>>>>
>>>>>>> couch_log.erl:
>>>>>>>
>>>>>>> % in log()
>>>>>>>
>>>>>>>  ...
>>>>>>>  couch_hooks:call(couch_log_hook, Args),
>>>>>>>  ...
>>>>>>>
>>>>>>> The main code would define what the hook name and arguments are and
>>> the
>>>>>>> plugin would have to conform. The plugin registry would just manage
>>> the
>>>>>>> registration and calling of functions for a hook, but nothing more.
>>>>>>>
>>>>>>> * * *
>>>>>>>
>>>>>>> This is just my first stab at this not thinking about it too much a=
nd
>>> I
>>>>>>> likely miss some subtleties in Erlang that make this not work (hot
>>> code
>>>>>>> upgrades e.g.).
>>>>>>>
>>>>>>>
>>>>>>> How do you think we should implement a hooks feature in CouchDB?
>>>>>>>
>>>>>>>
>>>>>>> Thanks!
>>>>>>> Jan
>>>>>>> --
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>
>>>
>