couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Davis <paul.joseph.da...@gmail.com>
Subject Re: [PLUGINS] Plugin Hooks
Date Thu, 08 Aug 2013 12:45:09 GMT
A behavior isn't quite right because as Jan points out, you may not
want to implement every hook. Thinking briefly on the idea I think the
general idea is good but I'd implement it via message passing rather
than callbacks. While in the happy case callbacks are usually faster
than message passing its fairly easy to get into a situation where a
particular callback ends up being slow which will slow down the
calling process (it sounds obvious and it kind of is, but its easy to
miss places where a function that's usually quick suddenly starts
taking milliseconds to run which causes the mailbox to explode if it
can't receive messages fast enough).

A good example of this is how couch_db_update_notifier works with
gen_event. When a node gets overloaded due to changes feed listeners
or other random slowdown in this function it can lead to the entire
system halting because its waiting for events to be processed through
this function. After having dealt with that its become fairly obvious
to me that gen_event is only useful when the number of callbacks is a
constant (or rather, not dependent on user input).

On the one hand plugins using hooks should be a relatively small
number of well known functions that we can patch if they end up
causing problems. On the other hand if we ever plan on supporting
closed source plugins then I'd vote for the safer message passing
approach as a preventative measure against misbehaving code.

On Thu, Aug 8, 2013 at 7:25 AM, Jan Lehnardt <jan@apache.org> wrote:
>
> On Aug 8, 2013, at 14:05 , Jason Smith <jhs@apache.org> wrote:
>
>> Note, I intentially began this discussion as a question, not a statement.
>> Totally thinking "aloud."
>
> Same here, thanks for exploring this. I do like `on` :)
>
> Jan
> --
>
>>
>> I think it is like gen_server. For gen_server all you need is a
>> handle_call/3 and handle_cast/2. The real API is inside that.
>>
>> For example, maybe all that's required for the behaviour is on/1 and we can
>> make a flexible hook system from that. (Now why would "on" be a meaningful
>> or useful function name?)
>>
>> on({log, Severity, Message}) -> ok
>>    , io:format("Couch says ~s\n", [Message])
>>    ;
>>
>> % Suppose I want a database that stays synced to my config
>> % (something I personally need at the moment).
>>
>> on({startup}) -> ok
>>    , io:format("CouchDB started!\n")
>>    , init_my_config_db(couch_config:get("*/*")) % Pseudocode
>>    ;
>>
>> on({config, Section, Key, Value}) -> ok
>>    , io:format("Config update: ~s/~s = ~p\n", [Section, Key, Value])
>>    , update_my_config_db(Section, Key, Value)
>>    ;
>>
>> on(Unknown) -> ok
>>    , io:format("Did you know there is a ~p hook?\n", [element(1, Unknown)])
>>    .
>>
>>
>>
>> On Thu, Aug 8, 2013 at 5:42 PM, Jan Lehnardt <jan@apache.org> wrote:
>>
>>>
>>> On Aug 8, 2013, at 12:39 , Jason Smith <jhs@apache.org> wrote:
>>>
>>>> Well, I just googled it. Basically there is a couchdb_plugin.erl which
>>>> tells Erlang what a behavior looks like. And all that does is define the
>>>> functions and arity which a couchdb_plugin would have to export.
>>>>
>>>> Probably there are some better Erlangers on the list who might chime in.
>>> It
>>>> looks like okay bang-for-buck; only not much bang or much buck.
>>>
>>> how would this work if a plugin is only interested in handling a single
>>> hook,
>>> would it have to implement mock funs for all hooks then?
>>>
>>>> On Thu, Aug 8, 2013 at 5:26 PM, Jan Lehnardt <jan@apache.org> wrote:
>>>>
>>>>> how would this look in code?
>>>>>
>>>>> On Aug 8, 2013, at 12:21 , Jason Smith <jhs@apache.org> wrote:
>>>>>
>>>>>> Perhaps a custom behaviour to help catch API problems at compile
time?
>>>>>>
>>>>>>  -behaviour(couchdb_plugin).
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Aug 8, 2013 at 3:47 PM, Jan Lehnardt <jan@apache.org>
wrote:
>>>>>>
>>>>>>> Heya,
>>>>>>>
>>>>>>> I’m toying with the idea of moving some of my experimental
into
>>>>>>> bona-fide plugins. One of them is my log_to_db branch that on
top of
>>>>>>> writing log messages to a text file also writes a document to
a log
>>>>>>> database.
>>>>>>>
>>>>>>> Conceptually, this is the perfect use of a plugin: the feature
is not
>>>>>>> useful in the general case, because *any* activity creates write
load
>>>>>>> on a single database, but for certain low-volume installations,
this
>>>>>>> might be a useful feature (I wouldn’t have written it, if I
hadn’t
>>>>>>> needed it at some point) so allowing users to enable it as a
plugin
>>>>>>> would be nice.
>>>>>>>
>>>>>>> But regardless of whether my plugin is useful, it illustrates
an
>>>>>>> interesting point:
>>>>>>>
>>>>>>> A log_to_db plugin would need to register for logging events
or, if it
>>>>>>> doesn’t want to duplicate all the logging-level logic in couch_log,
it
>>>>>>> would need some way of injecting a function call into
>>>>>>> `couch_log:log().`. We could of course try and find a way where
a
>>>>>>> plugin would be able to provide an API compatible version of
a CouchDB
>>>>>>> module and swap it out for it’s custom one, but that’s hardly
a great
>>>>>>> idea.
>>>>>>>
>>>>>>> Other software has the notion of “hooks” (some may call it
something
>>>>>>> else) where at well defined points in the main code base, external
>>>>>>> functions get called with certain parameters. To make things
dynamic,
>>>>>>> there might be a way for plugins to register to be called by
those
>>>>>>> hooks and the main code then asks the registry whether there
are any
>>>>>>> plugin functions to call.
>>>>>>>
>>>>>>> In the log_to_db example, we’d have something like this:
>>>>>>>
>>>>>>> couch_log_to_db.erl:
>>>>>>>
>>>>>>>  init() ->
>>>>>>>      couch_hooks:register(couch_log_hook, log_hook_fun/1),
>>>>>>>      ok.
>>>>>>>
>>>>>>>  log_hook_fun(Log) ->
>>>>>>>      % do the log_to_db magic
>>>>>>>      ok.
>>>>>>>
>>>>>>>
>>>>>>> couch_hooks.erl:
>>>>>>>
>>>>>>>  register(Hook, Fun) ->
>>>>>>>      % store the Fun with the Hook somewhere
>>>>>>>      ok.
>>>>>>>
>>>>>>>  call(Hook, Args) ->
>>>>>>>       % retrieve Fun for Hook from somewhere
>>>>>>>      Fun(Args).
>>>>>>>
>>>>>>> couch_log.erl:
>>>>>>>
>>>>>>> % in log()
>>>>>>>
>>>>>>>  ...
>>>>>>>  couch_hooks:call(couch_log_hook, Args),
>>>>>>>  ...
>>>>>>>
>>>>>>> The main code would define what the hook name and arguments are
and
>>> the
>>>>>>> plugin would have to conform. The plugin registry would just
manage
>>> the
>>>>>>> registration and calling of functions for a hook, but nothing
more.
>>>>>>>
>>>>>>> * * *
>>>>>>>
>>>>>>> This is just my first stab at this not thinking about it too
much and
>>> I
>>>>>>> likely miss some subtleties in Erlang that make this not work
(hot
>>> code
>>>>>>> upgrades e.g.).
>>>>>>>
>>>>>>>
>>>>>>> How do you think we should implement a hooks feature in CouchDB?
>>>>>>>
>>>>>>>
>>>>>>> Thanks!
>>>>>>> Jan
>>>>>>> --
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>
>>>
>

Mime
View raw message