incubator-couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jan Lehnardt <...@apache.org>
Subject Re: [PLUGINS] Plugin Hooks
Date Thu, 08 Aug 2013 12:52:27 GMT

On Aug 8, 2013, at 14:45 , Paul Davis <paul.joseph.davis@gmail.com> wrote:

> A behavior isn't quite right because as Jan points out, you may not
> want to implement every hook. Thinking briefly on the idea I think the
> general idea is good but I'd implement it via message passing rather
> than callbacks. While in the happy case callbacks are usually faster
> than message passing its fairly easy to get into a situation where a
> particular callback ends up being slow which will slow down the
> calling process (it sounds obvious and it kind of is, but its easy to
> miss places where a function that's usually quick suddenly starts
> taking milliseconds to run which causes the mailbox to explode if it
> can't receive messages fast enough).
> 
> A good example of this is how couch_db_update_notifier works with
> gen_event. When a node gets overloaded due to changes feed listeners
> or other random slowdown in this function it can lead to the entire
> system halting because its waiting for events to be processed through
> this function. After having dealt with that its become fairly obvious
> to me that gen_event is only useful when the number of callbacks is a
> constant (or rather, not dependent on user input).
> 
> On the one hand plugins using hooks should be a relatively small
> number of well known functions that we can patch if they end up
> causing problems. On the other hand if we ever plan on supporting
> closed source plugins then I'd vote for the safer message passing
> approach as a preventative measure against misbehaving code.

Excellent, thanks Paul!

How would this work if we’d allow a plugin to, say, format the log
string before it goes into the regular log?

Happy to conclude that we don’t support that (maybe in v1), but we’d
be limiting what plugins can do.

As a general warning we should let our users know that plugins can
screw things up unpredictably (at least in a plugin’s early days).


> 
> On Thu, Aug 8, 2013 at 7:25 AM, Jan Lehnardt <jan@apache.org> wrote:
>> 
>> On Aug 8, 2013, at 14:05 , Jason Smith <jhs@apache.org> wrote:
>> 
>>> Note, I intentially began this discussion as a question, not a statement.
>>> Totally thinking "aloud."
>> 
>> Same here, thanks for exploring this. I do like `on` :)
>> 
>> Jan
>> --
>> 
>>> 
>>> I think it is like gen_server. For gen_server all you need is a
>>> handle_call/3 and handle_cast/2. The real API is inside that.
>>> 
>>> For example, maybe all that's required for the behaviour is on/1 and we can
>>> make a flexible hook system from that. (Now why would "on" be a meaningful
>>> or useful function name?)
>>> 
>>> on({log, Severity, Message}) -> ok
>>>   , io:format("Couch says ~s\n", [Message])
>>>   ;
>>> 
>>> % Suppose I want a database that stays synced to my config
>>> % (something I personally need at the moment).
>>> 
>>> on({startup}) -> ok
>>>   , io:format("CouchDB started!\n")
>>>   , init_my_config_db(couch_config:get("*/*")) % Pseudocode
>>>   ;
>>> 
>>> on({config, Section, Key, Value}) -> ok
>>>   , io:format("Config update: ~s/~s = ~p\n", [Section, Key, Value])
>>>   , update_my_config_db(Section, Key, Value)
>>>   ;
>>> 
>>> on(Unknown) -> ok
>>>   , io:format("Did you know there is a ~p hook?\n", [element(1, Unknown)])
>>>   .
>>> 
>>> 
>>> 
>>> On Thu, Aug 8, 2013 at 5:42 PM, Jan Lehnardt <jan@apache.org> wrote:
>>> 
>>>> 
>>>> On Aug 8, 2013, at 12:39 , Jason Smith <jhs@apache.org> wrote:
>>>> 
>>>>> Well, I just googled it. Basically there is a couchdb_plugin.erl which
>>>>> tells Erlang what a behavior looks like. And all that does is define
the
>>>>> functions and arity which a couchdb_plugin would have to export.
>>>>> 
>>>>> Probably there are some better Erlangers on the list who might chime
in.
>>>> It
>>>>> looks like okay bang-for-buck; only not much bang or much buck.
>>>> 
>>>> how would this work if a plugin is only interested in handling a single
>>>> hook,
>>>> would it have to implement mock funs for all hooks then?
>>>> 
>>>>> On Thu, Aug 8, 2013 at 5:26 PM, Jan Lehnardt <jan@apache.org> wrote:
>>>>> 
>>>>>> how would this look in code?
>>>>>> 
>>>>>> On Aug 8, 2013, at 12:21 , Jason Smith <jhs@apache.org> wrote:
>>>>>> 
>>>>>>> Perhaps a custom behaviour to help catch API problems at compile
time?
>>>>>>> 
>>>>>>> -behaviour(couchdb_plugin).
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> On Thu, Aug 8, 2013 at 3:47 PM, Jan Lehnardt <jan@apache.org>
wrote:
>>>>>>> 
>>>>>>>> Heya,
>>>>>>>> 
>>>>>>>> I’m toying with the idea of moving some of my experimental
into
>>>>>>>> bona-fide plugins. One of them is my log_to_db branch that
on top of
>>>>>>>> writing log messages to a text file also writes a document
to a log
>>>>>>>> database.
>>>>>>>> 
>>>>>>>> Conceptually, this is the perfect use of a plugin: the feature
is not
>>>>>>>> useful in the general case, because *any* activity creates
write load
>>>>>>>> on a single database, but for certain low-volume installations,
this
>>>>>>>> might be a useful feature (I wouldn’t have written it,
if I hadn’t
>>>>>>>> needed it at some point) so allowing users to enable it as
a plugin
>>>>>>>> would be nice.
>>>>>>>> 
>>>>>>>> But regardless of whether my plugin is useful, it illustrates
an
>>>>>>>> interesting point:
>>>>>>>> 
>>>>>>>> A log_to_db plugin would need to register for logging events
or, if it
>>>>>>>> doesn’t want to duplicate all the logging-level logic in
couch_log, it
>>>>>>>> would need some way of injecting a function call into
>>>>>>>> `couch_log:log().`. We could of course try and find a way
where a
>>>>>>>> plugin would be able to provide an API compatible version
of a CouchDB
>>>>>>>> module and swap it out for it’s custom one, but that’s
hardly a great
>>>>>>>> idea.
>>>>>>>> 
>>>>>>>> Other software has the notion of “hooks” (some may call
it something
>>>>>>>> else) where at well defined points in the main code base,
external
>>>>>>>> functions get called with certain parameters. To make things
dynamic,
>>>>>>>> there might be a way for plugins to register to be called
by those
>>>>>>>> hooks and the main code then asks the registry whether there
are any
>>>>>>>> plugin functions to call.
>>>>>>>> 
>>>>>>>> In the log_to_db example, we’d have something like this:
>>>>>>>> 
>>>>>>>> couch_log_to_db.erl:
>>>>>>>> 
>>>>>>>> init() ->
>>>>>>>>     couch_hooks:register(couch_log_hook, log_hook_fun/1),
>>>>>>>>     ok.
>>>>>>>> 
>>>>>>>> log_hook_fun(Log) ->
>>>>>>>>     % do the log_to_db magic
>>>>>>>>     ok.
>>>>>>>> 
>>>>>>>> 
>>>>>>>> couch_hooks.erl:
>>>>>>>> 
>>>>>>>> register(Hook, Fun) ->
>>>>>>>>     % store the Fun with the Hook somewhere
>>>>>>>>     ok.
>>>>>>>> 
>>>>>>>> call(Hook, Args) ->
>>>>>>>>      % retrieve Fun for Hook from somewhere
>>>>>>>>     Fun(Args).
>>>>>>>> 
>>>>>>>> couch_log.erl:
>>>>>>>> 
>>>>>>>> % in log()
>>>>>>>> 
>>>>>>>> ...
>>>>>>>> couch_hooks:call(couch_log_hook, Args),
>>>>>>>> ...
>>>>>>>> 
>>>>>>>> The main code would define what the hook name and arguments
are and
>>>> the
>>>>>>>> plugin would have to conform. The plugin registry would just
manage
>>>> the
>>>>>>>> registration and calling of functions for a hook, but nothing
more.
>>>>>>>> 
>>>>>>>> * * *
>>>>>>>> 
>>>>>>>> This is just my first stab at this not thinking about it
too much and
>>>> I
>>>>>>>> likely miss some subtleties in Erlang that make this not
work (hot
>>>> code
>>>>>>>> upgrades e.g.).
>>>>>>>> 
>>>>>>>> 
>>>>>>>> How do you think we should implement a hooks feature in CouchDB?
>>>>>>>> 
>>>>>>>> 
>>>>>>>> Thanks!
>>>>>>>> Jan
>>>>>>>> --
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>> 
>>>>>> 
>>>> 
>>>> 
>> 


Mime
View raw message