couchdb-erlang mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dave Cottlehuber <...@jsonified.com>
Subject Re: Learning Erlang/OTP and CouchDB internals together
Date Mon, 19 Nov 2012 14:16:57 GMT
On 19 November 2012 10:13, Binbin Wang <binbinwang118@gmail.com> wrote:
> Replying the question *"2. how they're subsequently mutated/abused over the
> _config REST API?*". Does it means the codepath of the _config rest api? If
> it is, the excellent explanation of Jan on the codepath of PUT /DB/DocID
> will help to better understand other rest api, such as the  _config.

Binbin, looking good!

Yup I was referring to the _config API, 2 things below that I found
interesting/awkward.

1. _config uses strings, OTP uses atoms for the keys.
2. OTP from what I can see doesn't offer the ability to subscribe to
config changes and take action. But couch does.

So for my homework;

1. how are configs loaded at startup in CouchDB?

from the erl commandline, the couch application is started and ini
files passed. The may override what's in couch.app

...
    {mod, {couch_app, [
        "/usr/local/etc/couchdb/default.ini",
        "/usr/local/etc/couchdb/local.ini"
    ]}},

Which uses the typical OTP style {some_atom, ["array","of", "stuff"]}. See
http://www.erlang.org/doc/design_principles/applications.html and
http://learnyousomeerlang.com/building-otp-applications#the-application-resource-file

These config files are then loaded into ets tables. See Binbin's
excellent answers :-)

2. how they're subsequently mutated/abused over the _config REST API?

during loading of ini files, (etc/couchdb/default.ini) is used to
register a new handler

     _config = {couch_httpd_misc_handlers, handle_config_req}

So let's head over to couch_httpd_misc_handlers:handle_config_req and see
what happens.

The mochi req is parsed, matched, and then minor munging of config
parameters followed
by calling couch_config:set(…) which promptly sends itself a message:

    gen_server:call(?MODULE, {set, Section, Key, Value, Persist}).

Note that this is the function that would need to change to be able to
handle atoms instead of strings/binaries to share config settings
between app.config and the HTTP API.

strings in, convert to binary, update ets tables, then trigger callbacks:

        true = ets:insert(?MODULE, {{Sec, Key}, Val}),
        spawn_link(fun() ->
            [catch F(Sec, Key, Val, Persist) || {_Pid, F} <-
Config#config.notify_funs],
                gen_server:reply(From, ok)
        end),

Nifty.

3. how are these events managed elsewhere e.g. in couch httpd when IP changes?

I think this is quite clever: in couch_httpd.erl we:

- register a callback

couch_config:register(fun ?MODULE:config_change/2, Pid),

- and have the exported function following:

config_change("httpd", "bind_address") ->
    ?MODULE:stop();

?MODULE:config_change provides an API to manage the updates. In this
case it simply stops the module, and relies on the supervision tree to
restart itself cleanly. See previous question for how these callbacks
are called.

So that's the sort of thing I would need for a pluggable metrics API then.

4. compare with the sys.config approach used in OTP [1]

The interesting thing here is that everything that comes in over
_config is treated as a string, but stored in ETS as binaries. This is
not compatible with the OTPish way of storing startup configurations
in an app.config or sys.config file, where typically those keys are
actually atoms.

5. can you see a way to have the best of both worlds?

It's clearly possible to use to_existing_atom calls for those strings,
mutating them in the config handler, and then potentially have the
best of both worlds.

I think the _config API is a great thing for CouchDB, and it
definitely avoids the need to understand erlang from most
users/admins. So we'd need a way to hook the _config API into updating
app.config both in memory, and either only using app.config (erlang
syntax) for couchdb, like riak and other erlang apps, or providing a
merge function that updates both ini and app.config in-memory.

A+
Dave

Mime
View raw message