couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Benoit Chesneau <bchesn...@gmail.com>
Subject Re: Logging into a database
Date Sun, 18 Oct 2009 15:31:57 GMT
On Sun, Oct 18, 2009 at 5:07 PM, Adam Kocoloski <kocolosk@apache.org> wrote:
> On Oct 17, 2009, at 9:15 PM, Jan Lehnardt wrote:
>
>> Hey all,
>>
>> I had an itch to scratch and now there is an experimental branch that
>> enables logging to a database:
>>
>>  http://github.com/janl/couchdb/commits/log-to-db
>>
>> Check out the test case on how it is supposed to work:
>>
>>
>> http://github.com/janl/couchdb/blob/5cbfbeaddea2152e35b88cae515152f78d6b7544/share/www/script/test/log_database.js
>>
>> You just create a `database = dbname` entry in your configuration under
>> the `[log]` section and CouchDB will start to write log entries to the
>> specified database.
>>
>> Depending on the log level this means that each single request to CouchDB
>> triggers multiple writes to the log database. The logger, for now, defaults
>> to delayed_commits. A future improvement, could be using the `batch=ok`
>> module.
>>
>> I just wanted to throw this out here inspired by Chris' "releasing things
>> before I go to sleep for the night".
>>
>> Let me know what you think.
>>
>> Cheers
>> Jan
>> --
>
> Hi Jan, cool!  I do have a word of caution, though ...
>
> I tried this a long time ago but never released it because it caused the
> logging queue to back up pretty badly under a moderate read load.  Logging
> events are asynchronous (as they should be), so if the logging event handler
> is slower than the system generating the events its entirely possible for
> the message queue in the gen_event manager to back up.  Unfortunately,
> Erlang has this ugly positive feedback loop where a large message queue will
> dramatically slow down that process.  Eventually (assuming a constant read
> load), the gen_event manager process used all available memory and things
> went kaboom.

maybe using gen_event2 from rabbitmq project would solve that since it buffer ?
http://hg.rabbitmq.com/rabbitmq-server/file/4bb959b80bac/src/gen_server2.erl

- benoit

Mime
View raw message