couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam Kocoloski <kocol...@apache.org>
Subject Re: Logging into a database
Date Sun, 18 Oct 2009 15:07:13 GMT
On Oct 17, 2009, at 9:15 PM, Jan Lehnardt wrote:

> Hey all,
>
> I had an itch to scratch and now there is an experimental branch  
> that enables logging to a database:
>
>  http://github.com/janl/couchdb/commits/log-to-db
>
> Check out the test case on how it is supposed to work:
>
> http://github.com/janl/couchdb/blob/5cbfbeaddea2152e35b88cae515152f78d6b7544/share/www/script/test/log_database.js
>
> You just create a `database = dbname` entry in your configuration  
> under the `[log]` section and CouchDB will start to write log  
> entries to the specified database.
>
> Depending on the log level this means that each single request to  
> CouchDB triggers multiple writes to the log database. The logger,  
> for now, defaults to delayed_commits. A future improvement, could be  
> using the `batch=ok` module.
>
> I just wanted to throw this out here inspired by Chris' "releasing  
> things before I go to sleep for the night".
>
> Let me know what you think.
>
> Cheers
> Jan
> --

Hi Jan, cool!  I do have a word of caution, though ...

I tried this a long time ago but never released it because it caused  
the logging queue to back up pretty badly under a moderate read load.   
Logging events are asynchronous (as they should be), so if the logging  
event handler is slower than the system generating the events its  
entirely possible for the message queue in the gen_event manager to  
back up.  Unfortunately, Erlang has this ugly positive feedback loop  
where a large message queue will dramatically slow down that process.   
Eventually (assuming a constant read load), the gen_event manager  
process used all available memory and things went kaboom.

Now, that was a long time ago (pre-0.9 IIRC) -- perhaps the  
performance profile has changed and logging to a DB is easy now.  In  
particular, I'm sure delayed_commits help a ton.  But I would be  
careful to try this setting out under a heavy read load and make sure  
that the system stays stable before adding it to trunk.  Cheers,

Adam


Mime
View raw message