couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jan Lehnardt <...@apache.org>
Subject Re: Logging into a database
Date Sun, 18 Oct 2009 16:10:10 GMT

On 18 Oct 2009, at 17:07, Adam Kocoloski wrote:

> On Oct 17, 2009, at 9:15 PM, Jan Lehnardt wrote:
>
>> Hey all,
>>
>> I had an itch to scratch and now there is an experimental branch  
>> that enables logging to a database:
>>
>> http://github.com/janl/couchdb/commits/log-to-db
>>
>> Check out the test case on how it is supposed to work:
>>
>> http://github.com/janl/couchdb/blob/5cbfbeaddea2152e35b88cae515152f78d6b7544/share/www/script/test/log_database.js
>>
>> You just create a `database = dbname` entry in your configuration  
>> under the `[log]` section and CouchDB will start to write log  
>> entries to the specified database.
>>
>> Depending on the log level this means that each single request to  
>> CouchDB triggers multiple writes to the log database. The logger,  
>> for now, defaults to delayed_commits. A future improvement, could  
>> be using the `batch=ok` module.
>>
>> I just wanted to throw this out here inspired by Chris' "releasing  
>> things before I go to sleep for the night".
>>
>> Let me know what you think.
>>
>> Cheers
>> Jan
>> --
>
> Hi Jan, cool!  I do have a word of caution, though ...
>
> I tried this a long time ago but never released it because it caused  
> the logging queue to back up pretty badly under a moderate read  
> load.  Logging events are asynchronous (as they should be), so if  
> the logging event handler is slower than the system generating the  
> events its entirely possible for the message queue in the gen_event  
> manager to back up.  Unfortunately, Erlang has this ugly positive  
> feedback loop where a large message queue will dramatically slow  
> down that process.  Eventually (assuming a constant read load), the  
> gen_event manager process used all available memory and things went  
> kaboom.
>
> Now, that was a long time ago (pre-0.9 IIRC) -- perhaps the  
> performance profile has changed and logging to a DB is easy now.  In  
> particular, I'm sure delayed_commits help a ton.  But I would be  
> careful to try this setting out under a heavy read load and make  
> sure that the system stays stable before adding it to trunk.  Cheers,

Yeah, this is definitely meant for low volume sites. I use this for my  
personal URL shortening service to see how often a short link got  
clicked. I could get the same from parsing the logfile but I thought  
the log-to-db feature was neat and easy to do.

Cheers
Jan
--


Mime
View raw message