couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Filipe Manana (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (COUCHDB-1218) Better logger performance
Date Wed, 13 Jul 2011 11:32:00 GMT

    [ https://issues.apache.org/jira/browse/COUCHDB-1218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13064495#comment-13064495
] 

Filipe Manana commented on COUCHDB-1218:
----------------------------------------

Yep, disk_log was made with the purpose of logging into files. It has many features, like
log rotation etc. It can log terms or raw data (i'm using the later), it can "repair" log
files, etc. I'm using it in the simplest way possible, to achieve exactly the same of what
is being done currently by couch_log.

I haven't seen increase in cpu and memory usage (via dstat and htop) compared to current trunk.
The async api basically puts the messages into a queue and then a disk_log worker is constantly
dequeing from that queue and writing to the file.

Something plugabble seems like a completely different issue and it's not what I'm trying to
address here. Plus depending on syslog, or something else external, doesn't seem a good thing
by default for me - how would it work on Windows, or mobile?

That said, I like the simplicity of our logger - nothing fancy, small code and plain text
files.

> Better logger performance
> -------------------------
>
>                 Key: COUCHDB-1218
>                 URL: https://issues.apache.org/jira/browse/COUCHDB-1218
>             Project: CouchDB
>          Issue Type: Improvement
>            Reporter: Filipe Manana
>            Assignee: Filipe Manana
>         Attachments: 0001-Better-logger-performance.patch
>
>
> I made some experiments with OTP's disk_log module (available since 2001 at least) to
use it to manage the log file.
> It turns out I got better throughput by using it. Basically it adopts a strategy similar
to the asynchronous couch_file Damien described in this thread:
> http://mail-archives.apache.org/mod_mbox/couchdb-dev/201106.mbox/%3C5C39FB5A-0ACA-4FF9-BD90-2EBECF271850@apache.org%3E
> Here's a benchmark with relaximation, 50 writers, 100 readers, documents of 1Kb, delayed_commits
set to false and 'info' log level (default):
> http://graphs.mikeal.couchone.com/#/graph/9e19f6d9eeb318c70cabcf67bc013c7f
> The reads got a better throughput (bottom graph, easier to visualize).
> The patch (also attached here), which has a descriptive comment, is at:
> https://github.com/fdmanana/couchdb/compare/logger_perf.patch

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message