couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Dionne <>
Subject Re: The perfect logger for development
Date Wed, 07 Dec 2011 12:28:38 GMT
One of the things we do in BigCouch is attach a unique identifier to the request also, so that
we can correlate a given request with other log messages that may appear from other internal
components. We call it an X-Request-ID or some such thing and users can "curl -v" and tell
us what that is. It's great for debugging

On Dec 6, 2011, at 11:24 PM, Jason Smith wrote:

> Brilliant, thanks!
> I think this is possible if the Req object is part of the log object.
> Then a formatter can access it there. Off the top of my head, it would
> have access to the source IP address, the HTTP method, the path and
> query string, and headers.
> On Wed, Dec 7, 2011 at 10:44 AM, kowsik <> wrote:
>> As a CouchDB administrator, I would want *all* exception dumps to be
>> prefaced by the inbound request URL with the query parameters
>> (assuming it's a web request that caused the exception). There are
>> case where I've seen a stack trace but couldn't tell which inbound
>> request caused the problem.
>> K.
>> ---
>> @pcapr
>> On Tue, Dec 6, 2011 at 5:51 PM, Jason Smith <> wrote:
>>> Hi, all. Iris Couch urgently needs improved logging facilities in
>>> CouchDB. My goal is to make something we all love and get it accepted
>>> upstream, God willing. Or committers willing. But I repeat myself!
>>> This is the brainstorming and requirements gathering phase. In the
>>> CouchDB of your dreams, logging system fits you like an old pair of
>>> sneakers. It's perfect. Now, what characteristics does that system
>>> exhibit? I will compile feedback into a spec on the wiki.
>>> I hope to avoid bikeshedding. Seriously, please don't even mention a
>>> product or project by name. At this phase I hope to stick to
>>> descriptions of functionality, goals, and non-goals. I want to
>>> evaluate tools later.
>>> To start the discussion: logging is viewed differently based on your
>>> relationship with CouchDB:
>>> 1. Developers of CouchDB
>>> 2. Administrators of a couch
>>> 3. Application developers
>>> My roles are administration, and a little bit of development.
>>> Requirements, in no order.
>>> * Idiomatic Erlang
>>> * Is a compelling place for new people to contribute. Miguel de Icaza
>>> talks about this. It's not enough that the source code is public. You
>>> have to provide a smooth on-ramp, where people people get great bang
>>> for their buck. They write a modest feature, and are rewarded by
>>> seeing its effects immediately. In other words: plugins. Or maybe a
>>> behaviour. Or some way to swap in formatters and data sinks. I don't
>>> want to write a Loggly target ( Loggly should be
>>> begging me to merge their module.
>>> * 1st cut, no change to the default behavior. You still get the that
>>> peculiar log file you know and love. People are parsing their log
>>> files, and might expect version 1.x not to change.
>>> * Existing code still works. No sweeping changes hitting every
>>> (Filipe, would you please share your thoughts on these? I think you
>>> struggled with the conflict between them recently.)
>>> * No performance impact (non-blocking)...
>>> * ... but also, impossible or difficult to overwhelm or crash or lose logs.
>>> (The next few points sort of fit together)
>>> * Logs are not strings, but data structures, with data (the log
>>> message) and metadata (severity, line number, maybe the call stack,
>>> etc.)
>>> * More log levels. Roughly: trace, debug, info, warn, error, fatal
>>> * Maybe automatic trace logs upon function entry/exit with return
>>> values. Not sure if this is doable. Maybe a compile option, for `make
>>> dev`
>>> * When you log something, your module, line number, and maybe PID are known
>>> * "Components" or categories, or tags, where multiple .erl files or
>>> individual log events can share a common property ("http", "views",
>>> * A policy decides what to do with logs based on level, module, or
>>> component. You can set the policy either via configuration or
>>> programatically.
>>> * There is a formatter API to serialize log data. Built-in formatters
>>> include the legacy format, and Jan's Apache httpd combined format.
>>> * There is a transport API to receive logs and DTRT.
>>> * I know this is insane, but kill -HUP <pid> should make couch reopen
>>> its log files. Okay, I'll settle down now.
>>> = Non Goals =
>>> * Log rotation. I have never seen a rotation feature in an application
>>> which was better than the OS tools. And you can get problem where both
>>> the server and the OS are rotating the same logs. I have seen that
>>> happen, twice. Or was it once? Of course, people could write a
>>> rotating file transport.
>>> --
>>> Iris Couch
> -- 
> Iris Couch

View raw message