incubator-chukwa-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Yang <>
Subject Re: log rotation
Date Wed, 27 Jan 2010 18:42:07 GMT
Hi Corbin,

Chukwa is intended for your use case, but it's not perfect in term of
functionality.  There are two possible solutions to refine Chukwa to do what
you need.  

First solution, extend UTF8 File Tailing Adaptor to delete the processed
file, and use that to tail the apache log file.  This method should be
fairly easy to implement.

Second solution, write an adaptor to receive log from named pipe, and
configure Apache to pipe the log to a perl script which writes to the named
pipe.  This method is great for reducing disk load for the webserver.  It is
more tricky to get it right.

Hope this helps.


On 1/27/10 10:18 AM, "Corbin Hoenes" <> wrote:

> I've noticed some chatter about log rotation and FileTailingAdaptor:
> Is there some kind of documentation on how it deals with say a directory of
> files that get rotated.  At first glance it looks like it keeps an adaptor
> open forever and there doesn't appear to be a way to know that a file has been
> processed by the adaptor so we can't really move files out, etc...  Right now
> we are piping apache access logs through rotatelogs and then using chukwa to
> tail these files.  Is there a better way?

View raw message