chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ari Rabkin (JIRA)" <>
Subject [jira] Commented: (CHUKWA-421) new logfile rotation detection scheme
Date Thu, 03 Dec 2009 21:02:20 GMT


Ari Rabkin commented on CHUKWA-421:

I am NOT proposing to increment a sequence id on the file names.  I'm proposing to use the
file mod date plus length to figure out which rotated files are how old.  Hash of last record
(or last 200 bytes or whatever) is also an option.  My intent is to make the code as modular
as possible so that we can try out a couple different approaches.

> new logfile rotation detection scheme
> -------------------------------------
>                 Key: CHUKWA-421
>                 URL:
>             Project: Hadoop Chukwa
>          Issue Type: Improvement
>          Components: data collection
>            Reporter: Ari Rabkin
>            Assignee: Ari Rabkin
> Currently, we keep files open in order to detect rotation. This is wasteful of file handles,
doesn't work correctly across reboots, and I think we can do better.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message