chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ari Rabkin (JIRA)" <j...@apache.org>
Subject [jira] Reopened: (CHUKWA-421) new logfile rotation detection scheme
Date Tue, 22 Dec 2009 20:52:29 GMT

     [ https://issues.apache.org/jira/browse/CHUKWA-421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ari Rabkin reopened CHUKWA-421:
-------------------------------


There's a subtle bug in how the new adaptor writes checkpoints. The last-mod date in the checkpoint
reflects data SENT, not data WRITTEN. So after a collector crash, this adaptor will resume
at the wrong place.  Fix is for the adaptor to keep a mapping from send offset to last-mod-date,
listen to commit reports, and write accordingly.

If other adaptors start needing to do this, we might need to pull this functionality out into
a utility class.

> new logfile rotation detection scheme
> -------------------------------------
>
>                 Key: CHUKWA-421
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-421
>             Project: Hadoop Chukwa
>          Issue Type: Improvement
>          Components: data collection
>            Reporter: Ari Rabkin
>            Assignee: Ari Rabkin
>         Attachments: CHUKWA-421.patch
>
>
> Currently, we keep files open in order to detect rotation. This is wasteful of file handles,
doesn't work correctly across reboots, and I think we can do better.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message