chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jerome Boulon (JIRA)" <j...@apache.org>
Subject [jira] Commented: (CHUKWA-421) new logfile rotation detection scheme
Date Thu, 03 Dec 2009 20:37:20 GMT

    [ https://issues.apache.org/jira/browse/CHUKWA-421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12785505#action_12785505
] 

Jerome Boulon commented on CHUKWA-421:
--------------------------------------

We can keep track of current offset and the last record and compare it.
This should work assuming that the date should not be the same and if it is and the record
is the same then for us it will be a duplicate anyway.


> new logfile rotation detection scheme
> -------------------------------------
>
>                 Key: CHUKWA-421
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-421
>             Project: Hadoop Chukwa
>          Issue Type: Improvement
>          Components: data collection
>            Reporter: Ari Rabkin
>
> Currently, we keep files open in order to detect rotation. This is wasteful of file handles,
doesn't work correctly across reboots, and I think we can do better.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message