chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Yang (JIRA)" <>
Subject [jira] Commented: (CHUKWA-185) ability to tail a whole directory
Date Mon, 15 Jun 2009 19:42:07 GMT


Eric Yang commented on CHUKWA-185:

If we go with time window approach, DTA will only work on files that have active updates.
 What if the user want to stream over files that were previously archived and no longer receiving
updates?  This is not in the previous identified use case, but it may make sense to include
this use case.

If we specify start time (as processed time flag), and time window size, the system could
process data in a queue and try to closing the gap between past and present.  The processed
time flag could be used as an indicator for resuming agent crash as well.

> ability to tail a whole directory
> ---------------------------------
>                 Key: CHUKWA-185
>                 URL:
>             Project: Hadoop Chukwa
>          Issue Type: New Feature
>          Components: data collection
>    Affects Versions: 0.1.2, 0.2.0
>            Reporter: Ari Rabkin
>            Assignee: Ari Rabkin
>             Fix For: 0.3.0
>         Attachments: CHUKWA-185.patch
> Right now, FileTailingAdaptors watch particular files.   It'd be great to be able to
watch a whole path: to say something like /var/logs/*, where new logs created in that directory
get picked up.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message