chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ari Rabkin (JIRA)" <>
Subject [jira] Commented: (CHUKWA-332) Remove first offset tracker from CharFileTailingAdaptorUTF8NewLineEscaped adaptor
Date Thu, 25 Jun 2009 00:38:07 GMT


Ari Rabkin commented on CHUKWA-332:

Hrm.  I just want to understand how we currently fix the problem that parameter was there
to address.

- Suppose I have a file, foo.txt, containing three chars "abcd".  
- An adaptor looks at the file, and sends a new chunk, containing "abcd", ending at offset
- The file then rotates, and now contains "efg".  What should the adaptor send?  

> Remove first offset tracker from CharFileTailingAdaptorUTF8NewLineEscaped adaptor
> ---------------------------------------------------------------------------------
>                 Key: CHUKWA-332
>                 URL:
>             Project: Hadoop Chukwa
>          Issue Type: Improvement
>          Components: data collection
>    Affects Versions: 0.3.0
>         Environment: Redhat EL 5.1, Java 6
>            Reporter: Eric Yang
>             Fix For: 0.3.0
> CharFileTailingAdaptorUTF8NewLineEscaped is currently taking Starting offset of the file,
and current streaming offset of the file.  The original idea of using starting offset as a
unique sequence tracker to track the unique record for the life time of the stream.  However
that idea didn't pan out.  Hence, we should remove the first offset to avoid confusion.  The
current streaming offset is sufficient to start tailing file.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message