chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sourygna Luangsay (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CHUKWA-686) Creating a new adaptor: FileTailingAdaptorPreserveLines
Date Sun, 21 Apr 2013 14:29:15 GMT

     [ https://issues.apache.org/jira/browse/CHUKWA-686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Sourygna Luangsay updated CHUKWA-686:
-------------------------------------

    Description: 
FileTailingAdaptor is great to tail log files and send them to Hadoop.
However, last line of the chunk is usually cut which leads to some errors.

CharFileTailingAdaptorUTF8 can solve such problem.
Nonetheless, this adaptor calls the MapProcessor.process() method for every
line in each chunk, thus slowing a lot the Demux phase.

This adaptor that mix the benefits of the two
adaptors: the (Demux) speed of FileTailingAdaptor and
the preservation of lines from CharFileTailingAdaptorUTF8.
    
> Creating a new adaptor: FileTailingAdaptorPreserveLines
> -------------------------------------------------------
>
>                 Key: CHUKWA-686
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-686
>             Project: Chukwa
>          Issue Type: Improvement
>          Components: Data Collection
>            Reporter: Sourygna Luangsay
>            Assignee: Sourygna Luangsay
>            Priority: Minor
>             Fix For: 0.6.0
>
>
> FileTailingAdaptor is great to tail log files and send them to Hadoop.
> However, last line of the chunk is usually cut which leads to some errors.
> CharFileTailingAdaptorUTF8 can solve such problem.
> Nonetheless, this adaptor calls the MapProcessor.process() method for every
> line in each chunk, thus slowing a lot the Demux phase.
> This adaptor that mix the benefits of the two
> adaptors: the (Demux) speed of FileTailingAdaptor and
> the preservation of lines from CharFileTailingAdaptorUTF8.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message