chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sourygna Luangsay (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CHUKWA-675) Bad sleeping condition in FileTailer.java
Date Mon, 17 Dec 2012 06:14:12 GMT

    [ https://issues.apache.org/jira/browse/CHUKWA-675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13533671#comment-13533671
] 

Sourygna Luangsay commented on CHUKWA-675:
------------------------------------------

I haven't tried it in production yet. Nonetheless, I have included this change in our benchmarks
configuration:
- without the change, the agent needs more or less 30 minutes to send one 490MB file with
the file tailer adaptor.
- with the change, it only needs 45 seconds

I don't really understand what you mean by "rate limiting". The code already offers some protection
to ensure that one (big) file to tail may not prevent other smaller files to be tailed (see
chukwaAgent.fileTailingAdaptor.maxReadSize).

What is more, I think my change is correct if you consider the comment at the beginning of
the class:
* For now, it tries each file in succession. If it gets through every file
* within two seconds, and no more data remain, it will sleep.
* 
* If there was still data available in any file, the adaptor will loop again.

                
> Bad sleeping condition in FileTailer.java
> -----------------------------------------
>
>                 Key: CHUKWA-675
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-675
>             Project: Chukwa
>          Issue Type: Bug
>          Components: Data Collection
>    Affects Versions: 0.6.0
>            Reporter: Sourygna Luangsay
>            Assignee: Sourygna Luangsay
>            Priority: Minor
>             Fix For: 0.6.0
>
>         Attachments: chukwa-675.patch
>
>
> Current "if condition" in the run() method is wrong.
> As a consequence, a big file to tail will take a lot of time.
> See: http://mail-archives.apache.org/mod_mbox/incubator-chukwa-dev/201212.mbox/%3CCAOTz1wD-abkJwh5GBnUi3o6xj2kiqYf7HvGDg80mdEk4h6y6+w@mail.gmail.com%3E

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message