chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stuti Awasthi (JIRA)" <j...@apache.org>
Subject [jira] Commented: (CHUKWA-580) Checkpoints increasing without sending data to collector
Date Tue, 18 Jan 2011 07:36:43 GMT

    [ https://issues.apache.org/jira/browse/CHUKWA-580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12983054#action_12983054
] 

Stuti Awasthi commented on CHUKWA-580:
--------------------------------------

Ok. Since this is the intended behaviour then it is not a bug. I did not found out any documentation
regarding this behaviour of DirTailing Adapter and since it minorly contradicts with theory
I had this query.

Thanks for explaining this behaviour.


> Checkpoints increasing without sending data to collector
> --------------------------------------------------------
>
>                 Key: CHUKWA-580
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-580
>             Project: Chukwa
>          Issue Type: Bug
>    Affects Versions: 0.4.0
>         Environment: RHEL
>            Reporter: Stuti Awasthi
>
> I have a query regarding the checkpoints in chukwa. According to theory :
> Every few minutes, each agent process polls a collector to find the length of each file
to which data is being written. The length of the file is then compared with the offset at
which each chunk was to be written. If the file length exceeds this value, then the data has
been committed and the agent process advances its checkpoint accordingly.(Note that the length
returned by the filesystem is the amount of data that has been successfully replicated.)
> This means that chukwa_agent_checkpoint would increase only when the agent receivers
and ack from the collectors. But in case of dirtailing adapter, this is not correct. I have
done the following steps to test this :
> -         Started agent with some dummy collector which was not present. 
> -         Added dirtailing adapter with Charfile tailing adapter
> I can see the following output in my checkpoint file :
> ADD adaptor_67653208e8dea46c798e46753fc19dad = org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8
Stuti 0 /root/Stuti/yum.log 0
> ADD adaptor_b505db62647203ffa3cfe17374042870 = org.apache.hadoop.chukwa.datacollection.adaptor.DirTailingAdaptor
Stuti /root/Stuti filetailer.CharFileTailingAdaptorUTF8 1295014173306
> Since data is not getting sent to collector, so checkpoints should not increase.
>  

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message