chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jerome Boulon (JIRA)" <j...@apache.org>
Subject [jira] Commented: (CHUKWA-265) FileAdaptor does not remove cleanly from chukwa agent
Date Thu, 28 May 2009 17:25:45 GMT

    [ https://issues.apache.org/jira/browse/CHUKWA-265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12714069#action_12714069
] 

Jerome Boulon commented on CHUKWA-265:
--------------------------------------

It's working fine for me. Tested using chukwa-0.1.2-rc4 
Time between the 2 commands was less that 10 seconds.

So can you do a list before adding the same file again? If the file is still there then, you
could use the agent.log to validate if the file has already been processed.
You should see something like:
2009-05-28 17:17:22,518 INFO Thread-39 FileAdaptor - Adding /homes/jboulon/test.txt to the
queue
2009-05-28 17:17:22,518 INFO Thread-39 FileAdaptor - /homes/jboulon/test.txt added to the
queue
2009-05-28 17:17:22,518 INFO Thread-39 ChukwaAgent - hardStop on adaptorId: 11, JobConf 0
/homes/jboulon/test.txt 10

If not then make sure that you inMemoryQueue is not full.


-- Trace -- 
Connected to localhost (127.0.0.1).
Escape character is '^]'.
add org.apache.hadoop.chukwa.datacollection.adaptor.FileAdaptor JobConf 0 /homes/jboulon/test.txt
0
OK add completed; new ID is 12
list
1)  org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped
chukwaAgent 0 /grid/0/chukwa/var/log/metrics/chukwa-gmon-chukwaAgent.log 381984
2)  org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped
Ps 0 /grid/0/chukwa/var/log/Ps.log 50995095
3)  org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped
chunkQueue 0 /grid/0/chukwa/var/log/metrics/chukwa-gmon-chunkQueue.log 378774
4)  org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped
chukwaHttpSender 0 /grid/0/chukwa/var/log/metrics/chukwa-gmon-chukwaHttpSender.log 599964
5)  org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped
Df 0 /grid/0/chukwa/var/log/Df.log 559980
6)  org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped
Top 0 /grid/0/chukwa/var/log/Top.log 17933888
7)  org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped
Netstat 0 /grid/0/chukwa/var/log/Netstat.log 112626
8)  org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped
Iostat 0 /grid/0/chukwa/var/log/Iostat.log 1509872
9)  org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped
Sar 0 /grid/0/chukwa/var/log/Sar.log 2817132
10)  org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped
HDFSUsage 0 /grid/0/chukwa/var/log/HDFSUsage.log 275070

add org.apache.hadoop.chukwa.datacollection.adaptor.FileAdaptor JobConf 0 /homes/jboulon/test.txt
0
OK add completed; new ID is 13


> FileAdaptor does not remove cleanly from chukwa agent
> -----------------------------------------------------
>
>                 Key: CHUKWA-265
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-265
>             Project: Hadoop Chukwa
>          Issue Type: Bug
>          Components: data collection
>    Affects Versions: 0.1.2, 0.2.0
>         Environment: Redhat EL 5.1, Java 6
>            Reporter: Eric Yang
>            Priority: Blocker
>
> When FileAdaptor shutdown, it should remove itself from adaptorPositions and adaptorByNumber.
 This doesn't appear to be happening correctly, and adding the same file for streaming fails.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message