chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Yang (JIRA)" <j...@apache.org>
Subject [jira] Commented: (CHUKWA-136) Duplicated adaptors
Date Thu, 16 Apr 2009 20:26:14 GMT

    [ https://issues.apache.org/jira/browse/CHUKWA-136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12699843#action_12699843
] 

Eric Yang commented on CHUKWA-136:
----------------------------------

Let's assume the checkpoint file writes:

add org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped
chukwaAgent 21281 /grid/0/chukwa/var/log/metrics/chukwa-gmon-chukwaAgent.log 38209

And next request adds:

add org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped
chukwaAgent 0 /grid/0/chukwa/var/log/metrics/chukwa-gmon-chukwaAgent.log 0

This would not be considered as duplicated streams from the patch above.  We may have to do
duplicate detection at adaptor level because the parameters passed to the adaptor are non-uniform.
 However, be careful of the implementation.  We don't want too much cost in doing the adaptor
duplication detection.  Add Ari, Jerome to charm in.

> Duplicated adaptors
> -------------------
>
>                 Key: CHUKWA-136
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-136
>             Project: Hadoop Chukwa
>          Issue Type: Bug
>          Components: data collection
>            Reporter: Cheng
>            Assignee: Cheng
>            Priority: Blocker
>         Attachments: chukwa-136.patch
>
>
> I did a telnet list command, it shows
> telnet localhost 9093
> Trying 127.0.0.1...
> Connected to localhost (127.0.0.1).
> Escape character is '^]'.
> list
> 1)  org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped
Iostat 0 /grid/0/chukwa/var/log/Iostat.log 145600
> 2)  org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped
Top 0 /grid/0/chukwa/var/log/Top.log 1742264
> 3)  org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped
chunkQueue 9339 /grid/0/chukwa/var/log/metrics/chukwa-gmon-chunkQueue.log 36825
> 4)  org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped
chukwaHttpSender 14777 /grid/0/chukwa/var/log/metrics/chukwa-gmon-chukwaHttpSender.log 58005
> 5)  org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped
chukwaAgent 12449 /grid/0/chukwa/var/log/metrics/chukwa-gmon-chukwaAgent.log 38209
> 6)  org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped
chunkQueue 12061 /grid/0/chukwa/var/log/metrics/chukwa-gmon-chunkQueue.log 36825
> 7)  org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped
Sar 0 /grid/0/chukwa/var/log/Sar.log 271400
> 8)  org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped
chukwaAgent 12817 /grid/0/chukwa/var/log/metrics/chukwa-gmon-chukwaAgent.log 38209
> 9)  org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped
chukwaHttpSender 19221 /grid/0/chukwa/var/log/metrics/chukwa-gmon-chukwaHttpSender.log 58005
> 10)  org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped
Ps 0 /grid/0/chukwa/var/log/Ps.log 3332933
> 11)  org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped
chukwaAgent 13185 /grid/0/chukwa/var/log/metrics/chukwa-gmon-chukwaAgent.log 38209
> 12)  org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped
Netstat 0 /grid/0/chukwa/var/log/Netstat.log 15537
> 13)  org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped
chunkQueue 11788 /grid/0/chukwa/var/log/metrics/chukwa-gmon-chunkQueue.log 36825
> 14)  org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped
chunkQueue 4767 /grid/0/chukwa/var/log/metrics/chukwa-gmon-chunkQueue.log 36825
> 15)  org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped
Df 0 /grid/0/chukwa/var/log/Df.log 54540
> 17)  org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped
chukwaAgent 9873 /grid/0/chukwa/var/log/metrics/chukwa-gmon-chukwaAgent.log 38209
> 16)  org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped
chukwaHttpSender 18887 /grid/0/chukwa/var/log/metrics/chukwa-gmon-chukwaHttpSender.log 58005
> 19)  org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped
chukwaAgent 0 /grid/0/chukwa/var/log/metrics/chukwa-gmon-chukwaAgent.log 38209
> 18)  org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped
chunkQueue 12334 /grid/0/chukwa/var/log/metrics/chukwa-gmon-chunkQueue.log 36825
> 21)  org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped
chukwaHttpSender 6077 /grid/0/chukwa/var/log/metrics/chukwa-gmon-chukwaHttpSender.log 58005
> 20)  org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped
chukwaHttpSender 18553 /grid/0/chukwa/var/log/metrics/chukwa-gmon-chukwaHttpSender.log 58005
> 23)  org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped
chunkQueue 20207 /grid/0/chukwa/var/log/metrics/chukwa-gmon-chunkQueue.log 36825
> 22)  org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped
chukwaAgent 21281 /grid/0/chukwa/var/log/metrics/chukwa-gmon-chukwaAgent.log 38209
> 25)  org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped
CC 0 /grid/0/chukwa/var/log/collector.log 1040900
> 24)  org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped
chukwaHttpSender 31669 /grid/0/chukwa/var/log/metrics/chukwa-gmon-chukwaHttpSender.log 58005
> chukwAgent is listed 6 times.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message