incubator-chukwa-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ariel Rabkin <asrab...@gmail.com>
Subject Re: Problem in collecting data from Agent
Date Fri, 19 Nov 2010 07:07:28 GMT
You apparently have your agent set to use the console connector. This
means that the agent is writing to standard out, not to HDFS. Did you
specify "local" when starting the agent?

--Ari

On Thu, Nov 18, 2010 at 10:59 PM, <VenkataHariShankar.S@cognizant.com> wrote:
>
> Hi All,
>
>
>
> When I try to send a log file from Agent to Collector, I didn’t get anything in HDFS
expect following two file,(it also have no data inside that)
>
> Please let me know, how to monitor wheather collector is receiving data..
>
>
>
> $ ./hadoop fs -ls /chukwa/logs/
>
> Found 2 items
>
> -rw-r--r--   3 hadoop supergroup          0 2010-11-19 17:14 /chukwa/logs/201019171406118_HADOOPSQOOP_4b2656712c63c0713a7fea.chukwa
>
> -rw-r--r--   3 hadoop supergroup        103 2010-11-19 17:25 /chukwa/logs/201019172022791_HADOOPSQOOP_1d40ead912c63f88f598000.done
>
>
>
> I was trying to send Iostat.log file which is given in chukwa/test/samples directory.
>
>
>
>
>
> Using the adapter
>
>
>
> add filetailer.FileTailingAdaptor logs /tmp/Iostat.log 0
>
> OK add completed; new ID is 4b08ea7f0635fff5d4de778f6782bd1b
>
>
>
> Collector.log Output:
>
> 2010-11-19 17:44:50,418 INFO Timer-3 LocalWriter - stat:datacollection.writer.local.LocalWriter
dataSize=0 dataRate=0
>
> 2010-11-19 17:45:18,183 INFO Timer-1 root - stats:ServletCollector,numberHTTPConnection:0,numberchunks:0
>
> 2010-11-19 17:45:19,243 INFO Timer-2 LocalWriter - start Date [Fri Nov 19 17:45:19 IST
2010]
>
> 2010-11-19 17:45:19,243 INFO Timer-2 LocalWriter - Rotate from Timer-2
>
> 2010-11-19 17:45:19,247 INFO Timer-2 LocalWriter - no chunks written to /tmp/chukwa/dataSink/201019174019240_HADOOPSQOOP_4c320ac712c63fd13837ffa.chukwa,
deleting
>
> 2010-11-19 17:45:20,419 INFO Timer-3 LocalWriter - stat:datacollection.writer.local.LocalWriter
dataSize=0 dataRate=0
>
> 2010-11-19 17:45:50,420 INFO Timer-3 LocalWriter - stat:datacollection.writer.local.LocalWriter
dataSize=0 dataRate=0
>
> 2010-11-19 17:46:18,184 INFO Timer-1 root - stats:ServletCollector,numberHTTPConnection:0,numberchunks:0
>
> 2010-11-19 17:46:20,420 INFO Timer-3 LocalWriter - stat:datacollection.writer.local.LocalWriter
dataSize=0 dataRate=0
>
>
>
> Agent Console Output:
>
> …….
>
> data type was logs
>
> data length was 131072, not printing
>
> -------------------
>
> Console out connector got event at offset 1441792
>
> data type was logs
>
> data length was 131072, not printing
>
> -------------------
>
> Console out connector got event at offset 1572864
>
> data type was logs
>
> data length was 131072, not printing
>
> -------------------
>
> Console out connector got event at offset 1703936
>
> data type was logs
>
> data length was 131072, not printing
>
> -------------------
>
> Console out connector got event at offset 1740827
>
> data type was logs
>
> data length was 36891, not printing
>
>
>
> Thanks & Regards
>
> Venkat


--
Ari Rabkin asrabkin@gmail.com
UC Berkeley Computer Science Department

Mime
View raw message