incubator-chukwa-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shouvanik Haldar <shouvanik.hal...@gmail.com>
Subject How to check whether chukwa collecting logs
Date Wed, 09 Jan 2013 10:54:22 GMT
I am trying to find out a check point , so that I can say that Chukwa is
running and collecting logs


I configured the /chukwa/etc/chukwa/initial_adaptors
with following

add FileTailer
/home/ec2-user/hadoop-1.0.4/logs/hadoop-ec2-user-datanode-domU-12-31-39-07-19-3B.log
0

.
then I started bin/chukwa agent.

Started the hadoop daemons.

then, bin/chukwa collector.

Next, I check the site
"http://ec2-204-236-192-203.compute-1.amazonaws.com:8080/chukwa?ping=true"

with following statistics

Date:1357728723103
Now:1357728765604
numberHTTPConnection in time window:17
numberchunks in time window:119
lifetimechunks:23029


and when I check hdfs I see

-rw-r--r--   1 ec2-user supergroup     657818 2013-01-08 11:09
/chukwa/logs/201308110441175_domU12313907193B_216836a313c19d565ee8000.done
-rw-r--r--   1 ec2-user supergroup     483520 2013-01-08 11:14
/chukwa/logs/201308110941296_domU12313907193B_216836a313c19d565ee7ffe.done
-rw-r--r--   1 ec2-user supergroup     483155 2013-01-08 11:19
/chukwa/logs/201308111441296_domU12313907193B_216836a313c19d565ee7ffc.done
-rw-r--r--   1 ec2-user supergroup     463708 2013-01-08 11:24
/chukwa/logs/201308111941296_domU12313907193B_216836a313c19d565ee7ffa.done
-rw-r--r--   1 ec2-user supergroup     386337 2013-01-08 11:36
/chukwa/logs/201308113134948_domU12313907193B_488afeb413c19ee05bf8000.done
-rw-r--r--   1 ec2-user supergroup     479656 2013-01-08 11:41
/chukwa/logs/201308113635034_domU12313907193B_488afeb413c19ee05bf7ffe.done
-rw-r--r--   1 ec2-user supergroup     487748 2013-01-08 11:46
/chukwa/logs/201308114135034_domU12313907193B_488afeb413c19ee05bf7ffb.done
-rw-r--r--   1 ec2-user supergroup     479696 2013-01-08 11:51
/chukwa/logs/201308114635034_domU12313907193B_488afeb413c19ee05bf7ff8.done
-rw-r--r--   1 ec2-user supergroup     488030 2013-01-08 11:56
/chukwa/logs/201308115135034_domU12313907193B_488afeb413c19ee05bf7ff5.done
-rw-r--r--   1 ec2-user supergroup     479754 2013-01-08 12:01
/chukwa/logs/201308115635034_domU12313907193B_488afeb413c19ee05bf7ff2.done
-rw-r--r--   1 ec2-user supergroup     488067 2013-01-08 12:06
/chukwa/logs/201308120135034_domU12313907193B_488afeb413c19ee05bf7fef.done
-rw-r--r--   1 ec2-user supergroup     479961 2013-01-08 12:11
/chukwa/logs/201308120635034_domU12313907193B_488afeb413c19ee05bf7fec.done
-rw-r--r--   1 ec2-user supergroup     487976 2013-01-08 12:16
/chukwa/logs/201308121135034_domU12313907193B_488afeb413c19ee05bf7fe9.done
-rw-r--r--   1 ec2-user supergroup     479863 2013-01-08 12:21
/chukwa/logs/201308121635034_domU12313907193B_488afeb413c19ee05bf7fe6.done
-rw-r--r--   1 ec2-user supergroup     649168 2013-01-08 12:42
/chukwa/logs/201308123755390_domU12313907193B_5ab1511313c1a2ac24f8000.done
-rw-r--r--   1 ec2-user supergroup     483972 2013-01-08 12:47
/chukwa/logs/201308124255466_domU12313907193B_5ab1511313c1a2ac24f7ffc.done
-rw-r--r--   1 ec2-user supergroup     483417 2013-01-08 12:52
/chukwa/logs/201308124755466_domU12313907193B_5ab1511313c1a2ac24f7ff8.done
-rw-r--r--   1 ec2-user supergroup     576711 2013-01-08 13:06
/chukwa/logs/201308130134839_domU12313907193B_2abeddbf13c1a406b0b8000.done
-rw-r--r--   1 ec2-user supergroup     483877 2013-01-08 13:11
/chukwa/logs/201308130634912_domU12313907193B_2abeddbf13c1a406b0b7ffc.done
-rw-r--r--   1 ec2-user supergroup     484352 2013-01-08 13:16
/chukwa/logs/201308131134912_domU12313907193B_2abeddbf13c1a406b0b7ff7.done
-rw-r--r--   1 ec2-user supergroup     486158 2013-01-08 13:21
/chukwa/logs/201308131634912_domU12313907193B_2abeddbf13c1a406b0b7ff2.done
-rw-r--r--   1 ec2-user supergroup     483925 2013-01-08 13:26
/chukwa/logs/201308132134913_domU12313907193B_2abeddbf13c1a406b0b7fed.done
-rw-r--r--   1 ec2-user supergroup     629788 2013-01-08 13:37
/chukwa/logs/201308133253308_domU12313907193B_7382697b13c1a5d14db8000.done
-rw-r--r--   1 ec2-user supergroup     485716 2013-01-08 13:42
/chukwa/logs/201308133753394_domU12313907193B_7382697b13c1a5d14db7ffb.done
-rw-r--r--   1 ec2-user supergroup     486475 2013-01-08 13:47
/chukwa/logs/201308134253394_domU12313907193B_7382697b13c1a5d14db7ff5.done
-rw-r--r--   1 ec2-user supergroup     481330 2013-01-08 13:52
/chukwa/logs/201308134753394_domU12313907193B_7382697b13c1a5d14db7fef.done
-rw-r--r--   1 ec2-user supergroup     488138 2013-01-08 13:57
/chukwa/logs/201308135253394_domU12313907193B_7382697b13c1a5d14db7fe9.done
-rw-r--r--   1 ec2-user supergroup     484294 2013-01-08 14:02
/chukwa/logs/201308135753394_domU12313907193B_7382697b13c1a5d14db7fe3.done
-rw-r--r--   1 ec2-user supergroup     480689 2013-01-08 14:07
/chukwa/logs/201308140253394_domU12313907193B_7382697b13c1a5d14db7fdd.done
-rw-r--r--   1 ec2-user supergroup     488018 2013-01-08 14:12
/chukwa/logs/201308140753395_domU12313907193B_7382697b13c1a5d14db7fd7.done
-rw-r--r--   1 ec2-user supergroup     287583 2013-01-08 14:23
/chukwa/logs/201308141253395_domU12313907193B_7382697b13c1a5d14db7fd1.done
-rw-r--r--   1 ec2-user supergroup     527781 2013-01-08 14:27
/chukwa/logs/201308142254785_domU12313907193B_3065fa8513c1a8ae1598000.done
-rw-r--r--   1 ec2-user supergroup     483844 2013-01-08 14:32
/chukwa/logs/201308142754897_domU12313907193B_3065fa8513c1a8ae1597ff9.done
-rw-r--r--   1 ec2-user supergroup     483447 2013-01-08 14:37
/chukwa/logs/201308143254897_domU12313907193B_3065fa8513c1a8ae1597ff3.done
-rw-r--r--   1 ec2-user supergroup     483915 2013-01-08 14:42
/chukwa/logs/201308143754897_domU12313907193B_3065fa8513c1a8ae1597fed.done
-rw-r--r--   1 ec2-user supergroup     483902 2013-01-08 14:47
/chukwa/logs/201308144254897_domU12313907193B_3065fa8513c1a8ae1597fe7.done
-rw-r--r--   1 ec2-user supergroup     484187 2013-01-08 14:52
/chukwa/logs/201308144754897_domU12313907193B_3065fa8513c1a8ae1597fe1.done
-rw-r--r--   1 ec2-user supergroup     295291 2013-01-08 17:04
/chukwa/logs/201308145254897_domU12313907193B_3065fa8513c1a8ae1597fdb.done
-rw-r--r--   1 ec2-user supergroup     524107 2013-01-08 17:08
/chukwa/logs/201308170310783_domU123139078691_50774b0013c1b1d9bd48000.done
-rw-r--r--   1 ec2-user supergroup     493338 2013-01-08 17:13
/chukwa/logs/201308170810879_domU123139078691_50774b0013c1b1d9bd47ff7.done
-rw-r--r--   1 ec2-user supergroup     485752 2013-01-08 17:18
/chukwa/logs/201308171310880_domU123139078691_50774b0013c1b1d9bd47ff1.done
-rw-r--r--   1 ec2-user supergroup     566482 2013-01-08 17:26
/chukwa/logs/201308172154776_domU123139078691_2f8337f413c1b2ec27b8000.done
-rw-r--r--   1 ec2-user supergroup     483669 2013-01-08 17:31
/chukwa/logs/201308172654963_domU123139078691_2f8337f413c1b2ec27b7ffa.done
-rw-r--r--   1 ec2-user supergroup     483893 2013-01-08 17:36
/chukwa/logs/201308173154963_domU123139078691_2f8337f413c1b2ec27b7ff3.done
-rw-r--r--   1 ec2-user supergroup     483996 2013-01-08 17:41
/chukwa/logs/201308173654964_domU123139078691_2f8337f413c1b2ec27b7fec.done
-rw-r--r--   1 ec2-user supergroup     483918 2013-01-08 17:46
/chukwa/logs/201308174154964_domU123139078691_2f8337f413c1b2ec27b7fe5.done
-rw-r--r--   1 ec2-user supergroup     484337 2013-01-08 17:51
/chukwa/logs/201308174654964_domU123139078691_2f8337f413c1b2ec27b7fde.done
-rw-r--r--   1 ec2-user supergroup     103077 2013-01-09 07:08
/chukwa/logs/201308175154964_domU123139078691_2f8337f413c1b2ec27b7fd7.done
-rw-r--r--   1 ec2-user supergroup     539279 2013-01-09 07:13
/chukwa/logs/201309070837366_ip10461114_5f07286f13c1e23a2908000.done
-rw-r--r--   1 ec2-user supergroup     450569 2013-01-09 07:18
/chukwa/logs/201309071337486_ip10461114_5f07286f13c1e23a2907ff9.done
-rw-r--r--   1 ec2-user supergroup     450802 2013-01-09 07:23
/chukwa/logs/201309071837486_ip10461114_5f07286f13c1e23a2907ff2.done
-rw-r--r--   1 ec2-user supergroup     450749 2013-01-09 07:28
/chukwa/logs/201309072337486_ip10461114_5f07286f13c1e23a2907feb.done
-rw-r--r--   1 ec2-user supergroup     545135 2013-01-09 08:19
/chukwa/logs/201309081415268_ip10461114_b00255013c1e5fb8f78000.done
-rw-r--r--   1 ec2-user supergroup     457343 2013-01-09 08:24
/chukwa/logs/201309081915384_ip10461114_b00255013c1e5fb8f77ff8.done
-rw-r--r--   1 ec2-user supergroup     452951 2013-01-09 08:29
/chukwa/logs/201309082415384_ip10461114_b00255013c1e5fb8f77ff0.done
-rw-r--r--   1 ec2-user supergroup     534336 2013-01-09 08:40
/chukwa/logs/201309083503153_ip10461114_4c587e7413c1e72c3ba8000.done
-rw-r--r--   1 ec2-user supergroup     454511 2013-01-09 08:45
/chukwa/logs/201309084003370_ip10461114_4c587e7413c1e72c3ba7ff8.done
-rw-r--r--   1 ec2-user supergroup     451026 2013-01-09 08:50
/chukwa/logs/201309084503370_ip10461114_4c587e7413c1e72c3ba7fef.done
-rw-r--r--   1 ec2-user supergroup     447653 2013-01-09 08:55
/chukwa/logs/201309085003370_ip10461114_4c587e7413c1e72c3ba7fe6.done
-rw-r--r--   1 ec2-user supergroup     451091 2013-01-09 09:00
/chukwa/logs/201309085503370_ip10461114_4c587e7413c1e72c3ba7fdd.done
-rw-r--r--   1 ec2-user supergroup     449429 2013-01-09 09:05
/chukwa/logs/201309090003371_ip10461114_4c587e7413c1e72c3ba7fd4.done
-rw-r--r--   1 ec2-user supergroup     456627 2013-01-09 09:10
/chukwa/logs/201309090503371_ip10461114_4c587e7413c1e72c3ba7fcb.done
-rw-r--r--   1 ec2-user supergroup     451078 2013-01-09 09:15
/chukwa/logs/201309091003371_ip10461114_4c587e7413c1e72c3ba7fc2.done
-rw-r--r--   1 ec2-user supergroup     447648 2013-01-09 09:20
/chukwa/logs/201309091503371_ip10461114_4c587e7413c1e72c3ba7fb9.done
-rw-r--r--   1 ec2-user supergroup     451155 2013-01-09 09:25
/chukwa/logs/201309092003371_ip10461114_4c587e7413c1e72c3ba7fb0.done
-rw-r--r--   1 ec2-user supergroup     455010 2013-01-09 09:30
/chukwa/logs/201309092503371_ip10461114_4c587e7413c1e72c3ba7fa7.done
-rw-r--r--   1 ec2-user supergroup     451450 2013-01-09 09:35
/chukwa/logs/201309093003372_ip10461114_4c587e7413c1e72c3ba7f9e.done
-rw-r--r--   1 ec2-user supergroup     447741 2013-01-09 09:40
/chukwa/logs/201309093503372_ip10461114_4c587e7413c1e72c3ba7f95.done
-rw-r--r--   1 ec2-user supergroup     451407 2013-01-09 09:45
/chukwa/logs/201309094003372_ip10461114_4c587e7413c1e72c3ba7f8c.done
-rw-r--r--   1 ec2-user supergroup     451304 2013-01-09 09:50
/chukwa/logs/201309094503373_ip10461114_4c587e7413c1e72c3ba7f83.done
-rw-r--r--   1 ec2-user supergroup     454591 2013-01-09 09:55
/chukwa/logs/201309095003374_ip10461114_4c587e7413c1e72c3ba7f7a.done
-rw-r--r--   1 ec2-user supergroup     451044 2013-01-09 10:00
/chukwa/logs/201309095503374_ip10461114_4c587e7413c1e72c3ba7f71.done
-rw-r--r--   1 ec2-user supergroup     448113 2013-01-09 10:05
/chukwa/logs/201309100003374_ip10461114_4c587e7413c1e72c3ba7f68.done
-rw-r--r--   1 ec2-user supergroup     451071 2013-01-09 10:10
/chukwa/logs/201309100503374_ip10461114_4c587e7413c1e72c3ba7f5f.done
-rw-r--r--   1 ec2-user supergroup     451235 2013-01-09 10:15
/chukwa/logs/201309101003374_ip10461114_4c587e7413c1e72c3ba7f56.done
-rw-r--r--   1 ec2-user supergroup     454562 2013-01-09 10:20
/chukwa/logs/201309101503374_ip10461114_4c587e7413c1e72c3ba7f4d.done
-rw-r--r--   1 ec2-user supergroup     451237 2013-01-09 10:25
/chukwa/logs/201309102003375_ip10461114_4c587e7413c1e72c3ba7f44.done
-rw-r--r--   1 ec2-user supergroup     451241 2013-01-09 10:30
/chukwa/logs/201309102503375_ip10461114_4c587e7413c1e72c3ba7f3b.done
-rw-r--r--   1 ec2-user supergroup     447653 2013-01-09 10:35
/chukwa/logs/201309103003375_ip10461114_4c587e7413c1e72c3ba7f32.done
-rw-r--r--   1 ec2-user supergroup     451351 2013-01-09 10:40
/chukwa/logs/201309103503375_ip10461114_4c587e7413c1e72c3ba7f29.done
-rw-r--r--   1 ec2-user supergroup     451706 2013-01-09 10:45
/chukwa/logs/201309104003376_ip10461114_4c587e7413c1e72c3ba7f20.done
-rw-r--r--   1 ec2-user supergroup     450777 2013-01-09 10:50
/chukwa/logs/201309104503375_ip10461114_4c587e7413c1e72c3ba7f17.done


Does it mean, Chukwa collected the logs.

Regards,
Shouvanik


-- 
Thanks,
*Shouvanik*

Mime
View raw message