chukwa-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shouvanik Haldar <shouvanik.hal...@gmail.com>
Subject Re: How to check whether chukwa collecting logs
Date Thu, 10 Jan 2013 07:11:03 GMT
Can you please help me find out why the following log is generated by the
collector

2013-01-10 07:09:23,864 INFO Timer-1 root -
stats:ServletCollector,numberHTTPConnection:18,numberchunks:166
2013-01-10 07:09:23,884 WARN LocalToRemoteHdfsMover CopySequenceFile -
Error during .chukwa file recovery
java.io.EOFException
        at java.io.DataInputStream.readFully(DataInputStream.java:180)
        at
org.apache.hadoop.io.DataOutputBuffer$Buffer.write(DataOutputBuffer.java:63)
        at
org.apache.hadoop.io.DataOutputBuffer.write(DataOutputBuffer.java:101)
        at
org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1988)
        at
org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1888)
        at
org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1934)
        at
org.apache.hadoop.chukwa.util.CopySequenceFile.createValidSequenceFile(CopySequenceFile.java:78)
        at
org.apache.hadoop.chukwa.datacollection.writer.localfs.LocalToRemoteHdfsMover.cleanup(LocalToRemoteHdfsMover.java:186)
        at
org.apache.hadoop.chukwa.datacollection.writer.localfs.LocalToRemoteHdfsMover.run(LocalToRemoteHdfsMover.java:216)
2013-01-10 07:09:23,891 INFO LocalToRemoteHdfsMover LocalToRemoteHdfsMover
- Deleted .recover file,
/tmp/chukwa/dataSink/201308112441296_domU12313907193B_216836a313c19d565ee7ff8.recover
2013-01-10 07:09:23,891 INFO LocalToRemoteHdfsMover LocalToRemoteHdfsMover
- Copying .chukwa file to valid sink file before moving,
/tmp/chukwa/dataSink/201310063103378_domU1231390B80BA_1082291f13c22c778057f42.chukwa
2013-01-10 07:09:23,899 WARN LocalToRemoteHdfsMover FSInputChecker -
Problem opening checksum file:
/tmp/chukwa/dataSink/201310063103378_domU1231390B80BA_1082291f13c22c778057f42.chukwa.
Ignoring exception: java.io.EOFException
        at java.io.DataInputStream.readFully(DataInputStream.java:180)
        at java.io.DataInputStream.readFully(DataInputStream.java:152)
        at
org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:134)
        at
org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:283)
        at
org.apache.hadoop.io.SequenceFile$Reader.openFile(SequenceFile.java:1495)
        at
org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1482)
        at
org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1475)
        at
org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1470)
        at
org.apache.hadoop.chukwa.util.CopySequenceFile.createValidSequenceFile(CopySequenceFile.java:73)
        at
org.apache.hadoop.chukwa.datacollection.writer.localfs.LocalToRemoteHdfsMover.cleanup(LocalToRemoteHdfsMover.java:186)
        at
org.apache.hadoop.chukwa.datacollection.writer.localfs.LocalToRemoteHdfsMover.run(LocalToRemoteHdfsMover.java:216)

2013-01-10 07:09:23,931 WARN LocalToRemoteHdfsMover CopySequenceFile -
Error during .chukwa file recovery
java.io.EOFException
        at java.io.DataInputStream.readFully(DataInputStream.java:180)
        at
org.apache.hadoop.io.DataOutputBuffer$Buffer.write(DataOutputBuffer.java:63)
        at
org.apache.hadoop.io.DataOutputBuffer.write(DataOutputBuffer.java:101)
        at
org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1988)
        at
org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1888)
        at
org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1934)
        at
org.apache.hadoop.chukwa.util.CopySequenceFile.createValidSequenceFile(CopySequenceFile.java:78)
        at
org.apache.hadoop.chukwa.datacollection.writer.localfs.LocalToRemoteHdfsMover.cleanup(LocalToRemoteHdfsMover.java:186)
        at
org.apache.hadoop.chukwa.datacollection.writer.localfs.LocalToRemoteHdfsMover.run(LocalToRemoteHdfsMover.java:216)
2013-01-10 07:09:24,348 INFO Timer-3 LocalWriter -
stat:datacollection.writer.local.LocalWriter dataSize=29534 dataRate=984
2013-01-10 07:09:54,349 INFO Timer-3 LocalWriter -
stat:datacollection.writer.local.LocalWriter dataSize=34440 dataRate=1147


Regards,
Shouvanik


On Wed, Jan 9, 2013 at 4:24 PM, Shouvanik Haldar <shouvanik.haldar@gmail.com
> wrote:

> I am trying to find out a check point , so that I can say that Chukwa is
> running and collecting logs
>
>
> I configured the /chukwa/etc/chukwa/initial_adaptors
> with following
>
> add FileTailer
> /home/ec2-user/hadoop-1.0.4/logs/hadoop-ec2-user-datanode-domU-12-31-39-07-19-3B.log
> 0
>
> .
> then I started bin/chukwa agent.
>
> Started the hadoop daemons.
>
> then, bin/chukwa collector.
>
> Next, I check the site
> "http://ec2-204-236-192-203.compute-1.amazonaws.com:8080/chukwa?ping=true"
>
> with following statistics
>
> Date:1357728723103
> Now:1357728765604
> numberHTTPConnection in time window:17
> numberchunks in time window:119
> lifetimechunks:23029
>
>
> and when I check hdfs I see
>
> -rw-r--r--   1 ec2-user supergroup     657818 2013-01-08 11:09 /chukwa/logs/201308110441175_domU12313907193B_216836a313c19d565ee8000.done
> -rw-r--r--   1 ec2-user supergroup     483520 2013-01-08 11:14 /chukwa/logs/201308110941296_domU12313907193B_216836a313c19d565ee7ffe.done
>
> -rw-r--r--   1 ec2-user supergroup     483155 2013-01-08 11:19 /chukwa/logs/201308111441296_domU12313907193B_216836a313c19d565ee7ffc.done
> -rw-r--r--   1 ec2-user supergroup     463708 2013-01-08 11:24 /chukwa/logs/201308111941296_domU12313907193B_216836a313c19d565ee7ffa.done
>
> -rw-r--r--   1 ec2-user supergroup     386337 2013-01-08 11:36 /chukwa/logs/201308113134948_domU12313907193B_488afeb413c19ee05bf8000.done
> -rw-r--r--   1 ec2-user supergroup     479656 2013-01-08 11:41 /chukwa/logs/201308113635034_domU12313907193B_488afeb413c19ee05bf7ffe.done
>
> -rw-r--r--   1 ec2-user supergroup     487748 2013-01-08 11:46 /chukwa/logs/201308114135034_domU12313907193B_488afeb413c19ee05bf7ffb.done
> -rw-r--r--   1 ec2-user supergroup     479696 2013-01-08 11:51 /chukwa/logs/201308114635034_domU12313907193B_488afeb413c19ee05bf7ff8.done
>
> -rw-r--r--   1 ec2-user supergroup     488030 2013-01-08 11:56 /chukwa/logs/201308115135034_domU12313907193B_488afeb413c19ee05bf7ff5.done
> -rw-r--r--   1 ec2-user supergroup     479754 2013-01-08 12:01 /chukwa/logs/201308115635034_domU12313907193B_488afeb413c19ee05bf7ff2.done
>
> -rw-r--r--   1 ec2-user supergroup     488067 2013-01-08 12:06 /chukwa/logs/201308120135034_domU12313907193B_488afeb413c19ee05bf7fef.done
> -rw-r--r--   1 ec2-user supergroup     479961 2013-01-08 12:11 /chukwa/logs/201308120635034_domU12313907193B_488afeb413c19ee05bf7fec.done
>
> -rw-r--r--   1 ec2-user supergroup     487976 2013-01-08 12:16 /chukwa/logs/201308121135034_domU12313907193B_488afeb413c19ee05bf7fe9.done
> -rw-r--r--   1 ec2-user supergroup     479863 2013-01-08 12:21 /chukwa/logs/201308121635034_domU12313907193B_488afeb413c19ee05bf7fe6.done
>
> -rw-r--r--   1 ec2-user supergroup     649168 2013-01-08 12:42 /chukwa/logs/201308123755390_domU12313907193B_5ab1511313c1a2ac24f8000.done
> -rw-r--r--   1 ec2-user supergroup     483972 2013-01-08 12:47 /chukwa/logs/201308124255466_domU12313907193B_5ab1511313c1a2ac24f7ffc.done
>
> -rw-r--r--   1 ec2-user supergroup     483417 2013-01-08 12:52 /chukwa/logs/201308124755466_domU12313907193B_5ab1511313c1a2ac24f7ff8.done
> -rw-r--r--   1 ec2-user supergroup     576711 2013-01-08 13:06 /chukwa/logs/201308130134839_domU12313907193B_2abeddbf13c1a406b0b8000.done
>
> -rw-r--r--   1 ec2-user supergroup     483877 2013-01-08 13:11 /chukwa/logs/201308130634912_domU12313907193B_2abeddbf13c1a406b0b7ffc.done
> -rw-r--r--   1 ec2-user supergroup     484352 2013-01-08 13:16 /chukwa/logs/201308131134912_domU12313907193B_2abeddbf13c1a406b0b7ff7.done
>
> -rw-r--r--   1 ec2-user supergroup     486158 2013-01-08 13:21 /chukwa/logs/201308131634912_domU12313907193B_2abeddbf13c1a406b0b7ff2.done
> -rw-r--r--   1 ec2-user supergroup     483925 2013-01-08 13:26 /chukwa/logs/201308132134913_domU12313907193B_2abeddbf13c1a406b0b7fed.done
>
> -rw-r--r--   1 ec2-user supergroup     629788 2013-01-08 13:37 /chukwa/logs/201308133253308_domU12313907193B_7382697b13c1a5d14db8000.done
> -rw-r--r--   1 ec2-user supergroup     485716 2013-01-08 13:42 /chukwa/logs/201308133753394_domU12313907193B_7382697b13c1a5d14db7ffb.done
>
> -rw-r--r--   1 ec2-user supergroup     486475 2013-01-08 13:47 /chukwa/logs/201308134253394_domU12313907193B_7382697b13c1a5d14db7ff5.done
> -rw-r--r--   1 ec2-user supergroup     481330 2013-01-08 13:52 /chukwa/logs/201308134753394_domU12313907193B_7382697b13c1a5d14db7fef.done
>
> -rw-r--r--   1 ec2-user supergroup     488138 2013-01-08 13:57 /chukwa/logs/201308135253394_domU12313907193B_7382697b13c1a5d14db7fe9.done
> -rw-r--r--   1 ec2-user supergroup     484294 2013-01-08 14:02 /chukwa/logs/201308135753394_domU12313907193B_7382697b13c1a5d14db7fe3.done
>
> -rw-r--r--   1 ec2-user supergroup     480689 2013-01-08 14:07 /chukwa/logs/201308140253394_domU12313907193B_7382697b13c1a5d14db7fdd.done
> -rw-r--r--   1 ec2-user supergroup     488018 2013-01-08 14:12 /chukwa/logs/201308140753395_domU12313907193B_7382697b13c1a5d14db7fd7.done
>
> -rw-r--r--   1 ec2-user supergroup     287583 2013-01-08 14:23 /chukwa/logs/201308141253395_domU12313907193B_7382697b13c1a5d14db7fd1.done
> -rw-r--r--   1 ec2-user supergroup     527781 2013-01-08 14:27 /chukwa/logs/201308142254785_domU12313907193B_3065fa8513c1a8ae1598000.done
>
> -rw-r--r--   1 ec2-user supergroup     483844 2013-01-08 14:32 /chukwa/logs/201308142754897_domU12313907193B_3065fa8513c1a8ae1597ff9.done
> -rw-r--r--   1 ec2-user supergroup     483447 2013-01-08 14:37 /chukwa/logs/201308143254897_domU12313907193B_3065fa8513c1a8ae1597ff3.done
>
> -rw-r--r--   1 ec2-user supergroup     483915 2013-01-08 14:42 /chukwa/logs/201308143754897_domU12313907193B_3065fa8513c1a8ae1597fed.done
> -rw-r--r--   1 ec2-user supergroup     483902 2013-01-08 14:47 /chukwa/logs/201308144254897_domU12313907193B_3065fa8513c1a8ae1597fe7.done
>
> -rw-r--r--   1 ec2-user supergroup     484187 2013-01-08 14:52 /chukwa/logs/201308144754897_domU12313907193B_3065fa8513c1a8ae1597fe1.done
> -rw-r--r--   1 ec2-user supergroup     295291 2013-01-08 17:04 /chukwa/logs/201308145254897_domU12313907193B_3065fa8513c1a8ae1597fdb.done
>
> -rw-r--r--   1 ec2-user supergroup     524107 2013-01-08 17:08 /chukwa/logs/201308170310783_domU123139078691_50774b0013c1b1d9bd48000.done
> -rw-r--r--   1 ec2-user supergroup     493338 2013-01-08 17:13 /chukwa/logs/201308170810879_domU123139078691_50774b0013c1b1d9bd47ff7.done
>
> -rw-r--r--   1 ec2-user supergroup     485752 2013-01-08 17:18 /chukwa/logs/201308171310880_domU123139078691_50774b0013c1b1d9bd47ff1.done
> -rw-r--r--   1 ec2-user supergroup     566482 2013-01-08 17:26 /chukwa/logs/201308172154776_domU123139078691_2f8337f413c1b2ec27b8000.done
>
> -rw-r--r--   1 ec2-user supergroup     483669 2013-01-08 17:31 /chukwa/logs/201308172654963_domU123139078691_2f8337f413c1b2ec27b7ffa.done
> -rw-r--r--   1 ec2-user supergroup     483893 2013-01-08 17:36 /chukwa/logs/201308173154963_domU123139078691_2f8337f413c1b2ec27b7ff3.done
>
> -rw-r--r--   1 ec2-user supergroup     483996 2013-01-08 17:41 /chukwa/logs/201308173654964_domU123139078691_2f8337f413c1b2ec27b7fec.done
> -rw-r--r--   1 ec2-user supergroup     483918 2013-01-08 17:46 /chukwa/logs/201308174154964_domU123139078691_2f8337f413c1b2ec27b7fe5.done
>
> -rw-r--r--   1 ec2-user supergroup     484337 2013-01-08 17:51 /chukwa/logs/201308174654964_domU123139078691_2f8337f413c1b2ec27b7fde.done
> -rw-r--r--   1 ec2-user supergroup     103077 2013-01-09 07:08 /chukwa/logs/201308175154964_domU123139078691_2f8337f413c1b2ec27b7fd7.done
>
> -rw-r--r--   1 ec2-user supergroup     539279 2013-01-09 07:13 /chukwa/logs/201309070837366_ip10461114_5f07286f13c1e23a2908000.done
> -rw-r--r--   1 ec2-user supergroup     450569 2013-01-09 07:18 /chukwa/logs/201309071337486_ip10461114_5f07286f13c1e23a2907ff9.done
>
> -rw-r--r--   1 ec2-user supergroup     450802 2013-01-09 07:23 /chukwa/logs/201309071837486_ip10461114_5f07286f13c1e23a2907ff2.done
> -rw-r--r--   1 ec2-user supergroup     450749 2013-01-09 07:28 /chukwa/logs/201309072337486_ip10461114_5f07286f13c1e23a2907feb.done
>
> -rw-r--r--   1 ec2-user supergroup     545135 2013-01-09 08:19 /chukwa/logs/201309081415268_ip10461114_b00255013c1e5fb8f78000.done
> -rw-r--r--   1 ec2-user supergroup     457343 2013-01-09 08:24 /chukwa/logs/201309081915384_ip10461114_b00255013c1e5fb8f77ff8.done
>
> -rw-r--r--   1 ec2-user supergroup     452951 2013-01-09 08:29 /chukwa/logs/201309082415384_ip10461114_b00255013c1e5fb8f77ff0.done
> -rw-r--r--   1 ec2-user supergroup     534336 2013-01-09 08:40 /chukwa/logs/201309083503153_ip10461114_4c587e7413c1e72c3ba8000.done
>
> -rw-r--r--   1 ec2-user supergroup     454511 2013-01-09 08:45 /chukwa/logs/201309084003370_ip10461114_4c587e7413c1e72c3ba7ff8.done
> -rw-r--r--   1 ec2-user supergroup     451026 2013-01-09 08:50 /chukwa/logs/201309084503370_ip10461114_4c587e7413c1e72c3ba7fef.done
>
> -rw-r--r--   1 ec2-user supergroup     447653 2013-01-09 08:55 /chukwa/logs/201309085003370_ip10461114_4c587e7413c1e72c3ba7fe6.done
> -rw-r--r--   1 ec2-user supergroup     451091 2013-01-09 09:00 /chukwa/logs/201309085503370_ip10461114_4c587e7413c1e72c3ba7fdd.done
>
> -rw-r--r--   1 ec2-user supergroup     449429 2013-01-09 09:05 /chukwa/logs/201309090003371_ip10461114_4c587e7413c1e72c3ba7fd4.done
> -rw-r--r--   1 ec2-user supergroup     456627 2013-01-09 09:10 /chukwa/logs/201309090503371_ip10461114_4c587e7413c1e72c3ba7fcb.done
>
> -rw-r--r--   1 ec2-user supergroup     451078 2013-01-09 09:15 /chukwa/logs/201309091003371_ip10461114_4c587e7413c1e72c3ba7fc2.done
> -rw-r--r--   1 ec2-user supergroup     447648 2013-01-09 09:20 /chukwa/logs/201309091503371_ip10461114_4c587e7413c1e72c3ba7fb9.done
>
> -rw-r--r--   1 ec2-user supergroup     451155 2013-01-09 09:25 /chukwa/logs/201309092003371_ip10461114_4c587e7413c1e72c3ba7fb0.done
> -rw-r--r--   1 ec2-user supergroup     455010 2013-01-09 09:30 /chukwa/logs/201309092503371_ip10461114_4c587e7413c1e72c3ba7fa7.done
>
> -rw-r--r--   1 ec2-user supergroup     451450 2013-01-09 09:35 /chukwa/logs/201309093003372_ip10461114_4c587e7413c1e72c3ba7f9e.done
> -rw-r--r--   1 ec2-user supergroup     447741 2013-01-09 09:40 /chukwa/logs/201309093503372_ip10461114_4c587e7413c1e72c3ba7f95.done
>
> -rw-r--r--   1 ec2-user supergroup     451407 2013-01-09 09:45 /chukwa/logs/201309094003372_ip10461114_4c587e7413c1e72c3ba7f8c.done
> -rw-r--r--   1 ec2-user supergroup     451304 2013-01-09 09:50 /chukwa/logs/201309094503373_ip10461114_4c587e7413c1e72c3ba7f83.done
>
> -rw-r--r--   1 ec2-user supergroup     454591 2013-01-09 09:55 /chukwa/logs/201309095003374_ip10461114_4c587e7413c1e72c3ba7f7a.done
> -rw-r--r--   1 ec2-user supergroup     451044 2013-01-09 10:00 /chukwa/logs/201309095503374_ip10461114_4c587e7413c1e72c3ba7f71.done
>
> -rw-r--r--   1 ec2-user supergroup     448113 2013-01-09 10:05 /chukwa/logs/201309100003374_ip10461114_4c587e7413c1e72c3ba7f68.done
> -rw-r--r--   1 ec2-user supergroup     451071 2013-01-09 10:10 /chukwa/logs/201309100503374_ip10461114_4c587e7413c1e72c3ba7f5f.done
>
> -rw-r--r--   1 ec2-user supergroup     451235 2013-01-09 10:15 /chukwa/logs/201309101003374_ip10461114_4c587e7413c1e72c3ba7f56.done
> -rw-r--r--   1 ec2-user supergroup     454562 2013-01-09 10:20 /chukwa/logs/201309101503374_ip10461114_4c587e7413c1e72c3ba7f4d.done
>
> -rw-r--r--   1 ec2-user supergroup     451237 2013-01-09 10:25 /chukwa/logs/201309102003375_ip10461114_4c587e7413c1e72c3ba7f44.done
> -rw-r--r--   1 ec2-user supergroup     451241 2013-01-09 10:30 /chukwa/logs/201309102503375_ip10461114_4c587e7413c1e72c3ba7f3b.done
>
> -rw-r--r--   1 ec2-user supergroup     447653 2013-01-09 10:35 /chukwa/logs/201309103003375_ip10461114_4c587e7413c1e72c3ba7f32.done
> -rw-r--r--   1 ec2-user supergroup     451351 2013-01-09 10:40 /chukwa/logs/201309103503375_ip10461114_4c587e7413c1e72c3ba7f29.done
>
> -rw-r--r--   1 ec2-user supergroup     451706 2013-01-09 10:45 /chukwa/logs/201309104003376_ip10461114_4c587e7413c1e72c3ba7f20.done
> -rw-r--r--   1 ec2-user supergroup     450777 2013-01-09 10:50 /chukwa/logs/201309104503375_ip10461114_4c587e7413c1e72c3ba7f17.done
>
>
> Does it mean, Chukwa collected the logs.
>
> Regards,
> Shouvanik
>
>
> --
> Thanks,
> *Shouvanik*
>



-- 
Thanks,
*Shouvanik*

Mime
View raw message