hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject checksum error
Date Mon, 24 May 2010 16:16:15 GMT
Hi,
We currently use hadoop 0.20.1

I found the following trace in our log:

 2010-05-17 16:19:55,389 INFO  [FSInputChecker] Found checksum error: b[0,
512]=53455106246f72672e6170616368652e6861646f6f702e696f2e426f6f6c65616e5772697461626c652b636f6d2e6361727269657269712e6d326d2e636f6d6d6f6e732e4f746155706c6f61645772697461626c6501002a6f72672e6170616368652e6861646f6f702e696f2e636f6d70726573732e44656661756c74436f64656300000000aa555ae318451e8e76c0230f56f74d1f000014660000000101789cdddb09785355da00e0ef26699bae49ba2f94de2ed436857ab3364194b414294a355296e29aa449079025968ae82f78cb22051c088202e2125040d9acb88bc36410441d9dbf50dc1e06c57143749c8e0e0c7bff73ee92a6e9a7f23f33cef33f7f1f0839eff9ee3df77ef7dcedeb03130facc763b437197d76abd96237983c6e8399731b3d564395d9e2b6797c6e60b9268fdb5d55d56c31180c6693c56bb21bc9327683d5e036dadd162fb056afc76ae17cd62a8bd1e7717b0d3e6f95d5e2357b4c55be260fd76406d6e76b32992c9e2aafdb62b659bccd9cdb63b134f92c4d4d1e5b95d5e401d66430d8dc06b7d76e68f61a9acd469bdb6a37703ea3c9e33556594c1c19a5b9c964f671568bc96b34b83d55551eb2850693db6d767306b3d10a6c1367367bb9660f89b47a7d469fd96771db7d06aec9cbd9c9c69351ec5683917ce38c16bbcd62b17bcc4dcd96263347f6c4d2ecf536937d31588c16a3bdb9
org.apache.hadoop.fs.ChecksumException: Checksum error:
file:/incoming_ub/ciq_dca_uploads/2010/03/26/100813/17_00.ub at 0
        at
org.apache.hadoop.fs.FSInputChecker.verifySum(FSInputChecker.java:277)
        at
org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:241)
        at org.apache.hadoop.fs.FSInputChecker.fill(FSInputChecker.java:176)
        at
org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:193)
        at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:158)
        at java.io.DataInputStream.readFully(DataInputStream.java:178)
        at java.io.DataInputStream.readFully(DataInputStream.java:152)
        at
org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1450)
        at
org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1428)
        at
org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1417)
        at
org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1412)
        at
org.apache.hadoop.mapred.SequenceFileRecordReader.<init>(SequenceFileRecordReader.java:43)
        at
com.ciq.m2m.platform.util.hadoop.ReportingSequenceFileRecordReader.<init>(ReportingSequenceFileRecordReader.java:54)
        at sun.reflect.GeneratedConstructorAccessor39.newInstance(Unknown
Source)
        at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
        at
com.ciq.m2m.platform.util.hadoop.MultiFileRecordReader.assureReader(MultiFileRecordReader.java:128)
        at
com.ciq.m2m.platform.util.hadoop.MultiFileRecordReader.readNextFromAnyReader(MultiFileRecordReader.java:146)
        at
com.ciq.m2m.platform.util.hadoop.MultiFileRecordReader.readNextFromAnyReader(MultiFileRecordReader.java:181)
        at
com.ciq.m2m.platform.util.hadoop.MultiFileRecordReader.next(MultiFileRecordReader.java:122)
        at
com.ciq.m2m.platform.util.hadoop.MultiFileRecordReader.next(MultiFileRecordReader.java:20)
        at
com.ciq.m2m.platform.mmp2.input.BaseOtaUploadRecordReader.readNextUpload(BaseOtaUploadRecordReader.java:125)
        at
com.ciq.m2m.platform.mmp2.input.OtaUploadToBinaryPackageRecordReader.next(OtaUploadToBinaryPackageRecordReader.java:176)
        at
com.ciq.m2m.platform.mmp2.input.OtaUploadToBinaryPackageRecordReader.next(OtaUploadToBinaryPackageRecordReader.java:56)
        at
org.apache.hadoop.mapred.MapTask$TrackedRecordReader.moveToNext(MapTask.java:192)
        at
org.apache.hadoop.mapred.MapTask$TrackedRecordReader.next(MapTask.java:176)
        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:48)
        at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
        at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:176)

If you saw similar trace before, please share your comment.

Thanks

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message