hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject ChecksumException
Date Thu, 11 Feb 2010 17:26:43 GMT
Hi,
I see the following in our log:
org.apache.hadoop.fs.ChecksumException: Checksum error:
file:/domain_crawler/crawlworker/crawl/777_pv_10_4_20100128_122735_353.done.tsk/segments/20100131163704/parse_data/part-00000/data
at 42990592
        at
org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.readChunk(ChecksumFileSystem.java:211)
        at
org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:238)
        at org.apache.hadoop.fs.FSInputChecker.fill(FSInputChecker.java:177)
        at
org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:194)
        at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:159)
        at java.io.DataInputStream.readFully(DataInputStream.java:178)
        at
org.apache.hadoop.io.DataOutputBuffer$Buffer.write(DataOutputBuffer.java:64)
        at
org.apache.hadoop.io.DataOutputBuffer.write(DataOutputBuffer.java:102)
        at
org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1933)
        at
org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1833)
        at
org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1879)
        at
com.rialto.webmap.common.SeqFileWriter.writeSeqFile(SeqFileWriter.java:189)

HFWriter uses  Hadoop version 0.18.3

Please share your experience solving this error.

Thanks

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message