hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "mawanqiang (JIRA)" <j...@apache.org>
Subject [jira] Created: (HADOOP-6091) Checksum Error
Date Fri, 19 Jun 2009 07:29:08 GMT
Checksum Error
--------------

                 Key: HADOOP-6091
                 URL: https://issues.apache.org/jira/browse/HADOOP-6091
             Project: Hadoop Core
          Issue Type: Bug
    Affects Versions: 0.19.1
         Environment: linux  ubuntu8.0.4  64bit   
10datanode  4G of memory per node  

            Reporter: mawanqiang


Approximately 1 million data used to create index when nutch1.0 error.
The error is:
java.lang.RuntimeException: problem advancing post rec#6758513
        at org.apache.hadoop.mapred.Task$ValuesIterator.next(Task.java:883)
        at org.apache.hadoop.mapred.ReduceTask$ReduceValuesIterator.moveToNext(ReduceTask.java:237)
        at org.apache.hadoop.mapred.ReduceTask$ReduceValuesIterator.next(ReduceTask.java:233)
        at org.apache.nutch.indexer.IndexerMapReduce.reduce(IndexerMapReduce.java:79)
        at org.apache.nutch.indexer.IndexerMapReduce.reduce(IndexerMapReduce.java:50)
        at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:436)
        at org.apache.hadoop.mapred.Child.main(Child.java:158)
Caused by: org.apache.hadoop.fs.ChecksumException: Checksum Error
        at org.apache.hadoop.mapred.IFileInputStream.doRead(IFileInputStream.java:153)
        at org.apache.hadoop.mapred.IFileInputStream.read(IFileInputStream.java:90)
        at org.apache.hadoop.mapred.IFile$Reader.readData(IFile.java:301)
        at org.apache.hadoop.mapred.IFile$Reader.rejigData(IFile.java:331)
        at org.apache.hadoop.mapred.IFile$Reader.readNextBlock(IFile.java:315)
        at org.apache.hadoop.mapred.IFile$Reader.next(IFile.java:377)
        at org.apache.hadoop.mapred.Merger$Segment.next(Merger.java:174)
        at org.apache.hadoop.mapred.Merger$MergeQueue.adjustPriorityQueue(Merger.java:277)
        at org.apache.hadoop.mapred.Merger$MergeQueue.next(Merger.java:297)
        at org.apache.hadoop.mapred.Task$ValuesIterator.readNextKey(Task.java:922)
        at org.apache.hadoop.mapred.Task$ValuesIterator.next(Task.java:881)
        ... 6 more

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message