hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Grant Ingersoll <gsing...@apache.org>
Subject "Merge of the inmemory files threw an exception" and diffs between 0.17.2 and 0.18.1
Date Tue, 28 Oct 2008 02:05:12 GMT
Hi,

Over in Mahout (lucene.a.o/mahout), we are seeing an oddity with some  
of our clustering code and Hadoop 0.18.1.  The thread in context is  
at:  http://mahout.markmail.org/message/vcyvlz2met7fnthr

The problem seems to occur when going from 0.17.2 to 0.18.1.  In the  
user logs, we are seeing the following exception:
2008-10-27 21:18:37,014 INFO org.apache.hadoop.mapred.Merger: Down to  
the last merge-pass, with 2 segments left of total size: 5011 bytes
2008-10-27 21:18:37,033 WARN org.apache.hadoop.mapred.ReduceTask:  
attempt_200810272112_0011_r_000000_0 Merge of the inmemory files threw  
an exception: java.io.IOException: Intermedate merge failed
         at org.apache.hadoop.mapred.ReduceTask$ReduceCopier 
$InMemFSMergeThread.doInMemMerge(ReduceTask.java:2147)
         at org.apache.hadoop.mapred.ReduceTask$ReduceCopier 
$InMemFSMergeThread.run(ReduceTask.java:2078)
Caused by: java.lang.NumberFormatException: For input string: "["
         at  
sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:1224)
         at java.lang.Double.parseDouble(Double.java:510)
         at  
org.apache.mahout.matrix.DenseVector.decodeFormat(DenseVector.java:60)
         at  
org 
.apache.mahout.matrix.AbstractVector.decodeVector(AbstractVector.java: 
256)
         at  
org 
.apache 
.mahout.clustering.kmeans.KMeansCombiner.reduce(KMeansCombiner.java:38)
         at  
org 
.apache 
.mahout.clustering.kmeans.KMeansCombiner.reduce(KMeansCombiner.java:31)
         at org.apache.hadoop.mapred.ReduceTask 
$ReduceCopier.combineAndSpill(ReduceTask.java:2174)
         at org.apache.hadoop.mapred.ReduceTask$ReduceCopier.access 
$3100(ReduceTask.java:341)
         at org.apache.hadoop.mapred.ReduceTask$ReduceCopier 
$InMemFSMergeThread.doInMemMerge(ReduceTask.java:2134)

And in the main output log (from running  bin/hadoop jar  mahout/ 
examples/build/apache-mahout-examples-0.1-dev.job  
org.apache.mahout.clustering.syntheticcontrol.kmeans.Job) we see:
08/10/27 21:18:41 INFO mapred.JobClient: Task Id :  
attempt_200810272112_0011_r_000000_0, Status : FAILED
java.io.IOException: attempt_200810272112_0011_r_000000_0The reduce  
copier failed
         at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:255)
         at org.apache.hadoop.mapred.TaskTracker 
$Child.main(TaskTracker.java:2207)

If I run this exact same job on 0.17.2 it all runs fine.  I suppose  
either a bug was introduced in 0.18.1 or a bug was fixed that we were  
relying on.  Looking at the release notes between the fixes, nothing  
in particular struck me as related.  If it helps, I can provide the  
instructions for how to run the example in question (they need to be  
written up anyway!)


I see some related things at http://hadoop.markmail.org/search/?q=Merge+of+the+inmemory+files+threw+an+exception

, but those are older, it seems, so not sure what to make of them.

Thanks,
Grant

Mime
View raw message