hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 姚吉龙 <geelong...@gmail.com>
Subject How to improve performance of this cluster
Date Wed, 17 Apr 2013 09:42:00 GMT
Hi everyone

We have a cluster of 31 datanodes with 1 namenode,each with 8-core cpu and
8G RAM
I am studying the approach to improve the performance of this cluster.Now
we have a datafile of 100G as the test case.
when I add the reduce number form 100 to 200, I did not see larger
improvment from 23m52s to 19m44s. Besides there two failed task appear in
this process:

java.lang.Throwable: Child Error at
org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271) Caused by:
java.io.IOException: Task process exit with nonzero status of 126. at
org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)

Here is my conf in mapred-site.xml:[image: 内嵌图片 1]

1.Could any help about the failed task? Why would this happen?
2.How can I continue to speed up the process of this case.
Any suggestion is welcome


BRs
Geelong

-- 
>From Good To Great

Mime
View raw message