hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sandeep Reddy P <sandeepreddy.3...@gmail.com>
Subject Map/Reduce Tasks Fails
Date Tue, 22 May 2012 14:02:23 GMT
We have a 5node cdh3u4 cluster running. When i try to do teragen/terasort
some of the map tasks are Failed/Killed and the logs show similar error on
all machines.

2012-05-22 09:43:50,831 INFO org.apache.hadoop.hdfs.DFSClient:
Exception in createBlockOutputStream
java.net.SocketTimeoutException: 69000 millis timeout while waiting
for channel to be ready for read. ch :
java.nio.channels.SocketChannel[connected local=/
2012-05-22 09:44:25,968 INFO org.apache.hadoop.hdfs.DFSClient:
Abandoning block blk_7260720956806950576_1825
2012-05-22 09:44:25,973 INFO org.apache.hadoop.hdfs.DFSClient:
Excluding datanode
2012-05-22 09:46:36,350 WARN org.apache.hadoop.mapred.Task: Parent
died.  Exiting attempt_201205211504_0007_m_000016_1.

Are these kind of errors common?? Atleast 1 map task is failing due to
above reason on all the machines.We are using 24 mappers for teragen.
For us it took 3hrs 44min 17 sec to generate 50Gb data with 24 mappers
and 17failed/8 killed task attempts.

24min 10 sec for 5GB data with 24 mappers and 9 killed Task attempts.
Cluster works good for small datasets.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message