hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arun C Murthy <ar...@yahoo-inc.com>
Subject Re: why it stopped at Reduce phase?
Date Tue, 13 May 2008 15:22:36 GMT
Wang,

On May 13, 2008, at 8:12 AM, wangxiaowei wrote:

> hi all:
>    I uses two computers A and B as a hadoop cluster,A is JobTracker  
> and NameNode,both A and B are slaves.
> The input data size is about 80MB,including 100,000records. The job  
> is to read one record a time and find some useful content in it,and  
> transmit it to reduce.
>    But when I submit it ,it just run map tasks,and the reduce task  
> did not run at all!This is the JobTacker`s log file:
> 2008-05-13 21:02:00,007 INFO  
> org.apache.hadoop.mapred.TaskInProgress: Error from  
> task_200805132055_0001_m_000000_0: FSError: java.io.IOException: No  
> space left on device
> 2008-05-13 21:02:11,952 INFO  
> org.apache.hadoop.mapred.TaskInProgress: Error from  
> task_200805132055_0001_m_000001_0: FSError: java.io.IOException: No  
> space left on device
> 2008-05-13 21:02:11,953 INFO  
> org.apache.hadoop.mapred.TaskInProgress: Error from  
> task_200805132055_0001_m_000006_0: FSError: java.io.IOException: No  
> space left on device
>    I think it is something wrong with the configuration file,Can  
> you give me some suggestions?
>    Do you meet the same questions?

It looks like your maps are failing since there isn't enough  
temporary disk space on your nodes.
Usually the outputs of maps are stored on the local-disk of the nodes  
on which they were executed, and it looks like your nodes don't have  
enough.
You can use multiple disks for your intermediate map-outputs by using  
the configuration knob: *mapred.local.dir*
(http://hadoop.apache.org/core/docs/current/hadoop- 
default.html#mapred.local.dir).

Arun

Mime
View raw message