hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mikkel Kamstrup Erlandsen <...@statsbiblioteket.dk>
Subject Re: Task never gets past Map phase
Date Fri, 25 Aug 2006 07:06:59 GMT
On Thu, 2006-08-24 at 09:07 -0700, Owen O'Malley wrote:
> On Aug 24, 2006, at 7:01 AM, Mikkel Kamstrup Erlandsen wrote:
> 
> > I have been stumped on this problem for a few days now, I hope  
> > somebody
> > here has a clue... I'm running Hadoop 0.5.0.
> >
> > My problem is that the map tasks run for a while seemingly without
> > problems, and then start spitting out a lot of messages like this:
> 
> Are the map tasks marked as finished?

No, not as far as I can tell (from web interface)...

> The messages that you showed are from reduces that are waiting to get  
> inputs from some (in your case 13) maps that the job tracker doesn't  
> think have finished. If you look at the web ui what does numbers does  
> it show for the map tasks? In particular, how many maps are running,  
> pending, and done. 

The messages starts rolling in when the webinterface says:
MAP:
Pending: 9
Running: 4
Complete: 0
Failed: 0

REDUCE:
Running: 3 (all others 0)


> In the past, we've seen cases that looked like 13  
> pending, 0 running, n-13 done. Those particular bugs were fixed, but  
> they involved a set of map tasks that got lost between the job  
> tracker and the task tracker.
> 
> -- Owen

Cheers,
Mikkel


Mime
View raw message