hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bejoy KS" <bejoy.had...@gmail.com>
Subject Re: one reducer is hanged in "reduce-> copy" phase
Date Tue, 28 Aug 2012 16:48:48 GMT
Hi Abhay

The map outputs are deleted only after the reducer runs to completion. 

>Is it possible to run the same attempt again? Does killing the child java process or tasktracker
on the node help? (since hadoop may schedule a reduce attempt on another node).

Yes,it is possible to re attempt the task again for that you need to fail the current attempt.


>Can I copy the map intermediate output required for this single reducer (which is hanged)
and rerun only the hang reducer?

It is not that easy to accomplish this. Better fail the task explicitly so that the it is
re attempted.

Regards
Bejoy KS

Sent from handheld, please excuse typos.

-----Original Message-----
From: Abhay Ratnaparkhi <abhay.ratnaparkhi@gmail.com>
Date: Tue, 28 Aug 2012 19:40:58 
To: <user@hadoop.apache.org>
Reply-To: user@hadoop.apache.org
Subject: one reducer is hanged in "reduce-> copy" phase

Hello,

I have a MR job which has 4 reducers running.
One of the reduce attempt is pending since long time in reduce->copy phase.

The job is not able to complete because of this.
I have seen that the child java process on tasktracker is running.

Is it possible to run the same attempt again? Does killing the child java
process or tasktracker on the node help? (since hadoop may schedule a
reduce attempt on another node).

Can I copy the map intermediate output required for this single reducer
(which is hanged) and rerun only the hang reducer?

Thank you in advance.
~Abhay


ask_201208250623_0005_r_000000<http://dpep089.innovate.ibm.com:50030/taskdetails.jsp?tipid=task_201208250623_0005_r_000000>
26.41%

reduce > copy(103 of 130 at 0.08 MB/s)
28-Aug-2012 03:09:34

Mime
View raw message