hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Re: Reducer stuck at pending state
Date Wed, 17 Feb 2010 04:18:16 GMT
Hi Song,

What version are you running? How much memory have you allocated to
the reducers in mapred.child.java.opts?

-Todd

On Tue, Feb 16, 2010 at 4:01 PM, Song Liu <lamfeeling2@gmail.com> wrote:
> Sorry, seems no attachment is allowed, I paste it here:
>
> Jobid    Priority    User    Name    Map % Complete    Map Total    Maps
> Completed    Reduce % Complete    Reduce Total    Reduces Completed    Job
> Scheduling Information
> job_2... NORMAL      sl9885    TF/IDF     100.00%          26
> 26                0.00%                 1                0
>    NA
> job_2... NORMAL      sl9885    Rank    100.00%          22          
 22
>            0.00%                 1                0        
           NA
> job_2... NORMAL      sl9885    TF/IDF     100.00%          20
> 20                0.00%                1                0
> NA
>
> The format is horrible, sorry for that, but it's the best I can do :(
>
> BTW, I guess it should not be my program's problem, since I have tested it
> on some other clusters before.
>
> Regards
> Song Liu
>
> On Tue, Feb 16, 2010 at 11:51 PM, Song Liu <lamfeeling2@gmail.com> wrote:
>
>> Hi all, I recently have me t a problem that sometimes, reducer hang up at
>> pending state, with 0% complete.
>>
>> It seems all the mappers are completely done, and when it just about to
>> start the reducer, the reducer stuck, without any given warnings and errors
>> and was staying at the pending state.
>>
>> I have a cluster with 12 nodes. But this situation only appears when the
>> scale of data is large (2GB or more), smaller cases never met this problem.
>>
>> Any one has met this issue before? I searched JIRA, some one proposed this
>> issue before, but no solution was given. (
>> https://issues.apache.org/jira/browse/MAPREDUCE-24?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12647230#action_12647230
>> )
>>
>> The typical case of this issue is captured in the attachment.
>>
>> Regards
>> Song Liu
>>
>

Mime
View raw message