hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nitin Pawar <nitinpawar...@gmail.com>
Subject Re: Pending reducers
Date Tue, 14 Aug 2012 11:56:22 GMT
whats the memory/cpu stats on the machines ? are they exhausted

On Tue, Aug 14, 2012 at 5:20 PM, Evert Lammerts <Evert.Lammerts@sara.nl> wrote:
>> reducers of multiple jobs do run con-currently as long as they have the
>> resources available.
>
> Yep, and that's what's not happening in my situation. 528 reduce slots, 400 taken by
one job, 26 of another job remain in pending state. What could explain this behavior?
>
> Evert
>
>>
>> If you want to limit someone overtaking the cluster, then you can
>> create different job queues and assign quota to each queue. You also
>> have the flexibility of allocating max quota per user in a queue as
>> well.
>>
>>
>>
>> On Tue, Aug 14, 2012 at 4:09 PM, Evert Lammerts
>> <Evert.Lammerts@sara.nl> wrote:
>> > Hi list,
>> >
>> > I have a cluster running Hadoop 0.20.205 with Kerberos enabled,
>> exposing 528 map slots and 528 reduce slots. Currently somebody is
>> running a NORMAL priority job with 7 mappers and 400 reducers. The
>> mappers have finished and the system is processing the reducers.
>> Another user is running a NORMAL priority job with 1 mapper and 26
>> reducers. The mapper has finished, but the reducers won't come out of
>> "pending" state. There are no other jobs running right now. We've not
>> yet installed a different scheduler, so right now the system is using
>> the default scheduler. How can this behavior be explained? I see
>> mappers of multiple jobs run concurrently, and I *thought* I've seen
>> reducers of multiple jobs run concurrently, but I'm not completely
>> sure. Any idea?
>> >
>> > Evert
>>
>>
>>
>> --
>> Nitin Pawar



-- 
Nitin Pawar

Mime
View raw message