hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Billy Pearson" <billy_pear...@sbcglobal.net>
Subject Re: Regarding Capacity Scheduler
Date Wed, 13 May 2009 21:46:51 GMT


I am seeing the the same problem posted on the list on the 11th and have not 
any reply.

Billy


----- Original Message ----- 
From: "Manish Katyal" 
<manish.katyal-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Newsgroups: gmane.comp.jakarta.lucene.hadoop.user
To: <core-user-7ArZoLwFLBtd/SJB6HiN2Ni2O/JbrIOy@public.gmane.org>
Sent: Wednesday, May 13, 2009 11:48 AM
Subject: Regarding Capacity Scheduler


> I'm experimenting with the Capacity scheduler (0.19.0) in a multi-cluster
> environment.
> I noticed that unlike the mappers, the reducers are not pre-empted?
>
> I have two queues (high and low) that are each running big jobs (70+ maps
> each).  The scheduler splits the mappers as per the queue
> guaranteed-capacity (5/8ths for the high and the rest for the low). 
> However,
> the reduce jobs are not interleaved -- the reduce job in the high queue is
> blocked waiting for the reduce job in the low queue to complete.
>
> Is this a bug or by design?
>
> *Low queue:*
> Guaranteed Capacity (%) : 37.5
> Guaranteed Capacity Maps : 3
> Guaranteed Capacity Reduces : *3*
> User Limit : 100
> Reclaim Time limit : 300
> Number of Running Maps : 3
> Number of Running Reduces : *7*
> Number of Waiting Maps : 131
> Number of Waiting Reduces : 0
> Priority Supported : NO
>
> *High queue:*
> Guaranteed Capacity (%) : 62.5
> Guaranteed Capacity Maps : 5
> Guaranteed Capacity Reduces : 5
> User Limit : 100
> Reclaim Time limit : 300
> Number of Running Maps : 4
> Number of Running Reduces : *0*
> Number of Waiting Maps : 68
> Number of Waiting Reduces : *7*
> Priority Supported : NO
> 



Mime
View raw message