hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bai Shen <baishen.li...@gmail.com>
Subject Re: Capacity Scheduler problem
Date Wed, 11 Jan 2012 21:10:52 GMT
Nope.  Like I said, all I did was change mapred.fairscheduler.assignmultiple
to false on my cluster and that fixed the issue.

On Wed, Jan 11, 2012 at 5:31 AM, Marek Miglinski <mmiglinski@seven.com>wrote:

>
> That's not the case, I've even removed
> hadoop-fairscheduler-0.20.2-cdh3u2.jar from hadoop lib folder and
> fair-scheduler.xml from hadoop conf folder and it didn't help... Any ideas?
>
> ________________________________
> From: Bai Shen [baishen.lists@gmail.com]
> Sent: Tuesday, January 10, 2012 9:35 PM
> To: mapreduce-user@hadoop.apache.org
> Subject: Re: Capacity Scheduler problem
>
> Turn off the fairscheduler multiple task assign setting.  I just had the
> same problem with my cluster.
>
> On Tue, Jan 10, 2012 at 11:34 AM, Marek Miglinski <mmiglinski@seven.com
> <mailto:mmiglinski@seven.com>> wrote:
> Hello guys,
>
> 1. I have concern with my 3 node cluster, I run capacity scheduler with 4
> queues and one has 30% of cluster resources, the problem is that when I
> schedule a job, all tasks are assigned to one single node which takes all
> of it's mappers and works quite slow. Are there any settings for
> mapred/capacity scheduler to assign mappers on all nodes by even number of
> tasks?
>
> 2. I've set my capacity-scheduler.xml settings
> "mapred.capacity-scheduler.queue.job1.capacity" for all queues as wanted
> but jobtracker doesn't reallocate resources from free queues if only one is
> working, why?
>
>
> Thanks,
> Marek M.
>
>

Mime
View raw message