hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marek Miglinski <mmiglin...@seven.com>
Subject RE: Capacity Scheduler problem
Date Wed, 11 Jan 2012 10:31:27 GMT

That's not the case, I've even removed hadoop-fairscheduler-0.20.2-cdh3u2.jar from hadoop
lib folder and fair-scheduler.xml from hadoop conf folder and it didn't help... Any ideas?

________________________________
From: Bai Shen [baishen.lists@gmail.com]
Sent: Tuesday, January 10, 2012 9:35 PM
To: mapreduce-user@hadoop.apache.org
Subject: Re: Capacity Scheduler problem

Turn off the fairscheduler multiple task assign setting.  I just had the same problem with
my cluster.

On Tue, Jan 10, 2012 at 11:34 AM, Marek Miglinski <mmiglinski@seven.com<mailto:mmiglinski@seven.com>>
wrote:
Hello guys,

1. I have concern with my 3 node cluster, I run capacity scheduler with 4 queues and one has
30% of cluster resources, the problem is that when I schedule a job, all tasks are assigned
to one single node which takes all of it's mappers and works quite slow. Are there any settings
for mapred/capacity scheduler to assign mappers on all nodes by even number of tasks?

2. I've set my capacity-scheduler.xml settings "mapred.capacity-scheduler.queue.job1.capacity"
for all queues as wanted but jobtracker doesn't reallocate resources from free queues if only
one is working, why?


Thanks,
Marek M.


Mime
View raw message