hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Manhee Jo ...@nttdocomo.com>
Subject two questions regarding hadoop fair scheduler
Date Thu, 09 Sep 2010 08:42:52 GMT
Hi all,

1. How can I assing a job to a specific pool from command line?
I succeeded it from MR programming by including configuration code below

conf.set("mapred.queue.name", "testpool")

I've tried
% hadoop jar -D mapred.queue.name=testpool my.jar myclass input output
from command line, which is no avail.

2. Let's say, I have a "testpool" with below configuration while the total # 
of map task capacity is 1000.

<allocations>
  <pool name=“testpool”>
    <minMaps>500</minMaps>
    <minReduces>100</minReduces>
    <maxRunningJobs>3</maxRunningJobs>
  </pool>
</allocation>

Job1 needs 1000 mappers.
Job2 needs 3000 mappers.
Both jobs are running in the "testpool", each with 500 map task slots.
Then, Job3 which needs 1000 mappers arrives.
Should Job3 wait until it acquires enough slots in order to guarantee 
minimum # of slots (=500) to Job1 and Job2?
Or are 333 task slots assigend to Job3 as well as Job1 and Job2?

Thanks,
Manhee



Mime
View raw message