hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Austin Chungath <austi...@gmail.com>
Subject Hadoop fair scheduler doubt: allocate jobs to pool
Date Wed, 29 Feb 2012 13:18:14 GMT
How can I set the fair scheduler such that all jobs submitted from a
particular user group go to a pool with the group name?

I have setup fair scheduler and I have two users: A and B (belonging to the
user group hadoop)

When these users submit hadoop jobs, the jobs from A got to a pool named A
and the jobs from B go to a pool named B.
 I want them to go to a pool with their group name, So I tried adding the
following to mapred-site.xml:

<property>
 <name>mapred.fairscheduler.poolnameproperty</name>
<value>group.name</value>
</property>

But instead the jobs now go to the default pool.
I want the jobs submitted by A and B to go to the pool named "hadoop". How
do I do that?
also how can I explicity set a job to any specified pool?

I have set the allocation file (fair-scheduler.xml) like this:

<allocations>
  <pool name="hadoop">
    <minMaps>1</minMaps>
    <minReduces>1</minReduces>
    <maxMaps>3</maxMaps>
    <maxReduces>3</maxReduces>
  </pool>
  <userMaxJobsDefault>5</userMaxJobsDefault>
</allocations>

Any help is greatly appreciated.
Thanks,
Austin

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message