hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Srinivas Surasani <vas...@gmail.com>
Subject Re: Understanding fair schedulers
Date Wed, 25 Jan 2012 12:45:28 GMT
Praveenesh,

You can try specifying "mapred.fairscheduler.pool" to your pool name while
running the job. By default, mapred.faircheduler.poolnameproperty set to
user.name ( each job run by user is allocated to his named pool ) and you
can also change this property to group.name.

Srinivas --

Also, you can set

On Wed, Jan 25, 2012 at 6:24 AM, praveenesh kumar <praveenesh@gmail.com>wrote:

> Understanding Fair Schedulers better.
>
> Can we create mulitple pools in Fair Schedulers. I guess Yes. Please
> correct me.
>
> Suppose I have 2 pools in my fair-scheduler.xml
>
> 1. Hadoop-users : Min map : 10, Max map : 50, Min Reduce : 10, Max Reduce :
> 50
> 2. Admin-users: Min map : 20, Max map : 80, Min Reduce : 20, Max Reduce :
> 80
>
> I have 5 users, who will be using these pools. How will I allocate specific
> pools to specific users ?
>
> Suppose I want user1,user2 to use "Hadoop-users" pool and user3,user4,user5
> to use "Admin users"
>
> In http://hadoop.apache.org/common/docs/r0.20.205.0/fair_scheduler.html
> they have mentioned allocations something like this.
>
> <?xml version="1.0"?>
> <allocations>
>  <pool name="sample_pool">
>    <minMaps>5</minMaps>
>    <minReduces>5</minReduces>
>    <maxMaps>25</maxMaps>
>    <maxReduces>25</maxReduces>
>    <minSharePreemptionTimeout>300</minSharePreemptionTimeout>
>  </pool>
>  <user name="sample_user">
>    <maxRunningJobs>6</maxRunningJobs>
>  </user>
>  <userMaxJobsDefault>3</userMaxJobsDefault>
>  <fairSharePreemptionTimeout>600</fairSharePreemptionTimeout>
> </allocations>
>
> I tried creating more pools, its happening, but how to allocate users to
> use specific pools ?
>
> Thanks,
> Praveenesh
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message