hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "hao.wang" <hao.w...@ipinyou.com>
Subject Fair Scheduler Problem
Date Wed, 07 Mar 2012 03:05:29 GMT
Hi ,All,
    I encountered a problem in using Cloudera Hadoop 0.20.2-cdh3u1. When I use the fair Scheduler
I find the scheduler seems  not support preemption.  
    Can anybody tell me whether preemption is supported in this version?
    This is my configration:
 mapred-site.xml   
<property> 
  <name>mapred.jobtracker.taskScheduler</name> 
  <value>org.apache.hadoop.mapred.FairScheduler</value> 
</property> 
<property>
      <name>mapred.fairscheduler.allocation.file</name>
      <value>/usr/lib/hadoop-0.20/conf/fair-scheduler.xml</value>
</property>
<property>
<name>mapred.fairscheduler.preemption</name>
<value>true</value>
</property>
<property>
<name>mapred.fairscheduler.preemption.only.log</name>
<value>true</value>
</property>
<property>
<name>mapred.fairscheduler.preemption.interval</name>
<value>15000</value>
</property>
<property>
  <name>mapred.fairscheduler.weightadjuster</name>
  <value>org.apache.hadoop.mapred.NewJobWeightBooster</value>
</property>
<property>
  <name>mapred.fairscheduler.sizebasedweight</name>
  <value>true</value>
</property>
fair-scheduler.xml 
<allocations>
   <pool name="root">
      <minMaps>10</minMaps>
    <minReduces>5</minReduces>
    <maxMaps>200</maxMaps>
   <maxReduces>80</maxReduces>
       <maxRunningJobs>100</maxRunningJobs>
      <minSharePreemptionTimeout>30</minSharePreemptionTimeout>
        <weight>1.0</weight>
  </pool>
  <pool name="hadoop">
       <minMaps>10</minMaps>
    <minReduces>5</minReduces>
   <maxMaps>80</maxMaps>
   <maxReduces>80</maxReduces>
        <maxRunningJobs>5</maxRunningJobs>
       <minSharePreemptionTimeout>30</minSharePreemptionTimeout>
       <weight>1.0</weight>
  </pool>
  <user name="user1">
       <maxRunningJobs>10</maxRunningJobs>
  </user>
    <poolMaxJobsDefault>20</poolMaxJobsDefault>
   <userMaxJobsDefault>10</userMaxJobsDefault>
   <defaultMinSharePreemptionTimeout>30</defaultMinSharePreemptionTimeout>
   <fairSharePreemptionTimeout>30</fairSharePreemptionTimeout>
</allocations>

regards,

2012-03-07 



hao.wang 

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message