hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: configure hadoop-0.22 fairscheduler
Date Sat, 08 Sep 2012 05:08:35 GMT
Hey Jameson,

When calling something inefficient, perhaps also share some details on
how/why/what? How else would we know what you wish to see and what
you're seeing instead? :)

On Thu, Sep 6, 2012 at 2:47 PM, Jameson Li <hovlj.ei@gmail.com> wrote:
> I want to test version hadoop-0.22.
> But when configurate the fairescheduler, I have some troublesome. The
> fairscheduler is not efficient.
> And I have configured this items in the mapred-site.xml, and also I hava
> copy the fairscheduler jar file to the $HADOOP_HOME/lib:
>
>   <property>
>     <name>mapreduce.jobtracker.taskScheduler</name>
>     <value>org.apache.hadoop.mapred.FairScheduler</value>
>   </property>
>
> <property>
>         <name>mapred.fairscheduler.allocation.file</name>
>         <value>conf/pools.xml</value>
> </property>
> <property>
>         <name>mapred.fairscheduler.preemption</name>
>         <value>true</value>
> </property>
> <property>
>         <name>mapred.fairscheduler.assignmultiple</name>
>         <value>true</value>
> </property>
> <property>
>         <name>mapred.fairscheduler.poolnameproperty</name>
>         <value>mapred.queue.name</value>
>         <description>job.set("mapred.queue.name",pool); // pool is set to
> either 'high' or 'low' </description>
> </property>
> <property>
>         <name>mapred.queue.names</name>
>         <value>default,aaa,bbb</value>
> </property>
>
> And the pools.xml in $HADOOP_HOME/conf 's content:
>
> <?xml version="1.0"?>
> <allocations>
>   <pool name="putindb">
>    <minMaps>72</minMaps>
>    <minReduces>16</minReduces>
>    <maxRunningJobs>20</maxRunningJobs>
>    <weight>3.0</weight>
>    <minSharePreemptionTimeout>60</minSharePreemptionTimeout>
>   </pool>
>
>   <pool name="machinelearning">
>    <minMaps>9</minMaps>
>    <minReduces>2</minReduces>
>    <maxRunningJobs>10</maxRunningJobs>
>    <weight>2.0</weight>
>    <minSharePreemptionTimeout>60</minSharePreemptionTimeout>
>   </pool>
>
>   <pool name="default">
>    <minMaps>9</minMaps>
>    <minReduces>2</minReduces>
>    <maxRunningJobs>10</maxRunningJobs>
>    <weight>1.0</weight>
>    <minSharePreemptionTimeout>60</minSharePreemptionTimeout>
>   </pool>
>
>   <defaultMinSharePreemptionTimeout>60</defaultMinSharePreemptionTimeout>
>   <fairSharePreemptionTimeout>60</fairSharePreemptionTimeout>
> </allocations>
>
> Can someone help me?
>
>
> 专注于Mysql,MSSQL,Oracle,Hadoop



-- 
Harsh J

Mime
View raw message