hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hariharan <hariharan...@gmail.com>
Subject Re: Phase level Scheduling algorithm in heterogeneous cluster
Date Sat, 22 Apr 2017 04:26:31 GMT
Regarding the implementation - without knowing much about the algorithm - I
would guess you would at minimum need to implement a new scheduler - see
here
<https://github.com/apache/hadoop/tree/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler>
for some examples (fair/capacity/fifo). You may also be able to modify one
of the existing schedulers if the new algorithm isn't too different. You
would likely also need to change the MRAppMaster
<https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/MRAppMaster.java>
and the Mapper
<https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Mapper.java>
and Reducer
<https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Reducer.java>
classes to work with this algorithm.


HTH,
Hariharan

On Sat, Apr 22, 2017 at 7:04 AM, Ahmed Altaj <ahmed.altaj@yahoo.com.invalid>
wrote:

> Hi,
>
> I'm trying to deploy PRISM algorithm which proposed in IEEE paper title
> "prism: Fine-Grained Resource-Aware Scheduling for MapReduce", for phase
> level scheduling work  in homogeneous cluster.
> I'm stack with first step how to implement this algorithm and configure
> it, which parts i have to change.
>
> Best Regards...
> Ahmed
>

Mime
View raw message