hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yuesheng Hu <yueshen...@gmail.com>
Subject Re: Hama Scheduler
Date Tue, 24 Dec 2013 14:52:08 GMT
Hi Lin,
Thank you for your reply.



2013/12/24 Chia-Hung Lin <clin4j@googlemail.com>

> BSPMaster makes use of TaskScheduler for scheduling tasks
>
> BSPMaster.java
> Class<? extends TaskScheduler> schedulerClass = conf.getClass(
>         "bsp.master.taskscheduler", SimpleTaskScheduler.class,
>         TaskScheduler.class);
> this.taskScheduler = ReflectionUtils.newInstance(schedulerClass, conf);
>
> Then in SimpleTaskScheduler, tasks are scheduled through schedule
> function. And tasks are obtained by related JobInProgress.
>
> That's roughly the execution path.
>
> IIRC the split size is more related to a single job so increasing
> split size may not allow more jobs to be ran on the same cluster.
>
> At the moment the scheduling mechanism is done by creating a task per
> GroomServer, and each GroomServer allows default maxTasks up to 3. So
> increasing maxTasks may be a way to increase job running concurrently;
> or restricting tasks scheduled to GroomServer, and then scheduling
> tasks in the new job to free slots may also help increase the
> concurrent job execution.
>
> Right now scheduling is just a simple FCFS. It's welcome the
> improvment on adding something like policy so that the scheduling
> mechanism is more flexible.
>
>
>
>
>
>
> On 24 December 2013 20:06, Yuesheng Hu <yueshenghu@gmail.com> wrote:
> > Hi, I want to implement a scheduler for hama as my thesis project.
> > Here are my ideas:
> > 1. make the split size as big as possible, so more jobs can be run on the
> > cluster;
> > 2. the scheduler can select a job based on the job's type, like graph job
> > (message intensive) or iterative job
> >
> > Any advice?
> >
> > Best Regards!
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message