mesos-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brian Candler <b.cand...@pobox.com>
Subject Re: Batch/queue frameworks?
Date Wed, 07 Oct 2015 08:37:43 GMT
On 07/10/2015 09:01, Nikolaos Ballas neXus wrote:
> Check for Marathon

I don't see how Marathon does what I want. Maybe I wasn't clear enough 
in explaining my requirements.

What I need is basically a supercomputer cluster where I can take a 
large computation job, break it into lots of sub-tasks, and run as many 
of those sub-tasks in parallel as possible given the CPU resources 
available, until all the sub-tasks are done.

The core of any sort of system like that is a "job queue" where all the 
sub-tasks are entered. The executor picks out another task whenever 
there is some free resource available, and when it finishes, it is 
removed from the queue.

I don't see how Marathon has such a job queue. As far as I can tell, 
Marathon is for starting long-lived applications; you define what things 
you want running, it starts them, and restarts them if they die for any 
reason.

Or have I misunderstood what Marathon is capable of? If so, can you 
point me at the relevant documentation?

The advantage of running such a supercomputer cluster under Mesos would 
be that I could run *other* applications (including those started by 
Marathon or Chronos) on the same hardware.

Thanks,

Brian.


Mime
View raw message