mesos-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paco Nathan <cet...@gmail.com>
Subject Re: Service Scheduling in Mesos
Date Fri, 20 Sep 2013 23:44:29 GMT
>From what I understand, the "Omega" paper was written in 2012. It's great.
Much has been added to Apache Mesos since. Particularly w.r.t. scheduling
services. Also, the two-level categorization arguably has evolved further.


On Thu, Sep 19, 2013 at 11:55 AM, Bernerd Schaefer
<bernerd@soundcloud.com>wrote:

> Thanks for the response, Bill. Some followups below.
>
>
>> I haven't found a great way to approach either of these in mesos without
>>> assuming that your framework has full control of the cluster.  This is
>>> covered a bit in the Omega paper [1]:
>>>
>>
>> *While a Mesos framework can use “filters” to describe **the kinds of
>> resources that it would like to be offered, it does **not have access to
>> a view of the overall cluster state – just the **resources it has been
>> offered. As a result, it cannot support **preemption or policies
>> requiring access to the whole cluster **state: a framework simply does
>> not have any knowledge **of resources that have been allocated to other
>> schedulers.*
>>
>>
> I'm curious about your take as a framework author on the Omega paper's
> evaluation of Mesos. I would summarize their valuation somewhere between --
> best-case -- "Mesos is currently non-optimal for running a service
> scheduler alongside other schedulers," and -- worst-case -- "Mesos is
> fundamentally unsuitable for service schedulers which do not own the entire
> cluster."
>
> The risk with this approach is that you wind up not playing nicely with
>> other frameworks, possibly starving them of offers.  Unfortunately this is
>> the best way i've found to glean the shape of the cluster.
>>
>
>> Aurora cheats here by 'pinning' tasks to the same machines all the time,
>> and (currently) not running anything else on those machines.  Of course,
>> this strategy falls apart when other frameworks are introduced.  I believe
>> mesos' reservations feature intends to address this.
>>
>
> Given these comments -- am I to gather that Aurora runs on its own
> dedicated Mesos cluster? Regardless, it sounds like you've had to make
> Aurora itself a monolithic scheduler, which is discouraging.
>
> To my mind, the promise of Mesos is that I shouldn't have to build a
> scheduler that works for all-different kinds of tasks. I dream of a Mesos
> where my scheduler for stateless services lives happily alongside both my
> haproxy, memcached, elasticsearch schedulers, and my hadoop, spark, storm
> schedulers. Is it just not there yet?
>
> Bernerd
> Engineer @ SoundCloud
>

Mime
View raw message