mesos-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vinod Kone <>
Subject Re: Allocation algorithm
Date Tue, 25 Aug 2015 18:43:25 GMT
The hierarchical allocator looks at one agent's resource at a time. For
each agent, it runs DRF to figure out the candidate framework.

More details here:

Regarding starvation you observed, yes that is possible with DRF. We plan
to address this by optimistic offers (no implementation yet) and quotas

On Mon, Aug 24, 2015 at 8:42 AM, Hans van den Bogert <>

> Can anyone tell how the Mesos allocation algorithm works:
> Does Mesos offer every free resource it has to one framework at a time? Or
> does the allocator divide the max offer size by the amount of
> active/registered frameworks?
>   and
> in case of:
>   FW1 has a high dominant resource fraction (>50%), which it does not
> release. FW2 and FW3 have a lot of churn for their tasks, both have
> outstanding short lived tasks in their queue (shorter than the mesos
> allocation interval), these 2 FWs accept all resources Mesos has to offer -
> if they get the offer.
> Reading the DRF paper and presentation, am I to assume the online DRF
> algorithm would favour FW2 and FW3 always before FW1? As one of the two
> (FW2/3) will always (or at least more likely to,) have a lower dominant
> resource than FW1. According to the presentation on DRF, the framework with
> the lowest dominant resource gets the offer. But this is a potential
> starvation e.g., if a framework has allocated memory, but needs a new offer
> with CPUs to actually do something. You might wonder why the framework
> didn’t use memory AND cpu from the same offer, but Spark for example does
> exactly this.
> To give some context, I think I’m seeing this behaviour with Spark in
> fine-grained mode. I have 4 spark instances which are long-lived, emulating
> interactive queries. The first Spark instance to get an offer “installs”
> executors (with high memory demand) on every slave node it sees. The next
> framework tries to do the same, but for these later instances, theres not
> always enough executor memory, that’s why I end up with an instance, which
> was first to get the offer, with a lot of memory it doesn’t let go, but it
> also gets way less offers for CPU afterwards. In contrast the later spark
> instances with less long-living executors do not have a high memory usage,
> and get relatively more CPU offers.
> Of course setting a max amount  of  Spark executors per framework instance
> would mitigate this, but then I’m basically back to static allocation of
> resources.
> Thanks in advance,
> Hans

View raw message