mesos-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Itamar Ostricher <ita...@yowza3d.com>
Subject Re: Is launchTasks() with multiple offers limited to a single slave?
Date Mon, 23 Mar 2015 10:59:00 GMT
Thanks Michael!

On Mon, Mar 23, 2015 at 7:59 AM, Michael Park <mcypark@gmail.com> wrote:

> Hi Itamar,
>
> Thanks for the patch! It looks like Niklas and Jie has looked at the patch
> and I'm sure they'll commit it soon, if not I'll nudge them :)
>
Great :-)

>
> 2. I would imagine there could be a "cleaner" way (from a framework author
>> perspective) to do that, by setting a policy or a filter or something, that
>> communicates to the master that the scheduler would like to receive only
>> offers that meet some criteria (e.g. min_cpu, min_mem, etc.). effectively,
>> moving the complexity of holding on to offers from the framework to the
>> master.
>>
>
> I don't think the ability to request for "only offer me resources that
> contains: [(cpus, 8), (mem, 1048)]" can actually replace the current
> ability of the frameworks, since the framework can currently build up the
> resources while holding onto the ones it was given. For example, suppose
> [(cpus, 2), (mem, 1048)] became available but the mesos master doesn't
> offer it to the framework because it does not meet the requirement, and
> instead offers it to another framework. Then [(cpus, 6)] becomes available
> and it still doesn't meet the requirement, so it gets offered to yet
> another framework. In order to avoid this situation, the framework would
> have to request for something like "offer me at least [(cpus, 8), (mem,
> 1048)] once you have built it up". I think we could support a mechanism
> like this via some form of "resource request".
>

I think your example is an excellent one to explain why I think the
"policy" approach can work better!
If framework A indeed requires [(cpus,8),(mem,1048)], and framework B is
happy with [(cpus,2),(mem,1048)], then with the current approach framework
A might want to hold on to [(cpus,2],(mem,1048)] until it gets another
[(cpus,6)] from the same slave. But the master may see that framework A is
not responding (because it's "holding on"), and offer the same
[(cpus,2),(mem,1048)] to framework B. Since framework B may immediately
accept, the offer will be rescinded from framework A.
This may make it difficult for framework A to get the resources it needs,
unless it has a way to tell the master that it wants [(cpus,8),(mem,1048)].
(unless I misunderstood how the master handles offering multiple resources
to multiple frameworks)


> It may also be satisfied by a reservation of some form. Some of these
> mechanisms such as "offer reservations" are described in MESOS-1791
> <https://issues.apache.org/jira/browse/MESOS-1791>.
>
> Is such a thing possible in mesos?
>
>
> Currently no, but it will be coming!
>
Great!

>
> Was it an explicit design decision to keep such logic at framework level?
>
>
> You may have noticed that there already exists a *Request* message in
> *mesos.proto* which currently does nothing. So while the logic lives at
> the framework-level right now, I don't think it was an explicit design
> decision to keep it there in the long run.
>
> MPark.
>

Mime
View raw message