mesos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Varun Gupta <var...@uber.com>
Subject Re: Add hostname or agentid in rescind offers callback
Date Mon, 07 May 2018 06:53:45 GMT
Does the feature request seems reasonable?
On Wed, May 2, 2018 at 7:35 PM Varun Gupta <varung@uber.com> wrote:

> Implementation is only needed for V1 API.
> On Wed, May 2, 2018 at 7:31 PM Varun Gupta <varung@uber.com> wrote:
>
>> We aggregate all the offers for a host, such that placement engine can
>> pack multiple tasks that can be launched on this host using aggregated
>> resources. If offers are unused for that host then they will be implicitly
>> declined.
>>
>> Placement cycle has two components
>> - determine tasks that can be launched on this host and enque those tasks
>> to be launched.
>> - second is to deque and launch them.
>>
>> During this placement cycle, offers can be rescinded and accordingly
>> placement has to be adjusted. For that we maintain second map of offer id
>> -> host name.
>>
>> Benefits of adding host name and agent id in rescind offer callback.
>> - Mutex locks to synchronize both maps, leads to some performance hit.
>> - Managing second map, is more code and prone to bugs.
>> - Little overhead on heap memory and GC.
>> On Wed, May 2, 2018 at 5:08 PM Benjamin Mahler <bmahler@apache.org>
>> wrote:
>>
>>> I'm a -1 on adding redundant information in the message.
>>>
>>> The scheduler can maintain an index of offers by offer id to address this
>>> issue:
>>>
>>> hostname -> offers
>>> offer_id -> offer
>>>
>>> On Wed, May 2, 2018 at 11:39 AM, Vinod Kone <vinodkone@apache.org>
>>> wrote:
>>>
>>> > Can I ask why you are indexing the offers by hostname? Is it to better
>>> > handle agent removal / unreachable signal?
>>> >
>>> > Looking at the code
>>> > <
>>> https://github.com/apache/mesos/blob/master/src/master/master.cpp#L11036
>>> >
>>> > ,
>>> > I think master has the requested information (agent id, hostname) so
>>> we can
>>> > include it in the rescind message!
>>> >
>>> > But there are couple things to discuss.
>>> >
>>> > The extra information to be included in rescind message is technically
>>> > redundant. So we need to figure out a guideline on what information
>>> should
>>> > be included / not included (e.g., should we include agent IP too) in
>>> such
>>> > calls.
>>> >
>>> > Second, adding this extra information in v1 scheduler API would be
>>> > relatively easy. But adding this to v0 API would be hard. Which API do
>>> you
>>> > need to be updated?
>>> >
>>> >
>>> > On Wed, May 2, 2018 at 10:31 AM, Varun Gupta <varung@uber.com> wrote:
>>> >
>>> > > Hi,
>>> > >
>>> > > Currently in our implementation we maintain two maps.
>>> > >
>>> > > Hostname -> []Offers
>>> > >
>>> > > offerID -> Hostname
>>> > >
>>> > > Second map is needed because rescind offers callback only provides
>>> > offerid
>>> > > and we need hostname to do performant lookup in first map.
>>> > >
>>> > > Is it feasible to add hostname or agentid in rescind offers?
>>> > >
>>> > > Thanks,
>>> > > Varun
>>> > >
>>> >
>>>
>>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message