mesos-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vinod Kone <vinodk...@gmail.com>
Subject Re: Question about Mesos protocol
Date Fri, 07 Mar 2014 02:02:29 GMT
Radek is spot on.


On Thu, Mar 6, 2014 at 5:58 PM, Radek Burkat <radek@pinkbike.com> wrote:

> Just reread you post and if you require launching a big job of multiple
> tasks at the same time, then I believe you would just cache the offers from
> multiple slaves and when ready, launch your job, which would just be
> launching separately tasks on each of those slaves/offers you have cached.
>
> Radek
>
>
> On Thu, Mar 6, 2014 at 5:45 PM, Radek Burkat <radek@pinkbike.com> wrote:
>
>> From what I understand...you receive offers and at that point you have to
>> decide if you are going to cache them, launch a task on them, or decline
>> them.  If you don't launch a task or don't decline, the allocator will
>> assume you are using the offer and count it as part of your share, and it
>> won't offer it to another framework.  ( perhaps the allocator later can
>> rescind to offer, but I have not seen that happen under my conditions )
>> So typically I think you want to decline it to be a good citizen.
>>
>> What happens if you start launching tasks that use say 1 cpu each of a
>> multi cpu slave offer, you will fragment the resources and start receiving
>> multiple offers for the _same_ slave. Each offer will contain a subset of
>> the original resources.  When you get this type of fragmenting it is then
>> that you can launch a single task on multiple offers of the same slave at
>> the same time, if the sum of those resources is what you need.
>>
>> In your case you are talking about launching a "big" task on multiple
>> "workers", and if in workers you mean multiple slaves, I don't think it is
>> possible to launch a task across multiple slaves.
>>
>> Disclaimer:  I'm still trying to understand mesos myself.
>>
>> Radek
>>
>>
>>
>>
>> On Thu, Mar 6, 2014 at 4:55 PM, Lin Zhao <lin@groupon.com> wrote:
>>
>>> The interaction between a scheduler and Mesos as I understand is:
>>>
>>>
>>> while(true): //resourceOffers method
>>>   Mesos sends a list of Offers to the scheduler. Each Offer contains
>>> resource info of a Mesos worker.
>>>   Scheduler determine what to do with the list of offers
>>>   continue
>>>
>>> My question is, when Mesos sends a new list of offers, are previous
>>> offers still valid? Is the scheduler being a good citizen by caching offers
>>> and launching it later, after other offer lists are received? Assuming no
>>> offerRecinced is called.
>>>
>>> The reason I ask is that I noticed in some cases Mesos may break down
>>> available resources in different offerings.
>>>
>>> Say there are 3 workers in the cluster (mesos1, mesos2, mesos3). After
>>> some time the schedule may see:
>>>
>>> *[offer1: mesos1, offer2: mesos2]*
>>>
>>> *(2 seconds later)*
>>>
>>> *[offer1: mesos3]*
>>>
>>> *(2 seconds later)*
>>>
>>> *(repeat)*
>>>
>>>
>>> As a schedule, I believe I can use all 3 offers to launch a big task. Is
>>> this true?
>>>
>>>
>>> --
>>> Lin Zhao
>>>
>>> https://wiki.groupondev.com/Message_Bus
>>> 3101 Park Blvd, Palo Alto, CA 94306
>>>
>>>
>>>
>>>
>>
>>
>> --
>> *RADEK BURKAT  **FOUNDER*
>>
>> *//pinkbike <http://www.pinkbike.com/>*
>>
>> *PHONE 1-604-351-2581*
>> *FAX *+1 403 668 0887
>> *SKYPE radek.burkat*
>>
>> *pb <http://radek.pinkbike.com/>  facebook
>> <http://www.facebook.com/pinkbikecom> * *twitter
>> <http://www.twitter.com/pinkbike>*
>>
>> *Confidentiality notice and disclaimer*: The information in this email
>> may be confidential and is intended only for the person or persons to whom
>> it is addressed. If you are not the individual to whom it is addressed, you
>> are warned that any disclosure, copying or dissemination of the information
>> is unauthorized. If you have received the email in error, please
>> immediately contact Pinkbike.com by telephone or email, to inform us of the
>> error. No liability is accepted for any unauthorized use of the information
>> contained in this email.
>>
>
>
>
> --
> *RADEK BURKAT  **FOUNDER*
>
> *//pinkbike <http://www.pinkbike.com/>*
>
> *PHONE 1-604-351-2581*
> *FAX *+1 403 668 0887
> *SKYPE radek.burkat*
>
> *pb <http://radek.pinkbike.com/>  facebook
> <http://www.facebook.com/pinkbikecom> * *twitter
> <http://www.twitter.com/pinkbike>*
>
> *Confidentiality notice and disclaimer*: The information in this email
> may be confidential and is intended only for the person or persons to whom
> it is addressed. If you are not the individual to whom it is addressed, you
> are warned that any disclosure, copying or dissemination of the information
> is unauthorized. If you have received the email in error, please
> immediately contact Pinkbike.com by telephone or email, to inform us of the
> error. No liability is accepted for any unauthorized use of the information
> contained in this email.
>

Mime
View raw message