mesos-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From haosdent <haosd...@gmail.com>
Subject Re: decline_offer_duration for spark frameworks
Date Fri, 15 Apr 2016 03:37:38 GMT
>I could not find any details on how set decline offer timeout in the spark
framework.
Seems spark framework don't support this while marathon support decline
offer duration.

Have you ever try set different role and reservation resources for
marathon? Related documents are [reservation.md](
https://github.com/apache/mesos/blob/master/docs/reservation.md) and [
weights.md](https://github.com/apache/mesos/blob/master/docs/weights.md)

On Fri, Apr 15, 2016 at 11:24 AM, David Greenberg <dsg123456789@gmail.com>
wrote:

> Hi Rodrick,
> You should check out Cook (github.com/twosigma/cook). It acts as a server
> for Spark drivers, except that it can manage preemption and fair-sharing
> between those drivers, so that you don't encounter starvation. If you'd
> like to discuss it, I'd be happy to hop on a call.
>
> Best,
> David
>
> On Thu, Apr 14, 2016 at 11:09 PM haosdent <haosdent@gmail.com> wrote:
>
>> Hi, @Rodrick Which spark mode are you running?  coarse-grained
>> or fine-grained?
>>
>> On Fri, Apr 15, 2016 at 10:05 AM, Rodrick Brown <rodrick@orchard-app.com>
>> wrote:
>>
>>> I have hundreds of small spark jobs running on my Mesos cluster
>>> causing starvation to other frameworks like Marathon on my cluster.
>>>
>>> Is their a way to prevent these frameworks from getting offers so often?
>>>
>>> Apr 15 02:00:12 prod-mesos-m-3.$SERVER.com mesos-master[10259]: I0415
>>> 02:00:12.503734 10266 master.cpp:3641] Processing DECLINE call for offers:
>>> [ 50ceafa4-f3c1-4738-a9eb-c5d3bf0ff742-O7112667 ] for
>>> framework 50ceafa4-f3c1-4738-a9eb-c5d3bf0ff742-15936 (KafkaDirectConsumer[trades-topic])
>>> at scheduler-9e557d33-e4a4-44ce-9dbe-0a7ca7c4842d@172.1.121.183:34858.
>>>
>>>
>>>
>>>
>>>
>>> --
>>>
>>> *Rodrick Brown* / Systems Engineer
>>>
>>> +1 917 445 6839 / rodrick@orchardplatform.com
>>> <charlie@orchardplatform.com>
>>>
>>> *Orchard Platform*
>>>
>>> 101 5th Avenue, 4th Floor, New York, NY 10003
>>>
>>> http://www.orchardplatform.com
>>>
>>> Orchard Blog <http://www.orchardplatform.com/blog/> | Marketplace
>>> Lending Meetup <http://www.meetup.com/Peer-to-Peer-Lending-P2P/>
>>>
>>> *NOTICE TO RECIPIENTS*: This communication is confidential and intended
>>> for the use of the addressee only. If you are not an intended recipient of
>>> this communication, please delete it immediately and notify the sender
>>> by return email. Unauthorized reading, dissemination, distribution or
>>> copying of this communication is prohibited. This communication does not constitute
>>> an offer to sell or a solicitation of an indication of interest to purchase
>>> any loan, security or any other financial product or instrument, nor is it
>>> an offer to sell or a solicitation of an indication of interest to purchase
>>> any products or services to any persons who are prohibited from receiving
>>> such information under applicable law. The contents of this communication
>>> may not be accurate or complete and are subject to change without notice.
>>> As such, Orchard App, Inc. (including its subsidiaries and affiliates,
>>> "Orchard") makes no representation regarding the accuracy or
>>> completeness of the information contained herein. The intended recipient is
>>> advised to consult its own professional advisors, including those
>>> specializing in legal, tax and accounting matters. Orchard does not
>>> provide legal, tax or accounting advice.
>>>
>>
>>
>>
>> --
>> Best Regards,
>> Haosdent Huang
>>
>


-- 
Best Regards,
Haosdent Huang

Mime
View raw message