mesos-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Greenberg <dsg123456...@gmail.com>
Subject Re: decline_offer_duration for spark frameworks
Date Fri, 15 Apr 2016 03:24:10 GMT
Hi Rodrick,
You should check out Cook (github.com/twosigma/cook). It acts as a server
for Spark drivers, except that it can manage preemption and fair-sharing
between those drivers, so that you don't encounter starvation. If you'd
like to discuss it, I'd be happy to hop on a call.

Best,
David

On Thu, Apr 14, 2016 at 11:09 PM haosdent <haosdent@gmail.com> wrote:

> Hi, @Rodrick Which spark mode are you running?  coarse-grained
> or fine-grained?
>
> On Fri, Apr 15, 2016 at 10:05 AM, Rodrick Brown <rodrick@orchard-app.com>
> wrote:
>
>> I have hundreds of small spark jobs running on my Mesos cluster
>> causing starvation to other frameworks like Marathon on my cluster.
>>
>> Is their a way to prevent these frameworks from getting offers so often?
>>
>> Apr 15 02:00:12 prod-mesos-m-3.$SERVER.com mesos-master[10259]: I0415
>> 02:00:12.503734 10266 master.cpp:3641] Processing DECLINE call for offers:
>> [ 50ceafa4-f3c1-4738-a9eb-c5d3bf0ff742-O7112667 ] for
>> framework 50ceafa4-f3c1-4738-a9eb-c5d3bf0ff742-15936 (KafkaDirectConsumer[trades-topic])
>> at scheduler-9e557d33-e4a4-44ce-9dbe-0a7ca7c4842d@172.1.121.183:34858.
>>
>>
>>
>>
>>
>> --
>>
>> *Rodrick Brown* / Systems Engineer
>>
>> +1 917 445 6839 / rodrick@orchardplatform.com
>> <charlie@orchardplatform.com>
>>
>> *Orchard Platform*
>>
>> 101 5th Avenue, 4th Floor, New York, NY 10003
>>
>> http://www.orchardplatform.com
>>
>> Orchard Blog <http://www.orchardplatform.com/blog/> | Marketplace
>> Lending Meetup <http://www.meetup.com/Peer-to-Peer-Lending-P2P/>
>>
>> *NOTICE TO RECIPIENTS*: This communication is confidential and intended
>> for the use of the addressee only. If you are not an intended recipient of
>> this communication, please delete it immediately and notify the sender
>> by return email. Unauthorized reading, dissemination, distribution or
>> copying of this communication is prohibited. This communication does not constitute
>> an offer to sell or a solicitation of an indication of interest to purchase
>> any loan, security or any other financial product or instrument, nor is it
>> an offer to sell or a solicitation of an indication of interest to purchase
>> any products or services to any persons who are prohibited from receiving
>> such information under applicable law. The contents of this communication
>> may not be accurate or complete and are subject to change without notice.
>> As such, Orchard App, Inc. (including its subsidiaries and affiliates,
>> "Orchard") makes no representation regarding the accuracy or
>> completeness of the information contained herein. The intended recipient is
>> advised to consult its own professional advisors, including those
>> specializing in legal, tax and accounting matters. Orchard does not
>> provide legal, tax or accounting advice.
>>
>
>
>
> --
> Best Regards,
> Haosdent Huang
>

Mime
View raw message