mesos-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From haosdent <>
Subject Re: Unable to start more than two executors on Mesos with Spark
Date Sat, 13 Feb 2016 09:25:08 GMT
Hi, I think in coarse grain mode, Spark would schedule and allocate
resource by itself, just like what it do in YARN. So you only could see two
I think you could get more details from

On Sat, Feb 13, 2016 at 4:16 AM, Rodrick Brown <>

> I'm having an issue with Spark when running on Mesos in coarse grain mode
> no matter what options I use or how many free resources I have I only ever
> see two executors start.
> If I run Spark in fine grain mode on Mesos I see tasks eventually ramp up
> to the desired number I specified.
> Anyone know what could be causing this?
> I'm starting my job using the following commands --
> /usr/bin/timeout 3600 /opt/spark-1.5.1/bin/spark-submit --master "
> mesos://zk://,,"
> --conf spark.ui.port=31278 --conf spark.mesos.coarse=true --conf
> spark.mesos.extra.cores=3 --conf spark.cores.max=5 --conf
> spark.mesos.constraints="rack:spark" --conf spark.sql.tungsten.enabled=true
> --class
> com.orchard.dataloader.library.originators.prosper.LoadTrade_Prosper
> --total-executor-cores 3 --driver-memory 1536M --executor-memory 3011M
> --jars /data/orchard/etc/load-tradedata-accumulo-prod
> /data/orchard/jars/dataloader-library-59e0441cd06544442d2a84f5842b701e81097988-assembled.jar
> I'm using spark-1.5.1 and Mesos 0.26 Chronos-2.4.0 is starting the spark
> driver.
> If anyone have any ideas on what/where I should be focusing on to debug
> this?
> --
> *Rodrick Brown* / Systems Engineer
> +1 917 445 6839 /
> <>
> *Orchard Platform*
> 101 5th Avenue, 4th Floor, New York, NY 10003
> Orchard Blog <> | Marketplace Lending
> Meetup <>
> *NOTICE TO RECIPIENTS*: This communication is confidential and intended
> for the use of the addressee only. If you are not an intended recipient of
> this communication, please delete it immediately and notify the sender by
> return email. Unauthorized reading, dissemination, distribution or copying
> of this communication is prohibited. This communication does not constitute
> an offer to sell or a solicitation of an indication of interest to purchase
> any loan, security or any other financial product or instrument, nor is it
> an offer to sell or a solicitation of an indication of interest to purchase
> any products or services to any persons who are prohibited from receiving
> such information under applicable law. The contents of this communication
> may not be accurate or complete and are subject to change without notice.
> As such, Orchard App, Inc. (including its subsidiaries and affiliates,
> "Orchard") makes no representation regarding the accuracy or completeness
> of the information contained herein. The intended recipient is advised to
> consult its own professional advisors, including those specializing in
> legal, tax and accounting matters. Orchard does not provide legal, tax or
> accounting advice.

Best Regards,
Haosdent Huang

View raw message