spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thodoris Zois <z...@ics.forth.gr>
Subject Re: Spark version for Mesos 0.27.0
Date Thu, 31 May 2018 09:22:45 GMT
Hello,
The reason is that I want to make some tests and use the
oversubscription feature of Mesos along with Spark. Intel and
Mesosphere have built a project, called Serenity that actually measures
the usage slack on each Mesos agent and returns resources to the
cluster. 
Unfortunately, Serenity is not compatible with newer versions of
Mesos...
- Thodoris

On Thu, 2018-05-31 at 11:08 +0200, Szuromi Tamás wrote:
> Hey,
> I'm sure we used Spark 1.6 on Mesos 0.27 as well but at that time we
> used with fine-grained scheduling and not dynamic allocation. Also,
> newer Spark versions should work with older mesos versions like 0.27.
> Why do you have Mesos 0.27 btw?
> 
> cheers,
> Tamas
> Thodoris Zois <zois@ics.forth.gr> ezt írta (időpont: 2018. máj. 31.,
> Cs, 1:56):
> > Hello,
> > I need Mesos 0.27 for specific purposes and unfortunately I can’t
> > use a newer version. Did you find anything? Could it be Spark 1.6? 
> > 
> > Except that, from which version Spark supports dynamic allocation
> > on Mesos? 
> > 
> > - Thodoris
> > On 25 May 2018, at 16:06, Jacek Laskowski <jacek@japila.pl> wrote:
> > 
> > > Hi,
> > > Mesos 0.27.0?! That's been a while. I'd search for the changes to
> > > pom.xml and see when the mesos dependency version changed. That'd
> > > give you the most precise answer. I think it could've been 1.5 or
> > > older.
> > > 
> > > Pozdrawiam,
> > > Jacek Laskowski
> > > ----
> > > https://about.me/JacekLaskowski
> > > Mastering Spark SQL https://bit.ly/mastering-spark-sql
> > > Spark Structured Streaming https://bit.ly/spark-structured-stream
> > > ing
> > > 
> > > Mastering Kafka Streams https://bit.ly/mastering-kafka-streams
> > > Follow me at https://twitter.com/jaceklaskowski
> > > 
> > > 
> > > On Fri, May 25, 2018 at 1:29 PM, Thodoris Zois <zois@ics.forth.gr
> > > > wrote:
> > > > Hello,
> > > > 
> > > > 
> > > > 
> > > > Could you please tell me which version of Spark works with
> > > > Apache Mesos
> > > > 
> > > >  version 0.27.0? (I cannot find anything on docs at github)
> > > > 
> > > > 
> > > > 
> > > > Thank you very much,
> > > > 
> > > > Thodoris Zois
> > > > 
> > > > 
> > > > 
> > > > -------------------------------------------------------------
> > > > --------
> > > > 
> > > > To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
> > > > 
> > > > 
> > > > 
Mime
View raw message