spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matei Zaharia <matei.zaha...@gmail.com>
Subject Re: Scala 2.10?
Date Thu, 19 Sep 2013 18:54:47 GMT
Hey Paul,

2.10 is definitely on our roadmap, and you can actually find a scala-2.10 branch in the repo
that has a bunch of the changes done. However, as Mark said, it won't be in 0.8 mostly because
we've had a lot of other changes in that release. One challenge for us is that we also make
some nontrivial changes to the Scala interpreter to support Spark's interpreter, which require
porting to each Scala release. In addition we had some issues with Akka compatibility -- Akka
2.0.x doesn't work on Scala 2.10, but Akka 2.1.x creates some extra non-daemon threads that
mess up our user programs. But we are working on fixing these and we are very excited about
the features in Scala 2.10 and 2.11. Stuff like the fast serialization library will really
help Spark.

Matei

On Sep 19, 2013, at 10:55 AM, "andy.petrella@gmail.com" <andy.petrella@gmail.com> wrote:

> +1 for the iteratee support.
> 
> ----- Reply message -----
> De : "Mark Hamstra" <mark@clearstorydata.com>
> Pour : <user@spark.incubator.apache.org>
> Objet : Scala 2.10?
> Date : jeu., sept. 19, 2013 19:37
> 
> 
> Can you motivate yourself enough, Paul, to enter into JIRA your feature requests for
Streaming and Spark?   That way we can avoid a bad combination of lazy and forgetful.
> 
> 
> On Thu, Sep 19, 2013 at 10:14 AM, Paul Snively <psnively@icloud.com> wrote:
> Hi team,
> 
> Please tell me the 0.8.0 release supports Scala 2.10. :-)
> 
> The BDAS stack is very exciting! Unfortunately, lagging the development of the language
itself (2.11 is at Milestone 4!) is increasingly problematic.
> 
> I would also like to encourage the Spark Streaming developers to head in the direction
of making DStream conform to the Scala and Java collections APIs. A possible model to emulate
here is Kafka's KafkaStream, which satisfies both Scala's and Java's Iterable interfaces,
making their effective use considerably more straightforward than, frankly, DStreams are at
the moment. Please also consider OOB support for Play! iteratees, which, as of the Play! 2.2.x
releases, are a nicely-separate module from the rest of Play!
> 
> Lest this sound like all complaints, I'm asking for this because I expect to use BDAS
for many years due to the lack of realistic competition (I've already explained in-house why
Summingbird isn't it) :-) and because I'm lazy. Please keep up the excellent work, and I hope
to make one of the Meetups next time I'm in the neighborhood.
> 
> Best regards,
> Paul
> 


Mime
View raw message