reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Saikat Kanjilal <sxk1...@gmail.com>
Subject Re: reef-runtime-spark--moving to Java1.8 [Discuss]
Date Wed, 11 Oct 2017 00:32:46 GMT
Hey Sergiy,
The features in spark API become easier to use if reef-runtime-spark moves to Java1.8 but
this is not strictly required at the moment.  Case in point , when using a flatmap operation
with Java1.8 it helps to be able to use lamdas from a code maintenance and readability perspective
, to workaround this I implemented anonymous inner classes instead using the FlatMapFunction
which compiles and works fine as a substitute.

It’s good to know that REEF works on 1.8, however the maven compilation flags point to compiling
for 1.7 which prevents a the use of lambdas.

Regards

Sent from my iPhone

> On Oct 10, 2017, at 5:22 PM, Sergiy Matusevych <sergiy.matusevych@gmail.com> wrote:
> 
> Hi Saikat,
> 
> REEF can run on JVM 1.8 just fine - we just don't use any Java 1.8 language
> features at the moment. (Note that we don't prevent REEF applications from
> using Java8!)
> 
> Are there any features in Spark API that might require reef-runtime-spark
> use Java8? If not, I would suggest sticking to Java7 for the time being and
> properly redesign REEF API to take full advantage of Java8 (say, using
> lambdas instead of EventHandler classes) later. I really don't want to mix
> our Spark integration effort with any other sub-projects.
> 
> What do you guys think?
> 
> Cheers,
> Sergiy.
> 
> 
>> On Tue, Oct 10, 2017 at 3:25 PM, Saikat Kanjilal <sxk1969@gmail.com> wrote:
>> 
>> Hello Reef community,
>> I'm doing some research on how to test the reef-runtime-spark end to end
>> and I wanted to bring up a few things:
>> 
>> 1) It seems that the spark community is going to leave Java1.7 behind and
>> has already done so in most versions of spark (post 2.0.0 and in some cases
>> even earlier versions are compiling with Java8)-->see here:
>> https://spark.apache.org/docs/2.1.0/
>> 2) As I mentioned before if we really want to take advantage of some of the
>> paradigms in spark moving to Java1.8 makes that path much easier
>> 3) Do we currently have clients who are using Java 1.8?
>> 
>> 
>> I would love to spearhead an effort to do this port sooner than later maybe
>> as part of developing the reef-runtime-spark component as a springboard to
>> do this.
>> 
>> What are your thoughts one way or another, would love a discussion on this.
>> 
>> Thanks in advance.
>> 

Mime
View raw message