ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Petr Ivanov <mr.wei...@gmail.com>
Subject Re: Spark examples on TC
Date Wed, 25 Sep 2019 08:15:12 GMT
I can help with JDK8 enforcement — just give me the links to target suites when agreed.


> On 25 Sep 2019, at 01:53, Denis Magda <dmagda@apache.org> wrote:
> 
> Nikolay,
> 
> Thanks for the details. How about enforcing JDK 8 for the Spark integration
> in general and run its tests on the VM only? We'll do this until Spark
> community fully support JDK 11+.
> 
> As for Spark 2.4.0 module [1], how about releasing Ignite 2.7.7 with Spark
> related changes only? As long as Ignite modularization is a long process,
> we can follow your advice and create two more modules ignite_spark_24 and
> ignite_spark_25 . Plus Alexey is working on extra Spark improvements.
> 
> [1] https://issues.apache.org/jira/browse/IGNITE-12054
> -
> Denis
> 
> 
> On Thu, Sep 19, 2019 at 11:58 PM Nikolay Izhikov <nizhikov@apache.org>
> wrote:
> 
>> Hello, Denis.
>> 
>> Spark uses Scala 2.11 [1]
>> Scala 2.11 compatible with the jdk version described on [2].
>> 
>> Few notes from the link:
>> 
>>> We recommend using Java 8 for compiling Scala code
>>> JDK 11 compatibility notes
>>> As of Scala 2.13.0, 2.12.8 and 2.11.12, JDK 11 support is incomplete
>> 
>> I also got several issues with the Spark itself when trying to run it on
>> the Java 11.
>> Please, take a look at [3], [4]
>> 
>> All in all, with the spark 2.3 we should stay on java8 to get things
>> worked.
>> 
>> [1] http://spark.apache.org/docs/2.4.0/
>> [2] https://docs.scala-lang.org/overviews/jdk-compatibility/overview.html
>> [3]
>> 
>> https://stackoverflow.com/questions/49961991/java-lang-illegalargumentexception-at-org-apache-xbean-asm5-classreader-initu?noredirect=1&lq=1
>> [4]
>> 
>> https://stackoverflow.com/questions/51352591/spark-java-illegalargumentexception-at-org-apache-xbean-asm5-classreader
>> 
>> 
>> чт, 19 сент. 2019 г. в 21:39, Denis Magda <dmagda@apache.org>:
>> 
>>> Nikolay,
>>> 
>>> What needs to be done to ensure Java 9++ works with Spark Examples? Do we
>>> need to create another Spark module or drop some scala version?
>>> 
>>> -
>>> Denis
>>> 
>>> 
>>> On Thu, Sep 19, 2019 at 9:23 AM Nikolay Izhikov <nizhikov@apache.org>
>>> wrote:
>>> 
>>>> Hello, Igniters.
>>>> 
>>>> Examples suite on TC doesn't work correctly under java9+. [1], [2]
>>>> 
>>>> The reason is the Spark examples.
>>>> 
>>>> Spark 2.3(our current version) work with scala 2.11 and scala 2.11
>>> support
>>>> of java9+ is "incomplete".
>>>> 
>>>> Can we stick example suite to jdk8 only?
>>>> 
>>>> [1]
>>>> 
>>> 
>> https://ci.ignite.apache.org/viewLog.html?buildId=4610507&buildTypeId=IgniteTests24Java8_Examples
>>>> [2]
>>>> 
>>> 
>> https://ci.ignite.apache.org/viewLog.html?buildId=4609013&buildTypeId=IgniteTests24Java8_Examples
>>>> 
>>> 
>> 


Mime
View raw message