ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Николай Ижиков" <nizhikov....@gmail.com>
Subject TC issues. IGNITE-3084. Spark Data Frame API
Date Fri, 24 Nov 2017 10:55:34 GMT
Hello, guys.

I have some issues on TC with my PR [1] for IGNITE-3084(Spark Data Frame API).
Can you, please, help me:


1. `Ignite RDD spark 2_10` -

Currently this build runs with following profiles: `-Plgpl,examples,scala-2.10,-clean-libs,-release`
[2]
That means `scala` profile is activated too for `Ignite RDD spark 2_10`
Because `scala` activation is done like [3]:

```
             <activation>
                 <property><name>!scala-2.10</name></property>
             </activation>
```

I think it a misconfiguration because scala(2.11) shouldn't be activated for 2.10 build.
Am I miss something?

Can someone edit build property?
	* Add `-scala` to profiles list
	* Or add `-Dscala-2.10` to jvm properties to turn off `scala` profile in this build.


2. `Ignite RDD` -

Currently this build run on jvm7 [4].
As I wrote in my previous mail [5] current version of spark(2.2) runs only on jvm8.

Can someone edit build property to run it on jvm8?


3. For now `Ignite RDD` and `Ignite RDD spark 2_10` only runs java tests [6] existing in `spark`
module.
There are several existing tests written in scala(i.e. scala-test) ignored in TC. IgniteRDDSpec
[7] for example.
Is it turned off by a purpose or I miss something?
Should we run scala-test for spark and spark_2.10 modules?


[1] https://github.com/apache/ignite/pull/2742
[2] https://ci.ignite.apache.org/viewLog.html?buildId=960220&buildTypeId=Ignite20Tests_IgniteRddSpark210&tab=buildLog&_focus=379#_state=371
[3] https://github.com/apache/ignite/blob/master/pom.xml#L533
[4] https://ci.ignite.apache.org/viewLog.html?buildId=960221&buildTypeId=Ignite20Tests_IgniteRdd&tab=buildParameters
[5] http://apache-ignite-developers.2346864.n4.nabble.com/Integration-of-Spark-and-Ignite-Prototype-tp22649p23099.html
[6] https://ci.ignite.apache.org/viewLog.html?buildId=960220&buildTypeId=Ignite20Tests_IgniteRddSpark210&tab=testsInfo
[7] https://github.com/apache/ignite/blob/master/modules/spark/src/test/scala/org/apache/ignite/spark/IgniteRDDSpec.scala

Mime
View raw message