ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Valentin Kulichenko <valentin.kuliche...@gmail.com>
Subject Re: TC issues. IGNITE-3084. Spark Data Frame API
Date Tue, 28 Nov 2017 00:33:51 GMT
Hi Nikolay,

Please see my responses inline.

-Val

On Fri, Nov 24, 2017 at 2:55 AM, Николай Ижиков <nizhikov.dev@gmail.com>
wrote:

> Hello, guys.
>
> I have some issues on TC with my PR [1] for IGNITE-3084(Spark Data Frame
> API).
> Can you, please, help me:
>
>
> 1. `Ignite RDD spark 2_10` -
>
> Currently this build runs with following profiles:
> `-Plgpl,examples,scala-2.10,-clean-libs,-release` [2]
> That means `scala` profile is activated too for `Ignite RDD spark 2_10`
> Because `scala` activation is done like [3]:
>
> ```
>             <activation>
>                 <property><name>!scala-2.10</name></property>
>             </activation>
> ```
>
> I think it a misconfiguration because scala(2.11) shouldn't be activated
> for 2.10 build.
> Am I miss something?
>
> Can someone edit build property?
>         * Add `-scala` to profiles list
>         * Or add `-Dscala-2.10` to jvm properties to turn off `scala`
> profile in this build.
>
>
Added '-Dscala-2.10' to the build config. Let me know if it helps.


>
> 2. `Ignite RDD` -
>
> Currently this build run on jvm7 [4].
> As I wrote in my previous mail [5] current version of spark(2.2) runs only
> on jvm8.
>
> Can someone edit build property to run it on jvm8?
>
>
Do you mean that IgniteRDD does not compile on JDK7? If yes, do we know the
reason? I don't think switching it to JDK8 is a solution as it should work
with both.


>
> 3. For now `Ignite RDD` and `Ignite RDD spark 2_10` only runs java tests
> [6] existing in `spark` module.
> There are several existing tests written in scala(i.e. scala-test) ignored
> in TC. IgniteRDDSpec [7] for example.
> Is it turned off by a purpose or I miss something?
> Should we run scala-test for spark and spark_2.10 modules?


I think all tests should be executed on TC. Can you check if they work and
add them to corresponding suites?


>
> [1] https://github.com/apache/ignite/pull/2742
> [2] https://ci.ignite.apache.org/viewLog.html?buildId=960220&bui
> ldTypeId=Ignite20Tests_IgniteRddSpark210&tab=buildLog&_
> focus=379#_state=371
> [3] https://github.com/apache/ignite/blob/master/pom.xml#L533
> [4] https://ci.ignite.apache.org/viewLog.html?buildId=960221&bui
> ldTypeId=Ignite20Tests_IgniteRdd&tab=buildParameters
> [5] http://apache-ignite-developers.2346864.n4.nabble.com/
> Integration-of-Spark-and-Ignite-Prototype-tp22649p23099.html
> [6] https://ci.ignite.apache.org/viewLog.html?buildId=960220&bui
> ldTypeId=Ignite20Tests_IgniteRddSpark210&tab=testsInfo
> [7] https://github.com/apache/ignite/blob/master/modules/spark/
> src/test/scala/org/apache/ignite/spark/IgniteRDDSpec.scala
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message