Mailing-List: contact dev-help@ignite.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@ignite.apache.org
MIME-Version: 1.0
In-Reply-To: <f8b2f12f-bd60-972e-3745-4f9be0599923@gmail.com>
References: <f8b2f12f-bd60-972e-3745-4f9be0599923@gmail.com>
From: Valentin Kulichenko <valentin.kulichenko@gmail.com>
Date: Mon, 27 Nov 2017 16:33:51 -0800
Message-ID: <CABuYRcq0+EDtT5zBQWz_0LJTE4rDAnoeGQVG+Y-2O1e9_AcV1w@mail.gmail.com>
Subject: Re: TC issues. IGNITE-3084. Spark Data Frame API
To: =?UTF-8?B?0J3QuNC60L7Qu9Cw0Lkg0JjQttC40LrQvtCy?= <nizhikov.dev@gmail.com>
Cc: dev@ignite.apache.org, avinogradov@gridgain.com
Content-Type: multipart/alternative; boundary="001a11445a8e22e8c5055f002eae"
archived-at: Tue, 28 Nov 2017 00:34:28 -0000

--001a11445a8e22e8c5055f002eae
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Nikolay,

Please see my responses inline.

-Val

On Fri, Nov 24, 2017 at 2:55 AM, =D0=9D=D0=B8=D0=BA=D0=BE=D0=BB=D0=B0=D0=B9=
 =D0=98=D0=B6=D0=B8=D0=BA=D0=BE=D0=B2 <nizhikov.dev@gmail.com>
wrote:

> Hello, guys.
>
> I have some issues on TC with my PR [1] for IGNITE-3084(Spark Data Frame
> API).
> Can you, please, help me:
>
>
> 1. `Ignite RDD spark 2_10` -
>
> Currently this build runs with following profiles:
> `-Plgpl,examples,scala-2.10,-clean-libs,-release` [2]
> That means `scala` profile is activated too for `Ignite RDD spark 2_10`
> Because `scala` activation is done like [3]:
>
> ```
>             <activation>
>                 <property><name>!scala-2.10</name></property>
>             </activation>
> ```
>
> I think it a misconfiguration because scala(2.11) shouldn't be activated
> for 2.10 build.
> Am I miss something?
>
> Can someone edit build property?
>         * Add `-scala` to profiles list
>         * Or add `-Dscala-2.10` to jvm properties to turn off `scala`
> profile in this build.
>
>
Added '-Dscala-2.10' to the build config. Let me know if it helps.


>
> 2. `Ignite RDD` -
>
> Currently this build run on jvm7 [4].
> As I wrote in my previous mail [5] current version of spark(2.2) runs onl=
y
> on jvm8.
>
> Can someone edit build property to run it on jvm8?
>
>
Do you mean that IgniteRDD does not compile on JDK7? If yes, do we know the
reason? I don't think switching it to JDK8 is a solution as it should work
with both.


>
> 3. For now `Ignite RDD` and `Ignite RDD spark 2_10` only runs java tests
> [6] existing in `spark` module.
> There are several existing tests written in scala(i.e. scala-test) ignore=
d
> in TC. IgniteRDDSpec [7] for example.
> Is it turned off by a purpose or I miss something?
> Should we run scala-test for spark and spark_2.10 modules?


I think all tests should be executed on TC. Can you check if they work and
add them to corresponding suites?


>
> [1] https://github.com/apache/ignite/pull/2742
> [2] https://ci.ignite.apache.org/viewLog.html?buildId=3D960220&bui
> ldTypeId=3DIgnite20Tests_IgniteRddSpark210&tab=3DbuildLog&_
> focus=3D379#_state=3D371
> [3] https://github.com/apache/ignite/blob/master/pom.xml#L533
> [4] https://ci.ignite.apache.org/viewLog.html?buildId=3D960221&bui
> ldTypeId=3DIgnite20Tests_IgniteRdd&tab=3DbuildParameters
> [5] http://apache-ignite-developers.2346864.n4.nabble.com/
> Integration-of-Spark-and-Ignite-Prototype-tp22649p23099.html
> [6] https://ci.ignite.apache.org/viewLog.html?buildId=3D960220&bui
> ldTypeId=3DIgnite20Tests_IgniteRddSpark210&tab=3DtestsInfo
> [7] https://github.com/apache/ignite/blob/master/modules/spark/
> src/test/scala/org/apache/ignite/spark/IgniteRDDSpec.scala
>

--001a11445a8e22e8c5055f002eae--