Mailing-List: contact dev-help@ignite.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@ignite.apache.org
Date: Wed, 21 Sep 2016 01:46:59 +0000 (UTC)
From: Roman Shtykh <rshtykh@yahoo.com.INVALID>
Reply-To: Roman Shtykh <rshtykh@yahoo.com>
To: "dev@ignite.apache.org" <dev@ignite.apache.org>
Message-ID: <1106313528.1943536.1474422419187@mail.yahoo.com>
In-Reply-To: <C7327651-19AC-4074-8402-70E76826DEE0@gridgain.com>
References: <DM5PR12MB17211418B7DB5196DEB4333D8AF40@DM5PR12MB1721.namprd12.prod.outlook.com> <CAMbfdjZSSYAenw4h8PmLu5ryQxowrYK=7DnVLZAjHsAz2Jtokw@mail.gmail.com> <DM5PR12MB1721E22AA43077163A9FB48D8AF40@DM5PR12MB1721.namprd12.prod.outlook.com> <CALNFXi2Z9cRrMwio7PKF-2dZRJ8=0ORV=wsgWT7XUR0N0TZdQQ@mail.gmail.com> <CAGt6o8kRFVHQ=naDeNQXrNHnyOeKHHtpsbuaYVMVw186dDAnRg@mail.gmail.com> <C7327651-19AC-4074-8402-70E76826DEE0@gridgain.com>
Subject: Re: Support for Spark 2.0
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----=_Part_1943535_794521008.1474422419177"
archived-at: Wed, 21 Sep 2016 01:47:23 -0000

------=_Part_1943535_794521008.1474422419177
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

There is a bunch of requests from users for this upgrade.
https://issues.apache.org/jira/browse/IGNITE-3822=C2=A0and related.

-Roman
=20

    On Wednesday, September 21, 2016 5:14 AM, Denis Magda <dmagda@gridgain.=
com> wrote:
=20

 Is this the only issue we have with Spark 2.0?
https://issues.apache.org/jira/browse/IGNITE-3596 <https://issues.apache.or=
g/jira/browse/IGNITE-3596>

=E2=80=94
Denis

> On Sep 20, 2016, at 11:54 AM, Vladimir Ozerov <vozerov@gridgain.com> wrot=
e:
>=20
> I think we can expect Spark 2.0 support to be added in one of the nearest
> releases (not sure if it will be 1.8 or 2.0).
>=20
> On Mon, Sep 19, 2016 at 10:20 PM, Richard Siebeling <rsiebeling@gmail.com=
>
> wrote:
>=20
>> I'd wish someone could fix this, unfortunately I can't do anything about=
 it
>> in the near future...
>>=20
>> On Mon, Sep 19, 2016 at 3:20 PM, Phadnis, Varun <phadnis@sky.optymyze.co=
m>
>> wrote:
>>=20
>>> Hello,
>>>=20
>>> Just to be clear, I am not explicitly building the Hadoop edition. I ge=
t
>>> the described error when I simply specify the Spark version:
>>>=20
>>>=C2=A0 mvn clean package -Dspark.version=3D2.0.0 -DskipTests
>>>=20
>>> Thanks for the quick response!
>>>=20
>>>=20
>>> -----Original Message-----
>>> From: Sergey Kozlov [mailto:skozlov@gridgain.com]
>>> Sent: 19 September 2016 05:56
>>> To: dev@ignite.apache.org
>>> Subject: Re: Support for Spark 2.0
>>>=20
>>> Hi
>>>=20
>>> It's a known issue IGNITE-3596 Hadoop edition can't be compiled against
>>> spark 2.0.0 <https://issues.apache.org/jira/browse/IGNITE-3596>
>>>=20
>>> Unfortunately there's no progress yet
>>>=20
>>> On Mon, Sep 19, 2016 at 1:36 PM, Phadnis, Varun <
>> phadnis@sky.optymyze.com>
>>> wrote:
>>>=20
>>>> Hello,
>>>>=20
>>>> Can someone please tell me when is the support for Spark 2.0 planned?
>>>>=20
>>>> Currently Ignite cannot be built for Spark 2.0. Attempting this yields
>>>> the following error :
>>>>=20
>>>>=20
>>>> ----------------------------------------------------------------------
>>>> --
>>>>=20
>>>> [INFO] Building ignite-spark 1.8.0-SNAPSHOT [INFO]
>>>>=20
>>>> ----------------------------------------------------------------------
>>>> --
>>>>=20
>>>> [INFO]
>>>>=20
>>>> [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ ignite-spark
>>>> --- [INFO] Deleting /home/spark/code/ignite/modules/spark/target
>>>>=20
>>>> [INFO]
>>>>=20
>>>> [INFO] --- flatten-maven-plugin:1.0.0-beta-3:clean
>>>> (flatten.clean.before) @ ignite-spark --- [INFO] Deleting
>>>> /home/spark/code/ignite/ modules/spark/pom-installed.xml
>>>>=20
>>>> [INFO]
>>>>=20
>>>> [INFO] --- maven-enforcer-plugin:1.4:enforce (default) @ ignite-spark
>>>> --- [INFO] [INFO] --- maven-remote-resources-plugin:1.5:process
>>>> (default) @ ignite-spark --- [INFO] [INFO] ---
>>>> maven-remote-resources-plugin:1.5:process
>>>> (ignite-dependencies) @ ignite-spark --- [INFO] [INFO] ---
>>>> maven-resources-plugin:2.6:resources (default-resources) @
>>>> ignite-spark
>>>> --- [INFO] Using 'UTF-8' encoding to copy filtered resources.
>>>>=20
>>>> [INFO] skip non existing resourceDirectory /home/spark/code/ignite/
>>>> modules/spark/src/main/resources
>>>>=20
>>>> [INFO] Copying 4 resources
>>>>=20
>>>> [INFO] Copying 4 resources
>>>>=20
>>>> [INFO]
>>>>=20
>>>> [INFO] --- flatten-maven-plugin:1.0.0-beta-3:flatten (flatten) @
>>>> ignite-spark --- [INFO] Generating flattened POM of project
>>>> org.apache.ignite:ignite-spark:jar:1.8.0-SNAPSHOT...
>>>>=20
>>>> [WARNING] Ignoring multiple XML header comment!
>>>>=20
>>>> [INFO]
>>>>=20
>>>> [INFO] --- scala-maven-plugin:3.2.0:add-source (scala-compile-first) @
>>>> ignite-spark --- [INFO] Add Source directory:
>>>>=20
>>>> /home/spark/code/ignite/modules/spark/src/main/scala
>>>>=20
>>>> [INFO] Add Test Source directory:
>>>>=20
>>>> /home/spark/code/ignite/modules/spark/src/test/scala
>>>>=20
>>>> [INFO]
>>>>=20
>>>> [INFO] --- scala-maven-plugin:3.2.0:compile (scala-compile-first) @
>>>> ignite-spark --- [WARNING]=C2=A0 Expected all dependencies to require =
Scala
>>>> version: 2.11.7 [WARNING]
>>>> org.apache.ignite:ignite-spark:1.8.0-SNAPSHOT
>>>> requires scala
>>>>=20
>>>> version: 2.11.7
>>>>=20
>>>> [WARNING]=C2=A0 com.twitter:chill_2.11:0.8.0 requires scala version: 2=
.11.7
>>>> [WARNING]=C2=A0 org.apache.spark:spark-core_2.11:2.0.0 requires scala
>>> version:
>>>>=20
>>>> 2.11.8
>>>>=20
>>>> [WARNING] Multiple versions of scala libraries detected!
>>>>=20
>>>> [INFO] /home/spark/code/ignite/modules/spark/src/main/scala:-1: info:
>>>>=20
>>>> compiling
>>>>=20
>>>> [INFO] Compiling 8 source files to
>>>>=20
>>>> /home/spark/code/ignite/modules/spark/target/classes at 1474028222935
>>>> [ERROR]
>>>>=20
>>>> /home/spark/code/ignite/modules/spark/src/main/scala/
>>>> org/apache/ignite/spark/IgniteContext.scala:25:
>>>>=20
>>>> error: object Logging is not a member of package org.apache.spark
>>>> [ERROR] import org.apache.spark.{Logging, SparkContext}
>>>>=20
>>>> [ERROR]=C2=A0 =C2=A0 =C2=A0 =C2=A0 ^
>>>>=20
>>>> [ERROR]
>>>>=20
>>>> /home/spark/code/ignite/modules/spark/src/main/scala/
>>>> org/apache/ignite/spark/IgniteContext.scala:37:
>>>>=20
>>>> error: not found: type Logging
>>>>=20
>>>> [ERROR]=C2=A0 =C2=A0 ) extends Serializable with Logging {
>>>>=20
>>>> [ERROR]=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 ^
>>>>=20
>>>> [ERROR]
>>>>=20
>>>> /home/spark/code/ignite/modules/spark/src/main/scala/
>>>> org/apache/ignite/spark/IgniteContext.scala:50:
>>>>=20
>>>> error: not found: value logInfo
>>>>=20
>>>> [ERROR]=C2=A0 =C2=A0 =C2=A0 =C2=A0 logInfo("Will start Ignite nodes on=
 " + workers + "
>>>>=20
>>>> workers")
>>>>=20
>>>> [ERROR]=C2=A0 =C2=A0 =C2=A0 =C2=A0 ^
>>>>=20
>>>> [ERROR]
>>>>=20
>>>> /home/spark/code/ignite/modules/spark/src/main/scala/
>>>> org/apache/ignite/spark/IgniteContext.scala:129:
>>>>=20
>>>> error: not found: value logInfo
>>>>=20
>>>> [ERROR]=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 logInfo("Setting IGNI=
TE_HOME from driver not as it
>> is
>>>>=20
>>>> not available on this worker: " + igniteHome)
>>>>=20
>>>> [ERROR]=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 ^
>>>>=20
>>>> [ERROR]
>>>>=20
>>>> /home/spark/code/ignite/modules/spark/src/main/scala/
>>>> org/apache/ignite/spark/IgniteContext.scala:146:
>>>>=20
>>>> error: not found: value logError
>>>>=20
>>>> [ERROR]=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 logErro=
r("Failed to start Ignite.", e)
>>>>=20
>>>> [INFO]=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 ^
>>>>=20
>>>> [ERROR]
>>>>=20
>>>> /home/spark/code/ignite/modules/spark/src/main/scala/
>>>> org/apache/ignite/spark/IgniteContext.scala:164:
>>>>=20
>>>> error: not found: value logInfo
>>>>=20
>>>> [ERROR]=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 logInfo=
("Will stop Ignite nodes on " + workers
>> +
>>> "
>>>>=20
>>>> workers")
>>>>=20
>>>> [ERROR]=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 ^
>>>>=20
>>>> [ERROR] 6 errors found
>>>>=20
>>>> [INFO]
>>>>=20
>>>> ----------------------------------------------------------------------
>>>> --
>>>>=20
>>>> Thanks!
>>>>=20
>>>=20
>>>=20
>>>=20
>>> --
>>> Sergey Kozlov
>>> GridGain Systems
>>> www.gridgain.com
>>>=20
>>=20


  =20
------=_Part_1943535_794521008.1474422419177--