spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron Ilovici <ailov...@wayfair.com>
Subject Re: Error in running JavaALSExample example from spark examples
Date Fri, 22 Jul 2016 18:01:33 GMT
Your error stems from spark.ml, and in your pom mllib is the only dependency that is 2.10.
Is there a reason for this? IE, you tell maven mllib 2.10 is provided at runtime. Is 2.10
on the machine, or is 2.11?

-Aaron

From: VG <vlinked@gmail.com>
Date: Friday, July 22, 2016 at 1:49 PM
To: Sean Owen <sowen@cloudera.com>
Cc: User <user@spark.apache.org>
Subject: Re: Error in running JavaALSExample example from spark examples

Sean,

I am only using the maven dependencies for spark in my pom file.
I don't have anything else. I guess maven dependency should resolve to the correct scala version
.. isn;t it ? Any ideas.

<dependency><!-- Spark -->
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.11</artifactId>
<version>2.0.0-preview</version>
<scope>provided</scope>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.spark/spark-sql_2.11 -->
<dependency><!-- Spark SQL -->
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_2.11</artifactId>
<version>2.0.0-preview</version>
<scope>provided</scope>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.spark/spark-streaming_2.11 -->
<dependency><!-- Spark Streaming -->
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming_2.11</artifactId>
<version>2.0.0-preview</version>
<scope>provided</scope>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.spark/spark-mllib_2.10 -->
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-mllib_2.10</artifactId>
<version>2.0.0-preview</version>
<scope>provided</scope>
</dependency>



On Fri, Jul 22, 2016 at 11:16 PM, Sean Owen <sowen@cloudera.com<mailto:sowen@cloudera.com>>
wrote:
-dev
Looks like you are mismatching the version of Spark you deploy on at
runtime then. Sounds like it was built for Scala 2.10

On Fri, Jul 22, 2016 at 6:43 PM, VG <vlinked@gmail.com<mailto:vlinked@gmail.com>>
wrote:
> Using 2.0.0-preview using maven
> So all dependencies should be correct I guess
>
> <dependency><!-- Spark -->
> <groupId>org.apache.spark</groupId>
> <artifactId>spark-core_2.11</artifactId>
> <version>2.0.0-preview</version>
> <scope>provided</scope>
> </dependency>
>
> I see in maven dependencies that this brings in
> scala-reflect-2.11.4
> scala-compiler-2.11.0
>
> and so on
>
>
>
> On Fri, Jul 22, 2016 at 11:04 PM, Aaron Ilovici <ailovici@wayfair.com<mailto:ailovici@wayfair.com>>
> wrote:
>>
>> What version of Spark/Scala are you running?
>>
>>
>>
>> -Aaron
>
>

Mime
View raw message