carbondata-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bill Speirs <wspe...@datto.com>
Subject Spark 2.1.1 with CarbonData 1.1.0
Date Wed, 24 May 2017 18:31:13 GMT
I'm trying to follow the directions for using Spark 2.1.1 with
CarbonData 1.1.0 found here:
http://carbondata.apache.org/quick-start-guide.html

I compiled CarbonData using: mvn -DskipTests -Pspark-2.1
-Dspark.version=2.1.1 clean package

I ran Spark with: ./bin/spark-shell --jars
../carbondata-parent-1.1.0/assembly/target/scala-2.11/carbondata_2.11-1.1.0-shade-hadoop2.2.0.jar

I imported the two classes (not sure if the second is a typo with the
underline):

scala> import org.apache.spark.sql.SparkSession
import org.apache.spark.sql.SparkSession

scala> import org.apache.spark.sql.CarbonSession._
import org.apache.spark.sql.CarbonSession._

Then tried to create the carbon variable:

scala> val carbon =
SparkSession.builder().config(sc.getConf).getOrCreateCarbonContext("hdfs://localhost:9000/carbondata")
<console>:28: error: value getOrCreateCarbonContext is not a member of
org.apache.spark.sql.SparkSession.Builder
       val carbon =
SparkSession.builder().config(sc.getConf).getOrCreateCarbonContext("hdfs://localhost:9000/carbondata")

Thoughts on what I'm doing wrong here?

Bill-

Mime
View raw message