carbondata-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Liang Chen (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CARBONDATA-562) Supporting Spark 1.6.3 integration in CarbonData
Date Sun, 22 Jan 2017 03:45:27 GMT

     [ https://issues.apache.org/jira/browse/CARBONDATA-562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Liang Chen updated CARBONDATA-562:
----------------------------------
    Issue Type: New Feature  (was: Bug)
       Summary: Supporting Spark 1.6.3 integration in CarbonData  (was: Carbon Context initialization
is failed with spark 1.6.3 )

> Supporting Spark 1.6.3 integration in CarbonData
> ------------------------------------------------
>
>                 Key: CARBONDATA-562
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-562
>             Project: CarbonData
>          Issue Type: New Feature
>          Components: core
>    Affects Versions: 0.2.0-incubating
>            Reporter: Babulal
>            Assignee: Naresh P R
>             Fix For: 1.0.0-incubating
>
>          Time Spent: 40m
>  Remaining Estimate: 0h
>
> Carbon is compiled with below command 
> mvn clean -P spark-1.6,provided -Dspark.version=1.6.3 -DskipTests package
> start carbon context in yarn client mode
>  bin/spark-shell --master yarn-client --executor-memory 2G  --executor-cores 2 --driver-memory
2G  --num-executors 3
>   ____              __
>      / __/__  ___ _____/ /__
>     _\ \/ _ \/ _ `/ __/  '_/
>    /___/ .__/\_,_/_/ /_/\_\   version 1.6.3
>       /_/
> Using Scala version 2.10.5 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_111)
> Type in expressions to have them evaluated.
> scala> import org.apache.spark.sql.CarbonContext;
> import org.apache.spark.sql.CarbonContext
> scala> val cc=new CarbonContext(sc,"hdfs://hacluser/opt/CarbonStore");
> e of dependencies)
> scala.tools.reflect.ToolBoxError: reflective compilation has failed:
> not enough arguments for constructor Optimizer: (conf: org.apache.spark.sql.catalyst.CatalystConf)org.apache.spark.sql.catalyst.optimizer.Optimizer.
> Unspecified value parameter conf.
>         at scala.tools.reflect.ToolBoxFactory$ToolBoxImpl$ToolBoxGlobal.throwIfErrors(ToolBoxFactory.scala:314)
> at org.apache.spark.sql.CodeGenerateFactory.<init>(CodeGenerateFactory.scala:32)
>         at org.apache.spark.sql.CodeGenerateFactory$.init(CodeGenerateFactory.scala:67)
>         at org.apache.spark.sql.CarbonContext.<init>(CarbonContext.scala:58)
>         at org.apache.spark.sql.CarbonContext.<init>(CarbonContext.scala:52)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message