kylin-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ShaoFeng Shi <shaofeng...@apache.org>
Subject Re: Some questions about Kylin2.0
Date Mon, 19 Jun 2017 16:28:20 GMT
For Q1, yes Kylin is running with Spark 1.6.3 now; Will upgrade to Spark
2.x in future releases.

For Q2, if you don't use Spark cubing, you can modify the start bash
scripts to disable the checking for Spark.

For Q3, can the host name "cdh5" be resolved on this machine?

2017-06-16 20:55 GMT+08:00 Alberto Ramón <a.ramonportoles@gmail.com>:

> About Q2:
> I'm agree with you, I think is a *issue*
> To start Kylin you must check exists one Source, one Engine and one Storage
> system
> (for example, is not necesary have Hive and Kafka)
>
> Example Spark
> <https://github.com/apache/kylin/blob/c38def7b53dae81f9fde0520b1fb27
> 0804dde728/build/bin/find-spark-dependency.sh#L38>
>
> On 16 June 2017 at 13:17, skyyws <skyyws@163.com> wrote:
>
> > For Q3, you can try to make soft links for both hdfs-site.xml and
> > mapred-site.xml.
> >
> > 2017-06-16
> >
> > skyyws
> >
> >
> >
> > 发件人:"lxw" <lxw1234@qq.com>
> > 发送时间:2017-06-13 11:41
> > 主题:Some questions about Kylin2.0
> > 收件人:"dev"<dev@kylin.apache.org>,"user"<user@kylin.apache.org>
> > 抄送:
> >
> > Hi,All :
> >
> >    I have some questions about Kylin2.0, and my environment:
> >     hadoop-2.6.0-cdh5.8.3
> >     hbase-1.2.0-cdh5.8.3
> >     apache-kylin-2.0.0-bin-cdh57
> >     spark-2.1.0-bin-hadoop2.6
> >
> >
> > Q1: Kylin2.0 not support Spark2.0?
> >
> >      find-spark-dependency.sh:
> >      spark_dependency=`find -L $spark_home -name
> > 'spark-assembly-[a-z0-9A-Z\.-]*.jar' ....
> >
> >
> > Q2: I want to use Kylin2.0 without Spark Cubing, but failed.
> >
> >
> >      kylin.sh:
> >      function retrieveDependency() {
> >          #retrive $hive_dependency and $hbase_dependency
> >          source ${dir}/find-hive-dependency.sh
> >          source ${dir}/find-hbase-dependency.sh
> >          source ${dir}/find-hadoop-conf-dir.sh
> >          source ${dir}/find-kafka-dependency.sh
> >          source ${dir}/find-spark-dependency.sh
> >
> >
> >      If not found spark dependencies, Kylin can not start :
> >
> >      [hadoop@hadoop10 bin]$ ./kylin.sh start
> >      Retrieving hadoop conf dir...
> >      KYLIN_HOME is set to /home/hadoop/bigdata/kylin/current
> >      Retrieving hive dependency...
> >      Retrieving hbase dependency...
> >      Retrieving hadoop conf dir...
> >      Retrieving kafka dependency...
> >      Retrieving Spark dependency...
> >      spark assembly lib not found.
> >
> >
> >      after modify kylin.sh “####source ${dir}/find-spark-dependency.sh”,
> > Kylin start success ..
> >
> >
> > Q3: Abount kylin_hadoop_conf_dir ?
> >
> >      I make some soft link under $KYLIN_HOME/hadoop-conf
> > (core-site.xml、yarn-site.xml、hbase-site.xml、hive-site.xml),
> >      and set "kylin.env.hadoop-conf-dir=/home/bigdata/kylin/current/
> hadoop-conf",
> > when I execute ./check-env.sh,
> >
> >
> >      [hadoop@hadoop10 bin]$ ./check-env.sh
> >      Retrieving hadoop conf dir...
> >     /home/bigdata/kylin/current/hadoop-conf is override as the
> > kylin_hadoop_conf_dir
> >     KYLIN_HOME is set to /home/hadoop/bigdata/kylin/current
> >     -mkdir: java.net.UnknownHostException: cdh5
> >     Usage: hadoop fs [generic options] -mkdir [-p] <path> ...
> >     Failed to create /kylin20. Please make sure the user has right to
> > access /kylin20
> >
> >
> >     My HDFS with HA, fs.defaultFS is "cdh5",when I don't set
> > "kylin.env.hadoop-conf-dir", and use HADOOP_CONF_DIR, HIVE_CONF,
> > HBASE_CONF_DIR from envionment variables (/etc/profile), it was correct.
> >
> >
> > Best Regards!
> > lxw
> >
>



-- 
Best regards,

Shaofeng Shi 史少锋

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message