kylin-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "朱真龙" <183920...@qq.com>
Subject 回复: mismatch hdfs addr error when using kylin2.1 on two hadoop cluster
Date Mon, 25 Dec 2017 03:47:09 GMT
thank you for your response. I am really glad that you are patient enough to read my email
which maybe have many solecisms on it .


I have configured right ,beause some model works well between two hadoop cluster .  I used
 core-site.xml copied from hive cluster's hdfs ,and in kylin.properties
I configured kylin.hbase.cluster.hdfs.config.file to hbase's hdfs-site.xml, and configured
kylin.storage.hbase.cluster-fs to hbase's hdfs cluster addr. 







I  used kylin 1.6 between two hadoop cluster for about one month already.   and  changed to
kylin2.1 last week.



------------------ 原始邮件 ------------------
发件人: "ShaoFeng Shi";<shaofengshi@apache.org>;
发送时间: 2017年12月22日(星期五) 晚上9:15
收件人: "user"<user@kylin.apache.org>;

主题: Re: mismatch hdfs addr error when using kylin2.1 on two hadoop cluster



When configure Kylin with two cluster (one for hive/mapreduce, the other for hbase), need:

1. in kylin node, the core-site.xml uses Hive cluster's HDFS as default FS.
2. in kylin.properties, configure "kylin.storage.hbase.cluster-fs" to the HDFS of HBase cluster


In your case, I guess you didn't configure 2); please double check


2017-12-22 16:10 GMT+08:00 jxs <jxskiss@126.com>:
I also deploy kylin on two small EMR clusters for resource isolation, I used MySQL for Hive
metastore and shared the same hive configuration between the two clusters.
You may try this.








在2017年12月22 13时08分, "朱真龙"<183920368@qq.com>写道:

    thank you for your attention first,I am a chinses kylin user that always read english
documents but write little, and i know most of kylin developer are chinese 。so ,if you
don't know what i mean,i will describe again in chinese。


     I am using kylin2.1.0 between two hadoop cluster (all configured HA) and hadoop have
same version(2.7.1).  by now runing well with sample model  , but  not good in my own model
which has many  columns encoding with dict。like this:



when  building cube , got error like this:



after looked into source code that the error metioned, find that the dictionary got hdfs path
from hive table desc , and when checking before load these hfiles into hbase ,found hfiles
and hbase table are on different hdfs cluster .


apache-kylin-2.1.0-src\apache-kylin-2.1.0\core-cube\src\main\java\org\apache\kylin\cube\CubeManager





so, could you tell me how could i do on this case ?
















-- 
Best regards,

Shaofeng Shi 史少锋
Mime
View raw message