kylin-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "朱真龙" <>
Subject 回复: mismatch hdfs addr error when using kylin2.1 on two hadoop cluster
Date Mon, 25 Dec 2017 03:47:09 GMT
thank you for your response. I am really glad that you are patient enough to read my email
which maybe have many solecisms on it .

I have configured right ,beause some model works well between two hadoop cluster .  I used
 core-site.xml copied from hive cluster's hdfs ,and in
I configured kylin.hbase.cluster.hdfs.config.file to hbase's hdfs-site.xml, and configured to hbase's hdfs cluster addr. 

I  used kylin 1.6 between two hadoop cluster for about one month already.   and  changed to
kylin2.1 last week.

------------------ 原始邮件 ------------------
发件人: "ShaoFeng Shi";<>;
发送时间: 2017年12月22日(星期五) 晚上9:15
收件人: "user"<>;

主题: Re: mismatch hdfs addr error when using kylin2.1 on two hadoop cluster

When configure Kylin with two cluster (one for hive/mapreduce, the other for hbase), need:

1. in kylin node, the core-site.xml uses Hive cluster's HDFS as default FS.
2. in, configure "" to the HDFS of HBase cluster

In your case, I guess you didn't configure 2); please double check

2017-12-22 16:10 GMT+08:00 jxs <>:
I also deploy kylin on two small EMR clusters for resource isolation, I used MySQL for Hive
metastore and shared the same hive configuration between the two clusters.
You may try this.

在2017年12月22 13时08分, "朱真龙"<>写道:

    thank you for your attention first,I am a chinses kylin user that always read english
documents but write little, and i know most of kylin developer are chinese 。so ,if you
don't know what i mean,i will describe again in chinese。

     I am using kylin2.1.0 between two hadoop cluster (all configured HA) and hadoop have
same version(2.7.1).  by now runing well with sample model  , but  not good in my own model
which has many  columns encoding with dict。like this:

when  building cube , got error like this:

after looked into source code that the error metioned, find that the dictionary got hdfs path
from hive table desc , and when checking before load these hfiles into hbase ,found hfiles
and hbase table are on different hdfs cluster .


so, could you tell me how could i do on this case ?

Best regards,

Shaofeng Shi 史少锋
View raw message