Hi zhenlong,

I'm afraid that patch is not all the changes; suggest you download the formal 2.2 release.

2017-12-25 11:53 GMT+08:00 朱真龙 <183920368@qq.com>:
Hi, ShaoFeng,

     thank you  , I'll add the patch to kylin2.1 and retry . thank you again.


------------------ 原始邮件 ------------------
发件人: "ShaoFeng Shi";<shaofengshi@apache.org>;
发送时间: 2017年12月25日(星期一) 中午11:09
收件人: "user"<user@kylin.apache.org>;
主题: Re: Kylin on EMR Installation Error

Hi zhenlong,

Thanks for the detailed analysis. You're correct, there was some inconsistency in Kylin 2.1, it should have been fixed in 2.2, could you please upgrade Kylin and then verify? 

The related JIRA is:

2017-12-25 10:56 GMT+08:00 朱真龙 <183920368@qq.com>:
thank you for your response.after check  https://issues.apache.org/jira/browse/KYLIN-3129 ,I find it's not the same problem as mine and  I have found the way to resolved my problem. I used kylin  between two hadoop cluster ,like this:



its works well when model is not too big. but some model with many row keys does not  .  when I configured fifteen columns for rowkey and setted dict encode like this :
when running to "Build Dimension Dictionary" step,  I got error as follow( it means  after writted snapshot to hdfs, when submit these hdfs file to hbase ,the files and hbaseTable are in different hdfs cluster , one in cluster 1 and one in cluster 2):


through looked into source code the error referred, I found that ,when writting "Dimension Dictionary" to hbase , kylin will judge if your keyvalue is larger than the configured Max_KeyValue size (hbase.client.keyvalue.maxsize is setted in hbase-site.xml),if you not setted ,it will get 10485760(10MB) as default .  if your keyValue size is smaller than hbase.client.keyvalue.maxsize ,then kylin 
will set your key and value into hbase's put object then submit. if your keyvalue is bigger than hbase.client.keyvalue.maxsize ,it will write snapshot into hdfs first then submit it via hbase table.  but I think it's maybe a bug on kylin2.1  when use kylin between two hadoop cluster and build Dimension with keyvalue bigger than hbase.client.keyvalue.maxsize. 

when keyvlaue size is bigger than hbase.client.keyvalue.maxsize , writte snapshot into hdfs( it got the hdfs path from hive table instead of hbase, and they are in different hdfs cluster) . so if you are using one hadoop cluster(hive and hbase are on one hadoop cluster ),it will works well . if you are using two hadoop cluster and your keyvalue size is smaller than hbase.client.keyvalue.maxsize ,it will also works well (use hbase's put object). but if you are using two hadoop cluster and your keyvalue is bigger than  hbase.client.keyvalue.maxsize , you will get then same error like mine(beause your hive and hbase are in different hdfs cluster) .
 
        I checked my hbase.client.keyvalue.maxsize in hbase-site.xml, and found ambari set 1MB as default ,and kylin set 10MB as default .so I changed this value to 10MB , and now my model works well.  If you find a better way to resolve this problem ,please tell me , thank you. 


apache-kylin-2.1.0-src\apache-kylin-2.1.0\storage-hbase\src\main\java\org\apache\kylin\storage\hbase\HBaseResourceStore.java




------------------ 原始邮件 ------------------
发件人: "ShaoFeng Shi";<shaofengshi@apache.org>;
发送时间: 2017年12月22日(星期五) 下午5:49
收件人: "user"<user@kylin.apache.org>;
主题: Re: Kylin on EMR Installation Error


2017-09-06 22:31 GMT+08:00 ShaoFeng Shi <shaofengshi@apache.org>:
Thanks, I will try EMR 5.8 later; 

At this moment, you can try emr 5.5; one thing to pay attention is, you need copy a configuration from hbase-site.xml to KYLIN_HOME/conf/kylin_job_conf.xml (replace the zk quorum name with yours):

<property>
  <name>hbase.zookeeper.quorum</name>
  <value>ip-nn-nn-nn-nn.ap-northeast-2.compute.internal</value>
</property>


2017-09-06 21:07 GMT+08:00 Mukund Agilisium <mukund.agilisium@gmail.com>:
Hi

I am using EMR 5.8.0 , the latest one. 

Thanks & Regards
Mukund

On Wed, Sep 6, 2017 at 8:44 AM, ShaoFeng Shi <shaofengshi@apache.org> wrote:
Hi Mukund, 

What's the EMR version?

Check if there is other version's jar in EMR node, e.g,: 

sudo find / -name  joda-time*.jar


2017-09-06 1:52 GMT+08:00 Mukund Agilisium <mukund.agilisium@gmail.com>:
Hi All

I am trying to install Kylin apache-kylin-2.1.0-bin-hbase1x on EMR and getting the following errors.

[root@ip-172-31-35-25 lib]# pwd
/etc/apache-kylin-2.1.0-bin-hbase1x/tomcat/webapps/kylin/WEB-INF/lib
[root@ip-172-31-35-25 lib]# ls -al | grep joda
-rw-r--r-- 1 root root  573912 May  6  2016 joda-time-2.2.jar

Looks like the jar is available inside the webinf/lib as well.

Any help is appreciated!!

kylin.log
2017-09-05 12:06:45,097 ERROR [localhost-startStop-1] context.ContextLoader:351 : Context initialization failed
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter': Instantiation of bean failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter]: Constructor threw exception; nested exception is java.lang.NoSuchMethodError: org.joda.time.format.DateTimeFormatter.withZoneUTC()Lorg/joda/time/format/DateTimeFormatter;
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateBean(AbstractAutowireCapableBeanFactory.java:1109)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1054)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:510)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:482)
        at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306)
        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302)
        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:778)
        at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:839)
        at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:538)
        at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:444)
        at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:326)
......


Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter]: Constructor threw exception; nested exception is java.lang.NoSuchMethodError: org.joda.time.format.DateTimeFormatter.withZoneUTC()Lorg/joda/time/format/DateTimeFormatter;
        at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:163)
        at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:89)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateBean(AbstractAutowireCapableBeanFactory.java:1102)
        ... 26 more

   
SEVERE: Exception sending context initialized event to listener instance of class org.springframework.web.context.ContextLoaderListener
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter': Instantiation of bean failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter]: Constructor threw exception; nested exception is java.lang.NoSuchMethodError: org.joda.time.format.DateTimeFormatter.withZoneUTC()Lorg/joda/time/format/DateTimeFormatter;
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateBean(AbstractAutowireCapableBeanFactory.java:1109)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1054)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:510)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:482)
        at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306)
        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302)
        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:778)
        at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:839)
 
kylin.out

There is not keystore files in the folder

[root@ip-172-31-35-25 conf]# pwd
/etc/apache-kylin-2.1.0-bin-hbase1x/tomcat/conf
[root@ip-172-31-35-25 conf]# ls *.keystore
ls: cannot access *.keystore: No such file or directory


SEVERE: Failed to load keystore type JKS with path conf/.keystore due to /etc/apache-kylin-2.1.0-bin-hbase1x/tomcat/conf/.keystore (No such file or directory)
java.io.FileNotFoundException: /etc/apache-kylin-2.1.0-bin-hbase1x/tomcat/conf/.keystore (No such file or directory)
        at java.io.FileInputStream.open0(Native Method)
        at java.io.FileInputStream.open(FileInputStream.java:195)
        at java.io.FileInputStream.<init>(FileInputStream.java:138)
        at java.io.FileInputStream.<init>(FileInputStream.java:93)
       

Sep 05, 2017 12:06:38 PM org.apache.catalina.core.StandardService initInternal
SEVERE: Failed to initialize connector [Connector[HTTP/1.1-7443]]
org.apache.catalina.LifecycleException: Failed to initialize component [Connector[HTTP/1.1-7443]]
        at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:106)
        at org.apache.catalina.core.StandardService.initInternal(StandardService.java:560)
        at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:102)
        at org.apache.catalina.core.StandardServer.initInternal(StandardServer.java:838)
        at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:102)



--
Best regards,

Shaofeng Shi 史少锋





--
Best regards,

Shaofeng Shi 史少锋




--
Best regards,

Shaofeng Shi 史少锋




--
Best regards,

Shaofeng Shi 史少锋




--
Best regards,

Shaofeng Shi 史少锋