kylin-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mvishnubhatta <mvishnubha...@argusinformation.com>
Subject Kylin Start fails with NoClassDefFoundError on AWS EMR Edge Node
Date Thu, 09 Apr 2020 01:22:59 GMT
Hi,

I am trying to set up a Kylin server on an AWS EMR edge node. When doing
that the web URL throws the error that "The origin server did not find a
current representation for the target resource or is not willing to disclose
that one exists".

The tomcat localhost log file shows a NoClassDefFoundError for the class
org/apache/hadoop/hive/metastore/api/NoSuchObjectException 

But I see this class in the hive library
/usr/lib/hive/lib/hive-metastore-2.3.5-amzn-0.jar and the path is exported
as HIVE_LIB and as HIVE_HOME (used HIVE_LIB based on suggestion here
https://issues.apache.org/jira/browse/KYLIN-2511).

This appears to be a case where some CLASSPATH somewhere is not correctly
including this path.

So, I tried modifying the kylin.properties file to add a property called
kylin.job.mr.lib.dir pointing to the /usr/lib/hive/lib/ but that didnt help
either. (This too was based on another suggested workaround here).

Can someone point out what I am missing?

Here are the full details:

Hadoop version is Hadoop 2.8.5-amzn-4
Kylin version is 3.0.1
Hive 2.3.5-amzn-0

I have an AWS EMR Cluster and a separate EC2 machine that is set up as an
edge node. From this edge node I am able to run hive queries to connect to
the EMR Master node. Hbase also works on this node.

On this edge node, I are trying to start a Kylin server. I followed the
instructions given here: 
https://kylin.apache.org/docs/install/kylin_aws_emr.html
<https://kylin.apache.org/docs/install/kylin_aws_emr.html>  

I exported the following variables:
export KYLIN_HOME=/usr/local/kylin/apache-kylin-3.0.1-bin-hbase1x
#export HADOOP_HOME=/usr/lib/hadoop/etc/hadoop
export HADOOP_HOME=/usr/lib/hadoop
export HBASE_HOME=/usr/lib/hbase/
export HIVE_HOME=/usr/lib/hive/lib/
export HADOOP_CONF_DIR=/etc/hadoop/conf
export HIVE_LIB=/usr/lib/hive/lib
export HIVE_CONF=/etc/hive/conf
export HCAT_HOME=/usr/lib/hive-hcatalog


The command I run is: $KYLIN_HOME/bin/kylin.sh start

After starting it, I get this message:

/A new Kylin instance is started by hadoop. To stop it, run 'kylin.sh stop'
Check the log at
/usr/local/kylin/apache-kylin-3.0.1-bin-hbase1x/logs/kylin.log
Web UI is at http://ip-xxxxxxxxxxxxxxxxxxxxx.compute.internal:7070/kylin/

When I navigate to the URL, I get the following error.

"The origin server did not find a current representation for the target
resource or is not willing to disclose that one exists."

When I look into the log file which is at
$KYLIN_HOME/tomcat/logs/localhost.2020-04-08.log I see the following error:

SEVERE: Exception sending context initialized event to listener instance of
class org.springframework.web.context.ContextLoaderListener
org.springframework.beans.factory.BeanCreationException: Error creating bean
with name
'org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerMapping':
Invocation of init method failed; nested exception is
java.lang.NoClassDefFoundError:
org/apache/hadoop/hive/metastore/api/NoSuchObjectException
        at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1628)
        at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:555)
        at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483)
        at
org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306)
        at
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
        at
org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302)
        at
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
        at
org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:761)
        at
org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:867)
        at
org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:543)
        at
org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:443)
        at
org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:325)
        at
org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:107)
        at
org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:5157)
        at
org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5680)
        at
org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145)
        at
org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:1018)
        at
org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:994)
        at
org.apache.catalina.core.StandardHost.addChild(StandardHost.java:652)
        at
org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:1127)
        at
org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:2021)
        at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NoClassDefFoundError:
org/apache/hadoop/hive/metastore/api/NoSuchObjectException
        at java.lang.Class.getDeclaredMethods0(Native Method)
        at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
        at java.lang.Class.getDeclaredMethods(Class.java:1975)
        at
org.springframework.util.ReflectionUtils.getDeclaredMethods(ReflectionUtils.java:613)
        at
org.springframework.util.ReflectionUtils.doWithMethods(ReflectionUtils.java:524)
        at
org.springframework.core.MethodIntrospector.selectMethods(MethodIntrospector.java:68)
        at
org.springframework.web.servlet.handler.AbstractHandlerMethodMapping.detectHandlerMethods(AbstractHandlerMethodMapping.java:230)
        at
org.springframework.web.servlet.handler.AbstractHandlerMethodMapping.initHandlerMethods(AbstractHandlerMethodMapping.java:214)
        at
org.springframework.web.servlet.handler.AbstractHandlerMethodMapping.afterPropertiesSet(AbstractHandlerMethodMapping.java:184)
        at
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerMapping.afterPropertiesSet(RequestMappingHandlerMapping.java:127)
        at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1687)
        at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1624)
        ... 25 more
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.hive.metastore.api.NoSuchObjectException
        at
org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1928)




--
Sent from: http://apache-kylin.74782.x6.nabble.com/

Mime
View raw message