OK, May thx :-)

在 2016年3月18日,23:28,ShaoFeng Shi <shaofengshi@apache.org> 写道:

Thanks for the information; We will check this and try to fix it in next release, a JIRA is created for tracking this: https://issues.apache.org/jira/browse/KYLIN-1507

ianzeng, please try the workaround that Pierre mentioned, it should work definitely. 

Thanks Pierre for the sharing, it does save our effort on investigate this issue.

2016-03-18 23:10 GMT+08:00 Pierre Gunet-Caplain <pgc@deezer.com>:
Hi,

I also had this problem on a cloudera distribution. I found out there is a regex that filters the dependencies to keep only a few of them (hive-exec, hive-metastore and hive-hcatalog-core if I'm right), and the regex is filtering out the cloudera jars names.

So I used another way send the dependencies : in kylin.properties I added kylin.job.mr.lib.dir=/local/directory/kylin/dep . In this folder I created symbolic links to the cloudera jars enumerated above. These jars are shipped to tmpjars at execution, and it works.

I hope it help,

Pierre
Data Architect at Deezer



2016-03-18 15:51 GMT+01:00 ianzeng <claytonly@163.com>:
the find-hive-decency result is
 bin/find-hive-dependency.sh 
16/03/18 18:09:26 WARN conf.HiveConf: DEPRECATED: Configuration property hive.metastore.local no longer has any effect. Make sure to provide a valid value for hive.metastore.uris if you are connecting to a remote metastore.

Logging initialized using configuration in jar:file:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/jars/hive-common-0.13.1-cdh5.3.3.jar!/hive-log4j.properties
HCAT_HOME not found, try to find hcatalog path from hadoop home
hive dependency: /opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/lib/hive/conf:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/hbase-client.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/commons-compress-1.4.1.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/jline-0.9.94.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/jpam-1.1.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/hive-shims-0.23.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/hive-beeline.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/hive-hwi.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/junit-4.11.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/hive-cli.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/hbase-server.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/datanucleus-api-jdo-3.2.6.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/servlet-api-2.5.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/hive-ant-0.13.1-cdh5.3.3.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/jersey-server-1.14.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/xz-1.0.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/hive-exec-0.13.1-cdh5.3.3.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/servlet-api-2.5-20081211.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/hive-testutils-0.13.1-cdh5.3.3.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/asm-commons-3.1.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/asm-3.2.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/htrace-core.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/hive-cli-0.13.1-cdh5.3.3.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/hbase-common.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/avro.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/libfb303-0.9.0.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/geronimo-jta_1.1_spec-1.1.1.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/hive-testutils.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/hive-hwi-0.13.1-cdh5.3.3.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/hive-shims-0.23-0.13.1-cdh5.3.3.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/oro-2.0.8.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/commons-httpclient-3.0.1.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/derby-10.10.1.1.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/groovy-all-2.1.6.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/commons-logging-1.1.3.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/hive-shims.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/hive-hbase-handler.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/hamcrest-core-1.1.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/hive-jdbc-0.13.1-cdh5.3.3.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/datanucleus-rdbms-3.2.9.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/hive-service.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/jersey-servlet-1.14.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/libthrift-0.9.0-cdh5-2.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/geronimo-annotation_1.0_spec-1.1.1.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/hive-contrib.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/httpclient-4.2.5.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/asm-tree-3.1.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/commons-cli-1.2.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/zookeeper.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/hive-jdbc.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/hive-shims-common.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/hive-beeline-0.13.1-cdh5.3.3.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/hive-ant.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/guava-11.0.2.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/jdo-api-3.0.1.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/hbase-protocol.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/hive-common.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/hive-metastore.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/activation-1.1.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/ant-launcher-1.9.1.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/paranamer-2.3.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/hive-serde-0.13.1-cdh5.3.3.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/geronimo-jaspic_1.0_spec-1.0.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/jta-1.1.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/hive-shims-common-0.13.1-cdh5.3.3.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/hive-service-0.13.1-cdh5.3.3.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/mail-1.4.1.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/antlr-2.7.7.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/commons-lang-2.6.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/hbase-hadoop-compat.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/hive-exec.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/commons-codec-1.4.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/hbase-hadoop2-compat.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/hive-shims-scheduler.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/log4j-1.2.16.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/ST4-4.0.4.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/antlr-runtime-3.4.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/stringtemplate-3.2.1.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/tempus-fugit-1.1.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/opencsv-2.3.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/snappy-java-1.0.4.1.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/hive-shims-common-secure.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/jetty-all-7.6.0.v20120127.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/hive-serde.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/hive-metastore-0.13.1-cdh5.3.3.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/stax-api-1.0.1.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/hive-shims-common-secure-0.13.1-cdh5.3.3.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/velocity-1.5.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/hive-contrib-0.13.1-cdh5.3.3.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/hive-shims-0.13.1-cdh5.3.3.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/jsr305-1.3.9.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/hive-hbase-handler-0.13.1-cdh5.3.3.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/datanucleus-core-3.2.10.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/hive-common-0.13.1-cdh5.3.3.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/hive-shims-scheduler-0.13.1-cdh5.3.3.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/super-csv-2.2.0.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/commons-lang3-3.1.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/httpcore-4.2.5.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/jetty-util-6.1.26.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/commons-io-2.4.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/ant-1.9.1.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/bonecp-0.8.0.RELEASE.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive/lib/jetty-6.1.26.jar:/opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/hive-hcatalog/share/hcatalog/hive-hcatalog-core-0.13.1-cdh5.3.3.jar
在 2016年3月18日,22:33,ShaoFeng Shi <shaofengshi@apache.org> 写道:

Run $KYLIN_HOME/bin/find-hive-dependency.sh to see whether it can locate the HCat jar files in the machine that running Kylin, please paste the output here for trouble shotting. 

2016-03-18 18:28 GMT+08:00 ianzeng <claytonly@163.com>:
HI all,

	I has installed kylin 1.5 on redhead 6.3. I try build sample cube. But got error msg as follow:

2016-03-18 18:18:43,084 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: hadoop.ssl.server.conf;  Ignoring.
2016-03-18 18:18:43,093 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2016-03-18 18:18:43,509 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
2016-03-18 18:18:43,921 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2016-03-18 18:18:43,933 INFO [main] org.apache.hadoop.mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
2016-03-18 18:18:44,120 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hive.hcatalog.mapreduce.HCatInputFormat not found
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2047)
	at org.apache.hadoop.mapreduce.task.JobContextImpl.getInputFormatClass(JobContextImpl.java:184)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:746)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1642)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.lang.ClassNotFoundException: Class org.apache.hive.hcatalog.mapreduce.HCatInputFormat not found
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1953)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2045)
	... 8 more


What should I do to fix this problem ?



--
Best regards,

Shaofeng Shi






--
Best regards,

Shaofeng Shi