kylin-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From liuyiming....@gmail.com
Subject Re: URGENT !!! Help needed - Issue executing query in Kylin
Date Fri, 27 Jan 2017 13:37:00 GMT


For item a), correct. In your case, I think this is the root cause.

For item b), if the latest kylin could not work for the newer CDH, the older one could not
work either definetly. The dev team has no resource to verify CDH5.9 yet. There is no special
APIs used for CDH5.7+. So the CDH release should work for you.





Yiming Liu






发件人: Manoj Jain

发送时间: 1月27日 星期五 20:48

主题: Re: URGENT !!! Help needed - Issue executing query in Kylin

收件人: user@kylin.apache.org

抄送: dev@kylin.apache.org, Sudeep Dey, Raja Tallapaneni, Sanjeev Singh



Thanks for your quick response. Need few clarification:



a)  "When you upgrade, please follow the upgrade instructuon, such as redeploy the coprocessor
jar.” - I believe you are saying when we upgrade the kylin version, we need to redeploy
the coprocessor jar. Confirm my understanding.



b) We plan to deploy the newer version of the Kylin. However, the new  Kylin version does
not mention compatibility with CDH 5.9.


I need your advice whether we upgrade to newer version of Kylin or apply some patch that will
resolve the current issue.



http://kylin.apache.org/download/


1.6.0


In this version Kylin has better support for Kafka as data source, read blog New NRT Streaming
in Apache Kylin.Release NotesSource download: apache-kylin-1.6.0-src.tar.gz [asc] [md5]Binary
download:for HBase 0.98 (includes HDP 2.2, CDH 5.2/5.3) - apache-kylin-1.6.0-bin.tar.gz [asc]
[md5]for HBase 1.x (includes HDP 2.3/2.4, EMR 5.0) - apache-kylin-1.6.0-hbase1.x-bin.tar.gz [asc]
[md5]for CDH 5.5/5.7/5.8 - apache-kylin-1.6.0-cdh5.7-bin.tar.gz [asc] [md5]


Thanks,


Manoj Jain



On Jan 26, 2017, at 11:59 PM, liuyiming.vip@gmail.com wrote:



When you upgrade, please follow the upgrade instructuon, such as redeploy the coprocessor
jar.



Yiming Liu







On Fri, Jan 27, 2017 at 12:33 PM +0800, "Manoj Jain" <mjain@zaloni.com> wrote:








We were running klyin 1.5 on CDH 5.7 and was working fine.



Post upgrade to CDH 5.9, we are seeing an exception while running the queries in Kylin.



I have restarted the Kylin 1.5 instance and also build the test cube from scratch. The cube
built was successful but seeing an exception in the Kylin.log when running the queries (in
the Kyiin ‘Insight’ tab).



 Please help us to resolve this issue at the earliest. Let us know if  you need any other
information.



Kylin Version: 1.5.2



Excerpts from the Server Environment


HBASE_LOG_DIR = /var/log/hbase


HBASE_HOME = /opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.21/lib/hbase/bin/..


HBASE_OPTS = -Xmx268435456 -XX:+HeapDumpOnOutOfMemoryError -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode
-Djava.net.preferIPv4Stack=true  -Djava.security.auth.login.config=/etc/hbase/conf/jaas.conf
 -Dhbase.log.dir=/var/log/hbase -Dhbase.log.file=hbase.log -Dhbase.home.dir=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.21/lib/hbase/bin/..
-Dhbase.id.str=hbase -Dhbase.root.logger=INFO,console -Djava.library.path=/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.21/lib/hadoop/lib/native:/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.21/lib/hbase/bin/../lib/native/Linux-amd64-64
-Dhbase.security.logger=INFO,NullAppender


HBASE_ENV_INIT = true


hbase_dependency = /opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.21/lib/hbase/bin/../lib/hbase-common-1.2.0-cdh5.9.0.jar





Kylin log exceprts



2017-01-26 18:33:50,419 INFO  [http-bio-10101-exec-7] v2.CubeHBaseRPC:271 : Visiting hbase
table KYLIN_XXXXXXXXX: cuboid exact match, from 2 to 2 Start: \x00\x00\x00\x00\x00\x0


0\x00\x00\x00\x02\x00 (\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00) Stop:  \x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\xFF\x00
(\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\xFF\x00), 


No Fuzzy Key


2017-01-26 18:33:50,419 DEBUG [http-bio-10101-exec-7] v2.CubeHBaseEndpointRPC:319 : Submitting
rpc to 1 shards starting from shard 0, scan range count 1


2017-01-26 18:33:50,419 INFO  [http-bio-10101-exec-7] v2.CubeHBaseEndpointRPC:102 : Timeout
for ExpectedSizeIterator is: 198000


2017-01-26 18:33:50,420 DEBUG [http-bio-10101-exec-7] enumerator.OLAPEnumerator:127 : return
TupleIterator...


2017-01-26 18:33:50,430 ERROR [pool-7-thread-5] util.LoggableCachedThreadPool:44 : Execution
exception when running task in pool-7-thread-5


2017-01-26 18:33:50,431 ERROR [pool-7-thread-5] util.LoggableCachedThreadPool:54 : Caught
exception in thread pool-7-thread-5: 


java.lang.RuntimeException: <sub-thread for GTScanRequest 1c42a42b> Error when visiting
cubes by endpoint


        at org.apache.kylin.storage.hbase.cube.v2.CubeHBaseEndpointRPC$1.run(CubeHBaseEndpointRPC.java:344)


        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)


        at java.util.concurrent.FutureTask.run(FutureTask.java:262)


        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)


        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)


        at java.lang.Thread.run(Thread.java:745)


Caused by: org.apache.hadoop.hbase.DoNotRetryIOException: org.apache.hadoop.hbase.DoNotRetryIOException:
java.lang.NoSuchMethodError: org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment.getRegion()Lorg/apache/hadoop/hbase/regionserver/HRegion;


        at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2212)


        at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:109)


        at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:185)


        at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:165)


Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment.getRegion()Lorg/apache/hadoop/hbase/regionserver/HRegion;


        at org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.CubeVisitService.visitCube(CubeVisitService.java:177)


        at org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitService.callMethod(CubeVisitProtos.java:4164)


        at org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:7797)


        at org.apache.hadoop.hbase.regionserver.RSRpcServices.execServiceOnRegion(RSRpcServices.java:1982)


        at org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:1964)


        at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:33652)


        at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2170)


        ... 3 more



        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)


        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)


        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)


        at java.lang.reflect.Constructor.newInstance(Constructor.java:526)


        at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)


        at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:95)


        at org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:327)


        at org.apache.hadoop.hbase.protobuf.ProtobufUtil.execService(ProtobufUtil.java:1631)


        at org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel$1.call(RegionCoprocessorRpcChannel.java:92)


        at org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel$1.call(RegionCoprocessorRpcChannel.java:89)


        at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:126)


        at org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel.callExecService(RegionCoprocessorRpcChannel.java:95)


        at org.apache.hadoop.hbase.ipc.CoprocessorRpcChannel.callMethod(CoprocessorRpcChannel.java:56)


        at org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitService$Stub.visitCube(CubeVisitProtos.java:4225)


        at org.apache.kylin.storage.hbase.cube.v2.CubeHBaseEndpointRPC$2.call(CubeHBaseEndpointRPC.java:392)


        at org.apache.kylin.storage.hbase.cube.v2.CubeHBaseEndpointRPC$2.call(CubeHBaseEndpointRPC.java:388)


        at org.apache.hadoop.hbase.client.HTable$15.call(HTable.java:1797)


        ... 4 more


Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.DoNotRetryIOException):
org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.NoSuchMethodError: org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment.getRegion()Lorg/apache/hadoop/hbase/regionserver/HRegion;


        at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2212)


        at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:109)


        at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:185)


        at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:165)


Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment.getRegion()Lorg/apache/hadoop/hbase/regionserver/HRegion;


        at org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.CubeVisitService.visitCube(CubeVisitService.java:177)


        at org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitService.callMethod(CubeVisitProtos.java:4164)


        at org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:7797)


        at org.apache.hadoop.hbase.regionserver.RSRpcServices.execServiceOnRegion(RSRpcServices.java:1982)


        at org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:1964)


        at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:33652)


        at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2170)


        ... 3 more



        at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1269)


        at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:226)


        at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:331)


        at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.execService(ClientProtos.java:34118)


        at org.apache.hadoop.hbase.protobuf.ProtobufUtil.execService(ProtobufUtil.java:1627)


        ... 13 more





Thanks,


Manoj Jain





Mime
View raw message