kylin-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 磊 王 <biolearn...@hotmail.com>
Subject Re: Problem while querying Cube
Date Mon, 06 Feb 2017 13:54:50 GMT
My bad, :)
I will go back to my own thread when needed.


Thx
Lei Wang

发自我的 iPhone

> 在 2017年2月6日,21:52,ShaoFeng Shi <shaofengshi@gmail.com> 写道:
> 
> seem be :) pls ignore and go ahead.
> Lei, pls use your origin thread to discuss the problem.
> 
> Get Outlook for iOS
> 
> 
> 
> 
> On Mon, Feb 6, 2017 at 10:39 PM +0900, "hongbin ma" <mahongbin@apache.org> wrote:
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> are we mixing two separate issues in the same thread here?
> 
>> On Mon, Feb 6, 2017 at 9:26 PM, ShaoFeng Shi  wrote:
>> 
>> NoSuchObjectException(message:default.kylin_intermediate_
>> kylin_sales_cube_desc_3a41df7d_93e1_4445_9
>> usually this error indicates that hive-site.xml wasn't on classpath; a
>> quick workaround is copying that file to $KYLIN_HOME/conf; the ultimate
>> solution is checking why bin/find-hive-dependency.sh didn't find it
>> automatically.
>> Get Outlook for iOS
>> 
>> 
>> 
>> 
>> On Mon, Feb 6, 2017 at 6:43 PM +0900, "磊 王" 
>> wrote:
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> I manually added core-site.xml under kylin conf, and then started Kylin.
>> When I build the sample, it seems go over the wrong place yesterday, but
>> errored at another place.
>> 
>> 
>> 
>> cat core-site.xml
>> 
>> 
>>        fs.defaultFS
>>        hdfs://sandbox.hortonworks.com:8020
>> 
>> 
>> 
>> 
>> 2017-02-06 09:34:53,357 INFO  [pool-8-thread-3]
>> metastore.MetaStoreDirectSql:140 : Using direct SQL, underlying DB is
>> DERBY
>> 2017-02-06 09:34:53,359 INFO  [pool-8-thread-3] metastore.ObjectStore:273
>> : Initialized ObjectStore
>> 2017-02-06 09:34:53,590 INFO  [pool-8-thread-3]
>> metastore.HiveMetaStore:664 : Added admin role in metastore
>> 2017-02-06 09:34:53,592 INFO  [pool-8-thread-3]
>> metastore.HiveMetaStore:673 : Added public role in metastore
>> 2017-02-06 09:34:53,651 INFO  [pool-8-thread-3]
>> metastore.HiveMetaStore:713 : No user is added in admin role, since config
>> is empty
>> 2017-02-06 09:34:53,770 INFO  [pool-8-thread-3]
>> metastore.HiveMetaStore:747 : 0: get_databases:
>> NonExistentDatabaseUsedForHealthCheck
>> 2017-02-06 09:34:53,771 INFO  [pool-8-thread-3] HiveMetaStore.audit:372 :
>> ugi=root      ip=unknown-ip-addr      cmd=get_databases:
>> NonExistentDatabaseUsedForHealthCheck
>> 2017-02-06 09:34:53,791 INFO  [pool-8-thread-3]
>> metastore.HiveMetaStore:747 : 0: get_table : db=default
>> tbl=kylin_intermediate_kylin_sales_cube_desc_3a41df7d_93e1_
>> 4445_9ca0_882f5f6e9d10
>> 2017-02-06 09:34:53,791 INFO  [pool-8-thread-3] HiveMetaStore.audit:372 :
>> ugi=root      ip=unknown-ip-addr      cmd=get_table : db=default
>> tbl=kylin_intermediate_kylin_sales_cube_desc_3a41df7d_93e1_
>> 4445_9ca0_882f5f6e9d10
>> 2017-02-06 09:34:53,812 INFO  [pool-8-thread-3]
>> common.AbstractHadoopJob:506 : tempMetaFileString is : null
>> 2017-02-06 09:34:53,814 ERROR [pool-8-thread-3]
>> common.MapReduceExecutable:127 : error execute MapReduceExecutable{id=
>> baa9531d-fc11-4a60-aa5e-a069d4bee3c2-02, name=Extract Fact Table Distinct
>> Columns, state=RUNNING}
>> java.lang.RuntimeException: java.io.IOException:
>> NoSuchObjectException(message:default.kylin_intermediate_
>> kylin_sales_cube_desc_3a41df7d_93e1_4445_9ca0_882f5f6e9d10 table not
>> found)
>>        at org.apache.kylin.source.hive.HiveMRInput$HiveTableInputFormat.
>> configureJob(HiveMRInput.java:110)
>>        at org.apache.kylin.engine.mr.steps.FactDistinctColumnsJob.
>> setupMapper(FactDistinctColumnsJob.java:119)
>>        at org.apache.kylin.engine.mr.steps.FactDistinctColumnsJob.
>> run(FactDistinctColumnsJob.java:103)
>>        at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:92)
>>        at org.apache.kylin.engine.mr.common.MapReduceExecutable.
>> doWork(MapReduceExecutable.java:120)
>>        at org.apache.kylin.job.execution.AbstractExecutable.
>> execute(AbstractExecutable.java:113)
>>        at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(
>> DefaultChainedExecutable.java:57)
>>        at org.apache.kylin.job.execution.AbstractExecutable.
>> execute(AbstractExecutable.java:113)
>>        at org.apache.kylin.job.impl.threadpool.DefaultScheduler$
>> JobRunner.run(DefaultScheduler.java:136)
>>        at java.util.concurrent.ThreadPoolExecutor.runWorker(
>> ThreadPoolExecutor.java:1145)
>>        at java.util.concurrent.ThreadPoolExecutor$Worker.run(
>> ThreadPoolExecutor.java:615)
>>        at java.lang.Thread.run(Thread.java:745)
>> Caused by: java.io.IOException: NoSuchObjectException(message:
>> default.kylin_intermediate_kylin_sales_cube_desc_3a41df7d_93e1_4445_9ca0_882f5f6e9d10
>> table not found)
>>        at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.
>> setInput(HCatInputFormat.java:97)
>>        at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.
>> setInput(HCatInputFormat.java:51)
>>        at org.apache.kylin.source.hive.HiveMRInput$HiveTableInputFormat.
>> configureJob(HiveMRInput.java:105)
>>        ... 11 more
>> Caused by: NoSuchObjectException(message:default.kylin_intermediate_
>> kylin_sales_cube_desc_3a41df7d_93e1_4445_9ca0_882f5f6e9d10 table not
>> found)
>>        at org.apache.hadoop.hive.metastore.HiveMetaStore$
>> HMSHandler.get_table_core(HiveMetaStore.java:1806)
>>        at org.apache.hadoop.hive.metastore.HiveMetaStore$
>> HMSHandler.get_table(HiveMetaStore.java:1776)
>>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>        at sun.reflect.NativeMethodAccessorImpl.invoke(
>> NativeMethodAccessorImpl.java:57)
>>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(
>> DelegatingMethodAccessorImpl.java:43)
>>        at java.lang.reflect.Method.invoke(Method.java:606)
>>        at org.apache.hadoop.hive.metastore.RetryingHMSHandler.
>> invoke(RetryingHMSHandler.java:107)
>>        at com.sun.proxy.$Proxy49.get_table(Unknown Source)
>>        at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.
>> getTable(HiveMetaStoreClient.java:1209)
>>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>        at sun.reflect.NativeMethodAccessorImpl.invoke(
>> NativeMethodAccessorImpl.java:57)
>>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(
>> DelegatingMethodAccessorImpl.java:43)
>>        at java.lang.reflect.Method.invoke(Method.java:606)
>>        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.
>> invoke(RetryingMetaStoreClient.java:152)
>>        at com.sun.proxy.$Proxy50.getTable(Unknown Source)
>>        at org.apache.hive.hcatalog.common.HCatUtil.getTable(
>> HCatUtil.java:180)
>>        at org.apache.hive.hcatalog.mapreduce.InitializeInput.
>> getInputJobInfo(InitializeInput.java:105)
>>        at org.apache.hive.hcatalog.mapreduce.InitializeInput.
>> setInput(InitializeInput.java:86)
>>        at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.
>> setInput(HCatInputFormat.java:95)
>>        ... 13 more
>> 2017-02-06 09:34:53,817 DEBUG [pool-8-thread-3] dao.ExecutableDao:210 :
>> updating job output, id: baa9531d-fc11-4a60-aa5e-a069d4bee3c2-02
>> 
>> 
>> Thx
>> Lei Wang
>> ________________________________
>> From: ShaoFeng Shi
>> Sent: Sunday, February 5, 2017 22:19
>> To: dev
>> Subject: Re: Problem while querying Cube
>> 
>> Most of the developers are on festival vacation, please allow some delay;
>> btw, could you please open a JIRA for it, it looks like a bug.
>> 
>> 2017-02-05 21:42 GMT+08:00 Shailesh Prajapati :
>> 
>>> What should i supposed to do now to make query work with limits? Because
>> i
>>> am keep getting this exception. What will happen if we remove this check?
>>> 
>>> Thanks.
>>> 
>>> On Sat, Feb 4, 2017 at 8:12 PM, ShaoFeng Shi
>>> wrote:
>>> 
>>>> I don't know what that check is necessary, as there is a todo which
>> says
>>>> will remove that someday:
>>>> https://github.com/apache/kylin/blob/master/core-
>>>> storage/src/main/java/org/apache/kylin/storage/gtrecord/
>>>> SortedIteratorMergerWithLimit.java#L127
>>>> 
>>>> @Hongbin, any idea?
>>>> 
>>>> 
>>>> 2017-02-04 17:05 GMT+08:00 Shailesh Prajapati :
>>>> 
>>>>> Hi ShaoFeng,
>>>>> 
>>>>> Here is the gist link,
>>>>> 
>>>>> https://gist.github.com/shaipraj/780a3dcc80aa2080911b7348c76f5b88
>> [https://avatars1.githubusercontent.com/u/22189344?v=3&s=400]
>> 
>> kylin.log
>> gist.github.com
>> property=
>> 
>> 
>> 
>>>>> 
>>>>> Thanks.
>>>>> 
>>>>> On Sat, Feb 4, 2017 at 2:24 PM, ShaoFeng Shi
>>>>> wrote:
>>>>> 
>>>>>> Hi Shailesh, there is no attachement (attachement isn't supported
>> by
>>>>>> mailing list); can you paste the content directly or put it to
>> gist?
>>>>>> 
>>>>>> 2017-02-04 15:49 GMT+08:00 Shailesh Prajapati  >:
>>>>>> 
>>>>>>> Hi ShaoFeng,
>>>>>>> 
>>>>>>> I am attaching a portion of kylin's log.
>>>>>>> 
>>>>>>> Thanks.
>>>>>>> 
>>>>>>> On Sat, Feb 4, 2017 at 12:59 PM, ShaoFeng Shi <
>>>> shaofengshi@apache.org>
>>>>>>> wrote:
>>>>>>> 
>>>>>>>> Hi Shailesh,
>>>>>>>> 
>>>>>>>> Could you please provide the error trace? We need to know
where
>>> the
>>>>>> error
>>>>>>>> got thrown. Thanks.
>>>>>>>> 
>>>>>>>> 2017-02-03 18:06 GMT+08:00 Shailesh Prajapati <
>>>> shailesh@infoworks.io
>>>>>> :
>>>>>>>> 
>>>>>>>>> Hi,
>>>>>>>>> 
>>>>>>>>> We are running Kylin 1.6 and successfully build Cube
on it.
>>>>> Aggregate
>>>>>>>>> queries are running fine But, with non aggregate query
we are
>>>>> getting
>>>>>>>>> following exception,
>>>>>>>>> 
>>>>>>>>> org.apache.kylin.rest.exception.InternalErrorException:
Not
>>>> sorted!
>>>>>>>> last:
>>>>>>>>> CUSTOMER_ID=0,ORDER_ID=10345,QUANTITY=null ... and other
>>> columns.
>>>>>>>>> 
>>>>>>>>> Query used: select ORDERS.STATUS from ORDER_DETAILS as
ORDERS
>>>> limit
>>>>> 5;
>>>>>>>>> 
>>>>>>>>> One more observation, with limit less than 5 even non
>> aggregate
>>>>>> queries
>>>>>>>> are
>>>>>>>>> also working.
>>>>>>>>> Please help us resolving this issue. let us know for
any other
>>>>>>>> information.
>>>>>>>>> 
>>>>>>>>> Thanks
>>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> --
>>>>>>>> Best regards,
>>>>>>>> 
>>>>>>>> Shaofeng Shi 史少锋
>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> --
>>>>>>> Shailesh
>>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> --
>>>>>> Best regards,
>>>>>> 
>>>>>> Shaofeng Shi 史少锋
>>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> Shailesh
>>>>> 
>>>> 
>>>> 
>>>> 
>>>> --
>>>> Best regards,
>>>> 
>>>> Shaofeng Shi 史少锋
>>>> 
>>> 
>>> 
>>> 
>>> --
>>> Shailesh
>>> 
>> 
>> 
>> 
>> --
>> Best regards,
>> 
>> Shaofeng Shi 史少锋
>> 
>> 
>> 
>> 
>> 
>> 
> 
> 
> -- 
> Regards,
> 
> *Bin Mahone | 马洪宾*
> 
> 
> 
> 
> 
Mime
View raw message