kylin-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "郭锋" <guof...@bcpeople.com>
Subject Re:答复: 答复: 【kylin2.5.0】中文查询结果问题
Date Thu, 18 Oct 2018 02:23:26 GMT
Hi:
     I have confirmed that  it don't  contain any special character ;
    this is strange, You can refer to the previous mail contents:
           when i use  where operation = "中国电信" , it returns 0;
             when i use  where operation != "中国电信" , it returns the correct result;




------------------


郭锋



百乘金科/金蛋分期/技术部/数据开发组

北京市朝阳区北辰世纪中心B座17层




 
 
 
------------------ Original ------------------
From:  "Na Zhai"<na.zhai@kyligence.io>;
Date:  Wed, Oct 17, 2018 05:12 PM
To:  "user@kylin.apache.org"<user@kylin.apache.org>; 

Subject:  答复: 答复: 【kylin2.5.0】中文查询结果问题

 
  
Hi, 郭锋
 
Can you check the 'operator' field whether contains the special character, like null or space
or others?
 
Attachment is the result that I query with Chinese.
 
 
 
发送自 Windows 10 版邮件应用
 
 
 
 
 发件人: 49188074@qq.com <49188074@qq.com>
 发送时间: Wednesday, October 17, 2018 11:38:43 AM
 收件人: user
 主题: 回复: 答复: 【kylin2.5.0】中文查询结果问题  
 
  Hi, Na Zhai
 
 
     The attachment is Cube's JSON. Thanks
 
 
 
   49188074@qq.com
 
 
   
   发件人: Na Zhai
 发送时间: 2018-10-17 10:52
 收件人: user@kylin.apache.org
 主题: 答复: 答复: 【kylin2.5.0】中文查询结果问题
 
 
    
 Hi, 郭锋
 
 Dict is ok. I can not reproduce this problem. So can you provide more information? Such as
your Cube's JSON file or other info that you can provide to help me reproduce this problem.
Thanks
 
  
 
  
 
 发送自 Windows 10 版邮件应用
 
  
 
 
 发件人: 49188074@qq.com <49188074@qq.com>
 发送时间: Wednesday, October 17, 2018 10:31:27 AM
 收件人: user
 主题: 回复: 答复: 【kylin2.5.0】中文查询结果问题  
 
  Hi,Na Zhai
 
 
    The encoding of the 'operator' field is Dict, which type should be used?
 
 
 
   49188074@qq.com
 
 
   
   发件人: Na Zhai
 发送时间: 2018-10-16 19:32
 收件人: user@kylin.apache.org
 主题: 答复: Re:答复: Re:答复: 【kylin2.5.0】中文查询结果问题
 
 
    
 Hi,郭锋
 
 What is the encoding of the ‘operator‘ field? Dict or fixed_length or others?
 
  
 
 发送自 Windows 10 版邮件应用
 
  
 
 
 发件人: 郭锋 <guofeng@bcpeople.com>
 发送时间: Tuesday, October 16, 2018 3:56:28 PM
 收件人: user
 主题: Re:答复: Re:答复: 【kylin2.5.0】中文查询结果问题  
 
  Hi,
      yes, we use the apache-kylin-2.5.0-bin-cdh60.tar.gz version on cdh6.0.0 cluster;
      the before three select clause log as follows:
 
 
  
 2018-10-16 15:49:59,532 INFO  [http-nio-7070-exec-18] service.QueryService:350 : Check query
permission in 0 ms.
 2018-10-16 15:49:59,533 DEBUG [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] util.CheckUtil:35
: query cache disabled in KylinConfig
 2018-10-16 15:49:59,533 INFO  [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] service.QueryService:386
: Using project: BI
 2018-10-16 15:49:59,534 INFO  [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] service.QueryService:387
: The original query:  select
 operator,
 count(*)
 from app_bi_user
 where operator = '中国电信'
 group by  operator;
 2018-10-16 15:49:59,536 INFO  [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] service.QueryService:563
: The corrected query: select
 operator,
 count(*)
 from app_bi_user
 where operator = '中国电信'
 group by  operator
 LIMIT 50000
 2018-10-16 15:49:59,543 INFO  [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] routing.QueryRouter:58
: Find candidates by table APP.APP_BI_USER and project=BI : CUBE[name=BI_user]
 2018-10-16 15:49:59,543 INFO  [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] routing.QueryRouter:51
: Applying rule: class org.apache.kylin.query.routing.rules.RemoveBlackoutRealizationsRule,
 realizations before: [CUBE[name=BI_user]], realizations after: [CUBE[name=BI_user]]
 2018-10-16 15:49:59,544 INFO  [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] routing.QueryRouter:51
: Applying rule: class org.apache.kylin.query.routing.rules.RemoveUncapableRealizationsRule,
 realizations before: [CUBE[name=BI_user]], realizations after: [CUBE[name=BI_user]]
 2018-10-16 15:49:59,544 INFO  [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] rules.RealizationSortRule:40
: CUBE[name=BI_user] priority 1 cost 162. 
 2018-10-16 15:49:59,544 INFO  [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] routing.QueryRouter:51
: Applying rule: class org.apache.kylin.query.routing.rules.RealizationSortRule,  realizations
before: [CUBE[name=BI_user]], realizations after: [CUBE[name=BI_user]]
 2018-10-16 15:49:59,544 INFO  [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] routing.QueryRouter:75
: The realizations remaining: [CUBE[name=BI_user]],and the final chosen  one for current olap
context 0 is CUBE[name=BI_user]
 2018-10-16 15:49:59,589 DEBUG [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] enumerator.OLAPEnumerator:105
: query storage...
 2018-10-16 15:49:59,589 DEBUG [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] gtrecord.GTCubeStorageQueryBase:333
: Need storage aggregation
 2018-10-16 15:49:59,590 INFO  [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] gtrecord.GTCubeStorageQueryBase:566
: exactAggregation is false because need storage aggregation
 2018-10-16 15:49:59,590 INFO  [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] gtrecord.GTCubeStorageQueryBase:311
: Filter column set for query: [APP.APP_BI_USER.OPERATOR]
 2018-10-16 15:49:59,590 INFO  [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] gtrecord.GTCubeStorageQueryBase:320
: Filter mask is: 1
 2018-10-16 15:49:59,590 DEBUG [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] gtrecord.GTCubeStorageQueryBase:435
: storageLimitLevel set to LIMIT_ON_RETURN_SIZE because  groupD is not clustered at head,
groupsD: [APP.APP_BI_USER.OPERATOR] with cuboid columns: [APP.APP_BI_USER.REGIST_TIME, APP.APP_BI_USER.OPERATOR]
 2018-10-16 15:49:59,591 INFO  [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] storage.StorageContext:167
: Enabling limit push down: 50000 at level: LIMIT_ON_RETURN_SIZE
 2018-10-16 15:49:59,591 INFO  [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] gtrecord.GTCubeStorageQueryBase:191
: Cuboid identified: cube=BI_user, cuboidId=1025, groupsD=[APP.APP_BI_USER.OPERATOR],  filterD=[APP.APP_BI_USER.OPERATOR],
limitPushdown=50000, limitLevel=LIMIT_ON_RETURN_SIZE, storageAggr=true
 2018-10-16 15:49:59,591 DEBUG [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] common.SegmentPruner:89
: Prune segment BI_user[FULL_BUILD] due to given filter
 2018-10-16 15:49:59,591 DEBUG [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] enumerator.OLAPEnumerator:117
: return TupleIterator...
 2018-10-16 15:49:59,592 INFO  [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] service.QueryService:1050
: Processed rows for each storageContext: 0 
 2018-10-16 15:49:59,592 INFO  [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] service.QueryService:455
: Stats of SQL response: isException: false, duration: 59, total  scan count 0
 2018-10-16 15:49:59,592 DEBUG [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] util.CheckUtil:35
: query cache is disabled
 2018-10-16 15:49:59,593 INFO  [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] service.QueryService:344
: 
 ==========================[QUERY]===============================
 Query Id: 23edaa0f-428f-36ef-4aac-fccb32c6652f
 SQL: select
 operator,
 count(*)
 from app_bi_user
 where operator = '中国电信'
 group by  operator;
 User: ADMIN
 Success: true
 Duration: 0.06
 Project: BI
 Realization Names: [CUBE[name=BI_user]]
 Cuboid Ids: [1025]
 Total scan count: 0
 Total scan bytes: 0
 Result row count: 0
 Accept Partial: true
 Is Partial Result: false
 Hit Exception Cache: false
 Storage cache used: false
 Is Query Push-Down: false
 Is Prepare: false
 Trace URL: null
 Message: null
 ==========================[QUERY]===============================
 
 
 2018-10-16 15:50:03,911 INFO  [http-nio-7070-exec-13] service.QueryService:350 : Check query
permission in 0 ms.
 2018-10-16 15:50:03,911 DEBUG [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] util.CheckUtil:35
: query cache disabled in KylinConfig
 2018-10-16 15:50:03,912 INFO  [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] service.QueryService:386
: Using project: BI
 2018-10-16 15:50:03,912 INFO  [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] service.QueryService:387
: The original query:  select
 operator,
 count(*)
 from app_bi_user
 where operator != '中国电信'
 group by  operator;
 2018-10-16 15:50:03,914 INFO  [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] service.QueryService:563
: The corrected query: select
 operator,
 count(*)
 from app_bi_user
 where operator <> '中国电信'
 group by  operator
 LIMIT 50000
 2018-10-16 15:50:03,920 INFO  [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] routing.QueryRouter:58
: Find candidates by table APP.APP_BI_USER and project=BI : CUBE[name=BI_user]
 2018-10-16 15:50:03,920 INFO  [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] routing.QueryRouter:51
: Applying rule: class org.apache.kylin.query.routing.rules.RemoveBlackoutRealizationsRule,
 realizations before: [CUBE[name=BI_user]], realizations after: [CUBE[name=BI_user]]
 2018-10-16 15:50:03,920 INFO  [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] routing.QueryRouter:51
: Applying rule: class org.apache.kylin.query.routing.rules.RemoveUncapableRealizationsRule,
 realizations before: [CUBE[name=BI_user]], realizations after: [CUBE[name=BI_user]]
 2018-10-16 15:50:03,921 INFO  [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] rules.RealizationSortRule:40
: CUBE[name=BI_user] priority 1 cost 162. 
 2018-10-16 15:50:03,921 INFO  [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] routing.QueryRouter:51
: Applying rule: class org.apache.kylin.query.routing.rules.RealizationSortRule,  realizations
before: [CUBE[name=BI_user]], realizations after: [CUBE[name=BI_user]]
 2018-10-16 15:50:03,921 INFO  [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] routing.QueryRouter:75
: The realizations remaining: [CUBE[name=BI_user]],and the final chosen  one for current olap
context 0 is CUBE[name=BI_user]
 2018-10-16 15:50:03,949 DEBUG [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] enumerator.OLAPEnumerator:105
: query storage...
 2018-10-16 15:50:03,949 DEBUG [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] gtrecord.GTCubeStorageQueryBase:333
: Need storage aggregation
 2018-10-16 15:50:03,949 INFO  [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] gtrecord.GTCubeStorageQueryBase:566
: exactAggregation is false because need storage aggregation
 2018-10-16 15:50:03,950 INFO  [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] gtrecord.GTCubeStorageQueryBase:311
: Filter column set for query: [APP.APP_BI_USER.OPERATOR]
 2018-10-16 15:50:03,950 INFO  [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] gtrecord.GTCubeStorageQueryBase:320
: Filter mask is: 1
 2018-10-16 15:50:03,950 DEBUG [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] gtrecord.GTCubeStorageQueryBase:435
: storageLimitLevel set to LIMIT_ON_RETURN_SIZE because  groupD is not clustered at head,
groupsD: [APP.APP_BI_USER.OPERATOR] with cuboid columns: [APP.APP_BI_USER.REGIST_TIME, APP.APP_BI_USER.OPERATOR]
 2018-10-16 15:50:03,951 INFO  [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] storage.StorageContext:167
: Enabling limit push down: 50000 at level: LIMIT_ON_RETURN_SIZE
 2018-10-16 15:50:03,951 INFO  [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] gtrecord.GTCubeStorageQueryBase:191
: Cuboid identified: cube=BI_user, cuboidId=1025, groupsD=[APP.APP_BI_USER.OPERATOR],  filterD=[APP.APP_BI_USER.OPERATOR],
limitPushdown=50000, limitLevel=LIMIT_ON_RETURN_SIZE, storageAggr=true
 2018-10-16 15:50:03,951 DEBUG [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] common.SegmentPruner:94
: Pruner passed on segment BI_user[FULL_BUILD]
 2018-10-16 15:50:03,951 INFO  [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] gtrecord.CubeSegmentScanner:61
: Init CubeSegmentScanner for segment FULL_BUILD
 2018-10-16 15:50:03,953 DEBUG [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] v2.CubeHBaseRPC:315
: hbase.rpc.timeout = 60000 ms, use 54000 ms as timeout for coprocessor
 2018-10-16 15:50:03,954 INFO  [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] v2.CubeHBaseEndpointRPC:141
: Serialized scanRequestBytes 523 bytes, rawScanBytesString 44  bytes
 2018-10-16 15:50:03,954 INFO  [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] v2.CubeHBaseEndpointRPC:143
: The scan 17cd05fa for segment BI_user[FULL_BUILD] is as below  with 1 separate raw scans,
shard part of start/end key is set to 0
 2018-10-16 15:50:03,954 INFO  [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] v2.CubeHBaseRPC:288
: Visiting hbase table KYLIN_LCGEU7PJ2E: cuboid require post aggregation,  from 1 to 1025
Start: \x00\x00\x00\x00\x00\x00\x00\x00\x04\x01\x00\x00\x00 (\x00\x00\x00\x00\x00\x00\x00\x00\x04\x01\x00\x00\x00)
Stop:  \x00\x00\x00\x00\x00\x00\x00\x00\x04\x01\xFF\xFF\xFF\x00 (\x00\x00\x00\x00\x00\x00\x00\x00\x04\x01\xFF\xFF\xFF\x00),
No  Fuzzy Key
 2018-10-16 15:50:03,955 DEBUG [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] v2.CubeHBaseEndpointRPC:148
: Submitting rpc to 1 shards starting from shard 0, scan range  count 1
 2018-10-16 15:50:03,955 INFO  [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] common.KylinConfig:332
: Loading kylin-defaults.properties from file:/usr/local/kylin/tomcat/webapps/kylin/WEB-INF/lib/kylin-core-common-2.5.0.jar!/kylin-defaults.properties
 2018-10-16 15:50:03,964 DEBUG [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] common.KylinConfig:291
: KYLIN_CONF property was not set, will seek KYLIN_HOME env variable
 2018-10-16 15:50:03,965 INFO  [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] gtrecord.SequentialCubeTupleIterator:73
: Using SortedIteratorMergerWithLimit to merge segment  results
 2018-10-16 15:50:03,965 DEBUG [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] enumerator.OLAPEnumerator:117
: return TupleIterator...
 2018-10-16 15:50:03,985 INFO  [kylin-coproc--pool6-t1369] v2.CubeHBaseEndpointRPC:203 : <sub-thread
for Query b32876d8-8bdc-dc67-1142-4769216778cd GTScanRequest 17cd05fa>Endpoint  RPC returned
from HTable KYLIN_LCGEU7PJ2E Shard \x4B\x59\x4C\x49\x4E\x5F\x4C\x43\x47\x45\x55\x37\x50\x4A\x32\x45\x2C\x2C\x31\x35\x33\x39\x36\x37\x30\x34\x30\x34\x30\x31\x30\x2E\x33\x35\x39\x32\x32\x36\x66\x38\x31\x34\x34\x64\x64\x61\x36\x33\x35\x35\x61\x37\x35\x37\x36\x64\x30\x39\x65\x36\x37\x61\x61\x62\x2E
 on host: prod-zeus-worker06.jdfq.com.Total scanned row: 1458. Total scanned bytes: 61836.
Total filtered row: 402. Total aggred row: 1052. Time elapsed in EP: 7(ms). Server CPU usage:
0.0011321913339388744, server physical mem left: 6.8799967232E10, server  swap mem left:4.2949632E9.Etc
message: start latency: 22@0,agg done@6,compress done@7,server stats done@7, debugGitTag:ac017f4f24534ccf44c65229e68b513bc7bd2755;.Normal
Complete: true.Compressed row size: 26
 2018-10-16 15:50:03,985 DEBUG [kylin-coproc--pool6-t1369] util.CompressionUtils:67 : Original:
26 bytes. Decompressed: 18 bytes. Time: 0
 2018-10-16 15:50:03,985 DEBUG [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] gtrecord.SortMergedPartitionResultIterator:76
: Using SortMergedPartitionResultIterator to  merge 1 partition results out of 1 partitions
 2018-10-16 15:50:03,987 INFO  [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] service.QueryService:1050
: Processed rows for each storageContext: 4 
 2018-10-16 15:50:03,988 INFO  [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] service.QueryService:455
: Stats of SQL response: isException: false, duration: 77, total  scan count 1458
 2018-10-16 15:50:03,988 DEBUG [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] util.CheckUtil:35
: query cache is disabled
 2018-10-16 15:50:03,988 INFO  [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] service.QueryService:344
: 
 ==========================[QUERY]===============================
 Query Id: b32876d8-8bdc-dc67-1142-4769216778cd
 SQL: select
 operator,
 count(*)
 from app_bi_user
 where operator != '中国电信'
 group by  operator;
 User: ADMIN
 Success: true
 Duration: 0.077
 Project: BI
 Realization Names: [CUBE[name=BI_user]]
 Cuboid Ids: [1025]
 Total scan count: 1458
 Total scan bytes: 61836
 Result row count: 4
 Accept Partial: true
 Is Partial Result: false
 Hit Exception Cache: false
 Storage cache used: false
 Is Query Push-Down: false
 Is Prepare: false
 Trace URL: null
 Message: null
 ==========================[QUERY]===============================
 
 
 2018-10-16 15:50:09,975 INFO  [http-nio-7070-exec-19] service.QueryService:350 : Check query
permission in 0 ms.
 2018-10-16 15:50:09,975 DEBUG [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] util.CheckUtil:35
: query cache disabled in KylinConfig
 2018-10-16 15:50:09,976 INFO  [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] service.QueryService:386
: Using project: BI
 2018-10-16 15:50:09,976 INFO  [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] service.QueryService:387
: The original query:  select
 operator,
 count(*)
 from app_bi_user
 group by  operator;
 2018-10-16 15:50:09,978 INFO  [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] service.QueryService:563
: The corrected query: select
 operator,
 count(*)
 from app_bi_user
 group by  operator
 LIMIT 50000
 2018-10-16 15:50:09,983 INFO  [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] routing.QueryRouter:58
: Find candidates by table APP.APP_BI_USER and project=BI : CUBE[name=BI_user]
 2018-10-16 15:50:09,983 INFO  [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] routing.QueryRouter:51
: Applying rule: class org.apache.kylin.query.routing.rules.RemoveBlackoutRealizationsRule,
 realizations before: [CUBE[name=BI_user]], realizations after: [CUBE[name=BI_user]]
 2018-10-16 15:50:09,983 INFO  [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] routing.QueryRouter:51
: Applying rule: class org.apache.kylin.query.routing.rules.RemoveUncapableRealizationsRule,
 realizations before: [CUBE[name=BI_user]], realizations after: [CUBE[name=BI_user]]
 2018-10-16 15:50:09,984 INFO  [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] rules.RealizationSortRule:40
: CUBE[name=BI_user] priority 1 cost 162. 
 2018-10-16 15:50:09,984 INFO  [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] routing.QueryRouter:51
: Applying rule: class org.apache.kylin.query.routing.rules.RealizationSortRule,  realizations
before: [CUBE[name=BI_user]], realizations after: [CUBE[name=BI_user]]
 2018-10-16 15:50:09,984 INFO  [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] routing.QueryRouter:75
: The realizations remaining: [CUBE[name=BI_user]],and the final chosen  one for current olap
context 0 is CUBE[name=BI_user]
 2018-10-16 15:50:10,028 DEBUG [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] enumerator.OLAPEnumerator:105
: query storage...
 2018-10-16 15:50:10,028 DEBUG [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] gtrecord.GTCubeStorageQueryBase:333
: Need storage aggregation
 2018-10-16 15:50:10,028 INFO  [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] gtrecord.GTCubeStorageQueryBase:566
: exactAggregation is false because need storage aggregation
 2018-10-16 15:50:10,029 INFO  [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] gtrecord.GTCubeStorageQueryBase:311
: Filter column set for query: []
 2018-10-16 15:50:10,029 INFO  [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] gtrecord.GTCubeStorageQueryBase:320
: Filter mask is: 0
 2018-10-16 15:50:10,029 DEBUG [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] gtrecord.GTCubeStorageQueryBase:435
: storageLimitLevel set to LIMIT_ON_RETURN_SIZE because  groupD is not clustered at head,
groupsD: [APP.APP_BI_USER.OPERATOR] with cuboid columns: [APP.APP_BI_USER.REGIST_TIME, APP.APP_BI_USER.OPERATOR]
 2018-10-16 15:50:10,029 INFO  [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] storage.StorageContext:167
: Enabling limit push down: 50000 at level: LIMIT_ON_RETURN_SIZE
 2018-10-16 15:50:10,030 INFO  [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] gtrecord.GTCubeStorageQueryBase:191
: Cuboid identified: cube=BI_user, cuboidId=1025, groupsD=[APP.APP_BI_USER.OPERATOR],  filterD=[],
limitPushdown=50000, limitLevel=LIMIT_ON_RETURN_SIZE, storageAggr=true
 2018-10-16 15:50:10,030 DEBUG [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] common.SegmentPruner:94
: Pruner passed on segment BI_user[FULL_BUILD]
 2018-10-16 15:50:10,030 INFO  [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] gtrecord.CubeSegmentScanner:61
: Init CubeSegmentScanner for segment FULL_BUILD
 2018-10-16 15:50:10,032 DEBUG [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] v2.CubeHBaseRPC:315
: hbase.rpc.timeout = 60000 ms, use 54000 ms as timeout for coprocessor
 2018-10-16 15:50:10,032 INFO  [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] v2.CubeHBaseEndpointRPC:141
: Serialized scanRequestBytes 474 bytes, rawScanBytesString 44  bytes
 2018-10-16 15:50:10,033 INFO  [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] v2.CubeHBaseEndpointRPC:143
: The scan 5f252d05 for segment BI_user[FULL_BUILD] is as below  with 1 separate raw scans,
shard part of start/end key is set to 0
 2018-10-16 15:50:10,033 INFO  [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] v2.CubeHBaseRPC:288
: Visiting hbase table KYLIN_LCGEU7PJ2E: cuboid require post aggregation,  from 1 to 1025
Start: \x00\x00\x00\x00\x00\x00\x00\x00\x04\x01\x00\x00\x00 (\x00\x00\x00\x00\x00\x00\x00\x00\x04\x01\x00\x00\x00)
Stop:  \x00\x00\x00\x00\x00\x00\x00\x00\x04\x01\xFF\xFF\xFF\x00 (\x00\x00\x00\x00\x00\x00\x00\x00\x04\x01\xFF\xFF\xFF\x00),
No  Fuzzy Key
 2018-10-16 15:50:10,033 DEBUG [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] v2.CubeHBaseEndpointRPC:148
: Submitting rpc to 1 shards starting from shard 0, scan range  count 1
 2018-10-16 15:50:10,034 INFO  [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] common.KylinConfig:332
: Loading kylin-defaults.properties from file:/usr/local/kylin/tomcat/webapps/kylin/WEB-INF/lib/kylin-core-common-2.5.0.jar!/kylin-defaults.properties
 2018-10-16 15:50:10,042 DEBUG [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] common.KylinConfig:291
: KYLIN_CONF property was not set, will seek KYLIN_HOME env variable
 2018-10-16 15:50:10,042 INFO  [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] gtrecord.SequentialCubeTupleIterator:73
: Using SortedIteratorMergerWithLimit to merge segment  results
 2018-10-16 15:50:10,043 DEBUG [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] enumerator.OLAPEnumerator:117
: return TupleIterator...
 2018-10-16 15:50:10,055 INFO  [kylin-coproc--pool6-t1370] v2.CubeHBaseEndpointRPC:203 : <sub-thread
for Query 76caaa08-bb31-f06b-d1ac-f575a983b06f GTScanRequest 5f252d05>Endpoint  RPC returned
from HTable KYLIN_LCGEU7PJ2E Shard \x4B\x59\x4C\x49\x4E\x5F\x4C\x43\x47\x45\x55\x37\x50\x4A\x32\x45\x2C\x2C\x31\x35\x33\x39\x36\x37\x30\x34\x30\x34\x30\x31\x30\x2E\x33\x35\x39\x32\x32\x36\x66\x38\x31\x34\x34\x64\x64\x61\x36\x33\x35\x35\x61\x37\x35\x37\x36\x64\x30\x39\x65\x36\x37\x61\x61\x62\x2E
 on host: prod-zeus-worker06.jdfq.com.Total scanned row: 1458. Total scanned bytes: 61836.
Total filtered row: 0. Total aggred row: 1453. Time elapsed in EP: 7(ms). Server CPU usage:
6.69361274875782E-4, server physical mem left: 6.8800565248E10, server swap  mem left:4.2949632E9.Etc
message: start latency: 15@0,agg done@6,compress done@6,server stats done@7, debugGitTag:ac017f4f24534ccf44c65229e68b513bc7bd2755;.Normal
Complete: true.Compressed row size: 32
 2018-10-16 15:50:10,055 DEBUG [kylin-coproc--pool6-t1370] util.CompressionUtils:67 : Original:
32 bytes. Decompressed: 23 bytes. Time: 0
 2018-10-16 15:50:10,055 DEBUG [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] gtrecord.SortMergedPartitionResultIterator:76
: Using SortMergedPartitionResultIterator to  merge 1 partition results out of 1 partitions
 2018-10-16 15:50:10,057 INFO  [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] service.QueryService:1050
: Processed rows for each storageContext: 5 
 2018-10-16 15:50:10,058 INFO  [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] service.QueryService:455
: Stats of SQL response: isException: false, duration: 83, total  scan count 1458
 2018-10-16 15:50:10,058 DEBUG [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] util.CheckUtil:35
: query cache is disabled
 2018-10-16 15:50:10,058 INFO  [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] service.QueryService:344
: 
 ==========================[QUERY]===============================
 Query Id: 76caaa08-bb31-f06b-d1ac-f575a983b06f
 SQL: select
 operator,
 count(*)
 from app_bi_user
 group by  operator;
 User: ADMIN
 Success: true
 Duration: 0.083
 Project: BI
 Realization Names: [CUBE[name=BI_user]]
 Cuboid Ids: [1025]
 Total scan count: 1458
 Total scan bytes: 61836
 Result row count: 5
 Accept Partial: true
 Is Partial Result: false
 Hit Exception Cache: false
 Storage cache used: false
 Is Query Push-Down: false
 Is Prepare: false
 Trace URL: null
 Message: null
 ==========================[QUERY]===============================
 
 
 2018-10-16 15:50:12,350 INFO  [FetcherRunner 1802655599-356] threadpool.DefaultFetcherRunner:96
: Job Fetcher: 0 should running, 0 actual running, 0 stopped, 0 ready, 7911  already succeed,
0 error, 100 discarded, 0 others
 2018-10-16 15:50:12,415 INFO  [http-nio-7070-exec-14] common.KylinConfig:332 : Loading kylin-defaults.properties
from file:/usr/local/kylin/tomcat/webapps/kylin/WEB-INF/lib/kylin-core-common-2.5.0.jar!/kylin-defaults.properties
 2018-10-16 15:50:12,425 DEBUG [http-nio-7070-exec-14] common.KylinConfig:291 : KYLIN_CONF
property was not set, will seek KYLIN_HOME env variable
  
 
 
 ------------------
   
  
 郭锋
 

 
 百乘金科/金蛋分期/技术部/数据开发组
 
 北京市朝阳区北辰世纪中心B座17层
 
 
 
 
  
   
  
  ------------------ Original ------------------
  From:  "Na Zhai"<na.zhai@kyligence.io>;
 Date:  Tue, Oct 16, 2018 03:17 PM
 To:  "user@kylin.apache.org"<user@kylin.apache.org>; 
 
 Subject:  答复: Re:答复: 【kylin2.5.0】中文查询结果问题
 
  
   
 Hi,郭锋
  Do you use apache-kylin-2.5.0-bin-cdh60.tar.gz? And can you provide the log information?

 
  
 
  
 
 发送自 Windows 10 版邮件应用
 
  
 
 
 发件人: 郭锋 <guofeng@bcpeople.com>
 发送时间: Tuesday, October 16, 2018 2:41:52 PM
 收件人: user
 主题: Re:答复: 【kylin2.5.0】中文查询结果问题  
 
  Hi,
      The three scenarios  as follows:
     
        1、when i use operator='中国电信',it return 0;         2、when i use operator
!='中国电信',it return correctly;        3、when i don't user where clause, it return
correctly;
  
              
                                                                       
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 ------------------
   
  
 郭锋
 

 
 百乘金科/金蛋分期/技术部/数据开发组
 
 北京市朝阳区北辰世纪中心B座17层
 
 
 
 
  
   
  
  ------------------ Original ------------------
  From:  "郭锋"<guofeng@bcpeople.com>;
 Date:  Tue, Oct 16, 2018 11:41 AM
 To:  "user"<user@kylin.apache.org>; 
 
 Subject:  Re:答复: 【kylin2.5.0】中文查询结果问题
 
  
  HI,  
       图片已添加到邮件附件
 
 
       谢谢!
 
 
  
 
 
 
 ------------------
   
  
 郭锋
 

 
 百乘金科/金蛋分期/技术部/数据开发组
 
 北京市朝阳区北辰世纪中心B座17层
 
 
 
 
  
   
  
  ------------------ Original ------------------
  From:  "Na Zhai"<na.zhai@kyligence.io>;
 Date:  Tue, Oct 16, 2018 11:16 AM
 To:  "user@kylin.apache.org"<user@kylin.apache.org>; 
 
 Subject:  答复: 【kylin2.5.0】中文查询结果问题
 
  
   
 Hi, guofeng
 
 the picture is not displayed, can you resend them? Does it happen accidentally?
 
  
 
 发送自 Windows 10 版邮件应用
 
  
  
 发件人: 郭锋
 发送时间: 2018年10月15日 19:41
 收件人: user
 主题: 【kylin2.5.0】中文查询结果问题
 
 
  
 
 hi all: 
  
        kylin 查询条件中包含中文,返回的查询结果有问题:
 
  
        现象:在where语句中用 “=” 返回结果为0;
 
  
                 在where语句中用 “like”时,返回正确结果;
 
  
  
 
  
        版本信息:Kylin的版本为2.5.0 , 部署在cdh6.0.0 上
 
  
        请问,是否遇到过这种情况,有什么结果方法,谢谢!
 
  
        截图如下
Mime
View raw message