kylin-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "49188074@qq.com" <49188...@qq.com>
Subject 回复: 答复: 【kylin2.5.0】中文查询结果问题
Date Wed, 17 Oct 2018 03:38:43 GMT
Hi, Na Zhai

    The attachment is Cube's JSON. Thanks



49188074@qq.com
 
发件人: Na Zhai
发送时间: 2018-10-17 10:52
收件人: user@kylin.apache.org
主题: 答复: 答复: 【kylin2.5.0】中文查询结果问题
Hi, 郭锋
Dict is ok. I can not reproduce this problem. So can you provide more information? Such as
your Cube's JSON file or other info that you can provide to help me reproduce this problem.
Thanks
 
 
发送自 Windows 10 版邮件应用
 


发件人: 49188074@qq.com <49188074@qq.com>
发送时间: Wednesday, October 17, 2018 10:31:27 AM
收件人: user
主题: 回复: 答复: 【kylin2.5.0】中文查询结果问题 
 
Hi,Na Zhai

   The encoding of the 'operator' field is Dict, which type should be used?



49188074@qq.com
 
发件人: Na Zhai
发送时间: 2018-10-16 19:32
收件人: user@kylin.apache.org
主题: 答复: Re:答复: Re:答复: 【kylin2.5.0】中文查询结果问题
Hi,郭锋
What is the encoding of the ‘operator‘ field? Dict or fixed_length or others?
 
发送自 Windows 10 版邮件应用
 


发件人: 郭锋 <guofeng@bcpeople.com>
发送时间: Tuesday, October 16, 2018 3:56:28 PM
收件人: user
主题: Re:答复: Re:答复: 【kylin2.5.0】中文查询结果问题 
 
Hi,
     yes, we use the apache-kylin-2.5.0-bin-cdh60.tar.gz version on cdh6.0.0 cluster;
     the before three select clause log as follows:


2018-10-16 15:49:59,532 INFO  [http-nio-7070-exec-18] service.QueryService:350 : Check query
permission in 0 ms.
2018-10-16 15:49:59,533 DEBUG [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] util.CheckUtil:35
: query cache disabled in KylinConfig
2018-10-16 15:49:59,533 INFO  [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] service.QueryService:386
: Using project: BI
2018-10-16 15:49:59,534 INFO  [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] service.QueryService:387
: The original query:  select
operator,
count(*)
from app_bi_user
where operator = '中国电信'
group by  operator;
2018-10-16 15:49:59,536 INFO  [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] service.QueryService:563
: The corrected query: select
operator,
count(*)
from app_bi_user
where operator = '中国电信'
group by  operator
LIMIT 50000
2018-10-16 15:49:59,543 INFO  [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] routing.QueryRouter:58
: Find candidates by table APP.APP_BI_USER and project=BI : CUBE[name=BI_user]
2018-10-16 15:49:59,543 INFO  [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] routing.QueryRouter:51
: Applying rule: class org.apache.kylin.query.routing.rules.RemoveBlackoutRealizationsRule,
realizations before: [CUBE[name=BI_user]], realizations after: [CUBE[name=BI_user]]
2018-10-16 15:49:59,544 INFO  [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] routing.QueryRouter:51
: Applying rule: class org.apache.kylin.query.routing.rules.RemoveUncapableRealizationsRule,
realizations before: [CUBE[name=BI_user]], realizations after: [CUBE[name=BI_user]]
2018-10-16 15:49:59,544 INFO  [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] rules.RealizationSortRule:40
: CUBE[name=BI_user] priority 1 cost 162. 
2018-10-16 15:49:59,544 INFO  [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] routing.QueryRouter:51
: Applying rule: class org.apache.kylin.query.routing.rules.RealizationSortRule, realizations
before: [CUBE[name=BI_user]], realizations after: [CUBE[name=BI_user]]
2018-10-16 15:49:59,544 INFO  [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] routing.QueryRouter:75
: The realizations remaining: [CUBE[name=BI_user]],and the final chosen one for current olap
context 0 is CUBE[name=BI_user]
2018-10-16 15:49:59,589 DEBUG [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] enumerator.OLAPEnumerator:105
: query storage...
2018-10-16 15:49:59,589 DEBUG [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] gtrecord.GTCubeStorageQueryBase:333
: Need storage aggregation
2018-10-16 15:49:59,590 INFO  [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] gtrecord.GTCubeStorageQueryBase:566
: exactAggregation is false because need storage aggregation
2018-10-16 15:49:59,590 INFO  [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] gtrecord.GTCubeStorageQueryBase:311
: Filter column set for query: [APP.APP_BI_USER.OPERATOR]
2018-10-16 15:49:59,590 INFO  [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] gtrecord.GTCubeStorageQueryBase:320
: Filter mask is: 1
2018-10-16 15:49:59,590 DEBUG [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] gtrecord.GTCubeStorageQueryBase:435
: storageLimitLevel set to LIMIT_ON_RETURN_SIZE because groupD is not clustered at head, groupsD:
[APP.APP_BI_USER.OPERATOR] with cuboid columns: [APP.APP_BI_USER.REGIST_TIME, APP.APP_BI_USER.OPERATOR]
2018-10-16 15:49:59,591 INFO  [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] storage.StorageContext:167
: Enabling limit push down: 50000 at level: LIMIT_ON_RETURN_SIZE
2018-10-16 15:49:59,591 INFO  [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] gtrecord.GTCubeStorageQueryBase:191
: Cuboid identified: cube=BI_user, cuboidId=1025, groupsD=[APP.APP_BI_USER.OPERATOR], filterD=[APP.APP_BI_USER.OPERATOR],
limitPushdown=50000, limitLevel=LIMIT_ON_RETURN_SIZE, storageAggr=true
2018-10-16 15:49:59,591 DEBUG [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] common.SegmentPruner:89
: Prune segment BI_user[FULL_BUILD] due to given filter
2018-10-16 15:49:59,591 DEBUG [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] enumerator.OLAPEnumerator:117
: return TupleIterator...
2018-10-16 15:49:59,592 INFO  [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] service.QueryService:1050
: Processed rows for each storageContext: 0 
2018-10-16 15:49:59,592 INFO  [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] service.QueryService:455
: Stats of SQL response: isException: false, duration: 59, total scan count 0
2018-10-16 15:49:59,592 DEBUG [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] util.CheckUtil:35
: query cache is disabled
2018-10-16 15:49:59,593 INFO  [Query 23edaa0f-428f-36ef-4aac-fccb32c6652f-72407] service.QueryService:344
: 
==========================[QUERY]===============================
Query Id: 23edaa0f-428f-36ef-4aac-fccb32c6652f
SQL: select
operator,
count(*)
from app_bi_user
where operator = '中国电信'
group by  operator;
User: ADMIN
Success: true
Duration: 0.06
Project: BI
Realization Names: [CUBE[name=BI_user]]
Cuboid Ids: [1025]
Total scan count: 0
Total scan bytes: 0
Result row count: 0
Accept Partial: true
Is Partial Result: false
Hit Exception Cache: false
Storage cache used: false
Is Query Push-Down: false
Is Prepare: false
Trace URL: null
Message: null
==========================[QUERY]===============================

2018-10-16 15:50:03,911 INFO  [http-nio-7070-exec-13] service.QueryService:350 : Check query
permission in 0 ms.
2018-10-16 15:50:03,911 DEBUG [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] util.CheckUtil:35
: query cache disabled in KylinConfig
2018-10-16 15:50:03,912 INFO  [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] service.QueryService:386
: Using project: BI
2018-10-16 15:50:03,912 INFO  [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] service.QueryService:387
: The original query:  select
operator,
count(*)
from app_bi_user
where operator != '中国电信'
group by  operator;
2018-10-16 15:50:03,914 INFO  [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] service.QueryService:563
: The corrected query: select
operator,
count(*)
from app_bi_user
where operator <> '中国电信'
group by  operator
LIMIT 50000
2018-10-16 15:50:03,920 INFO  [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] routing.QueryRouter:58
: Find candidates by table APP.APP_BI_USER and project=BI : CUBE[name=BI_user]
2018-10-16 15:50:03,920 INFO  [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] routing.QueryRouter:51
: Applying rule: class org.apache.kylin.query.routing.rules.RemoveBlackoutRealizationsRule,
realizations before: [CUBE[name=BI_user]], realizations after: [CUBE[name=BI_user]]
2018-10-16 15:50:03,920 INFO  [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] routing.QueryRouter:51
: Applying rule: class org.apache.kylin.query.routing.rules.RemoveUncapableRealizationsRule,
realizations before: [CUBE[name=BI_user]], realizations after: [CUBE[name=BI_user]]
2018-10-16 15:50:03,921 INFO  [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] rules.RealizationSortRule:40
: CUBE[name=BI_user] priority 1 cost 162. 
2018-10-16 15:50:03,921 INFO  [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] routing.QueryRouter:51
: Applying rule: class org.apache.kylin.query.routing.rules.RealizationSortRule, realizations
before: [CUBE[name=BI_user]], realizations after: [CUBE[name=BI_user]]
2018-10-16 15:50:03,921 INFO  [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] routing.QueryRouter:75
: The realizations remaining: [CUBE[name=BI_user]],and the final chosen one for current olap
context 0 is CUBE[name=BI_user]
2018-10-16 15:50:03,949 DEBUG [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] enumerator.OLAPEnumerator:105
: query storage...
2018-10-16 15:50:03,949 DEBUG [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] gtrecord.GTCubeStorageQueryBase:333
: Need storage aggregation
2018-10-16 15:50:03,949 INFO  [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] gtrecord.GTCubeStorageQueryBase:566
: exactAggregation is false because need storage aggregation
2018-10-16 15:50:03,950 INFO  [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] gtrecord.GTCubeStorageQueryBase:311
: Filter column set for query: [APP.APP_BI_USER.OPERATOR]
2018-10-16 15:50:03,950 INFO  [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] gtrecord.GTCubeStorageQueryBase:320
: Filter mask is: 1
2018-10-16 15:50:03,950 DEBUG [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] gtrecord.GTCubeStorageQueryBase:435
: storageLimitLevel set to LIMIT_ON_RETURN_SIZE because groupD is not clustered at head, groupsD:
[APP.APP_BI_USER.OPERATOR] with cuboid columns: [APP.APP_BI_USER.REGIST_TIME, APP.APP_BI_USER.OPERATOR]
2018-10-16 15:50:03,951 INFO  [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] storage.StorageContext:167
: Enabling limit push down: 50000 at level: LIMIT_ON_RETURN_SIZE
2018-10-16 15:50:03,951 INFO  [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] gtrecord.GTCubeStorageQueryBase:191
: Cuboid identified: cube=BI_user, cuboidId=1025, groupsD=[APP.APP_BI_USER.OPERATOR], filterD=[APP.APP_BI_USER.OPERATOR],
limitPushdown=50000, limitLevel=LIMIT_ON_RETURN_SIZE, storageAggr=true
2018-10-16 15:50:03,951 DEBUG [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] common.SegmentPruner:94
: Pruner passed on segment BI_user[FULL_BUILD]
2018-10-16 15:50:03,951 INFO  [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] gtrecord.CubeSegmentScanner:61
: Init CubeSegmentScanner for segment FULL_BUILD
2018-10-16 15:50:03,953 DEBUG [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] v2.CubeHBaseRPC:315
: hbase.rpc.timeout = 60000 ms, use 54000 ms as timeout for coprocessor
2018-10-16 15:50:03,954 INFO  [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] v2.CubeHBaseEndpointRPC:141
: Serialized scanRequestBytes 523 bytes, rawScanBytesString 44 bytes
2018-10-16 15:50:03,954 INFO  [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] v2.CubeHBaseEndpointRPC:143
: The scan 17cd05fa for segment BI_user[FULL_BUILD] is as below with 1 separate raw scans,
shard part of start/end key is set to 0
2018-10-16 15:50:03,954 INFO  [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] v2.CubeHBaseRPC:288
: Visiting hbase table KYLIN_LCGEU7PJ2E: cuboid require post aggregation, from 1 to 1025 Start:
\x00\x00\x00\x00\x00\x00\x00\x00\x04\x01\x00\x00\x00 (\x00\x00\x00\x00\x00\x00\x00\x00\x04\x01\x00\x00\x00)
Stop:  \x00\x00\x00\x00\x00\x00\x00\x00\x04\x01\xFF\xFF\xFF\x00 (\x00\x00\x00\x00\x00\x00\x00\x00\x04\x01\xFF\xFF\xFF\x00),
No Fuzzy Key
2018-10-16 15:50:03,955 DEBUG [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] v2.CubeHBaseEndpointRPC:148
: Submitting rpc to 1 shards starting from shard 0, scan range count 1
2018-10-16 15:50:03,955 INFO  [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] common.KylinConfig:332
: Loading kylin-defaults.properties from file:/usr/local/kylin/tomcat/webapps/kylin/WEB-INF/lib/kylin-core-common-2.5.0.jar!/kylin-defaults.properties
2018-10-16 15:50:03,964 DEBUG [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] common.KylinConfig:291
: KYLIN_CONF property was not set, will seek KYLIN_HOME env variable
2018-10-16 15:50:03,965 INFO  [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] gtrecord.SequentialCubeTupleIterator:73
: Using SortedIteratorMergerWithLimit to merge segment results
2018-10-16 15:50:03,965 DEBUG [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] enumerator.OLAPEnumerator:117
: return TupleIterator...
2018-10-16 15:50:03,985 INFO  [kylin-coproc--pool6-t1369] v2.CubeHBaseEndpointRPC:203 : <sub-thread
for Query b32876d8-8bdc-dc67-1142-4769216778cd GTScanRequest 17cd05fa>Endpoint RPC returned
from HTable KYLIN_LCGEU7PJ2E Shard \x4B\x59\x4C\x49\x4E\x5F\x4C\x43\x47\x45\x55\x37\x50\x4A\x32\x45\x2C\x2C\x31\x35\x33\x39\x36\x37\x30\x34\x30\x34\x30\x31\x30\x2E\x33\x35\x39\x32\x32\x36\x66\x38\x31\x34\x34\x64\x64\x61\x36\x33\x35\x35\x61\x37\x35\x37\x36\x64\x30\x39\x65\x36\x37\x61\x61\x62\x2E
on host: prod-zeus-worker06.jdfq.com.Total scanned row: 1458. Total scanned bytes: 61836.
Total filtered row: 402. Total aggred row: 1052. Time elapsed in EP: 7(ms). Server CPU usage:
0.0011321913339388744, server physical mem left: 6.8799967232E10, server swap mem left:4.2949632E9.Etc
message: start latency: 22@0,agg done@6,compress done@7,server stats done@7, debugGitTag:ac017f4f24534ccf44c65229e68b513bc7bd2755;.Normal
Complete: true.Compressed row size: 26
2018-10-16 15:50:03,985 DEBUG [kylin-coproc--pool6-t1369] util.CompressionUtils:67 : Original:
26 bytes. Decompressed: 18 bytes. Time: 0
2018-10-16 15:50:03,985 DEBUG [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] gtrecord.SortMergedPartitionResultIterator:76
: Using SortMergedPartitionResultIterator to merge 1 partition results out of 1 partitions
2018-10-16 15:50:03,987 INFO  [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] service.QueryService:1050
: Processed rows for each storageContext: 4 
2018-10-16 15:50:03,988 INFO  [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] service.QueryService:455
: Stats of SQL response: isException: false, duration: 77, total scan count 1458
2018-10-16 15:50:03,988 DEBUG [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] util.CheckUtil:35
: query cache is disabled
2018-10-16 15:50:03,988 INFO  [Query b32876d8-8bdc-dc67-1142-4769216778cd-50477] service.QueryService:344
: 
==========================[QUERY]===============================
Query Id: b32876d8-8bdc-dc67-1142-4769216778cd
SQL: select
operator,
count(*)
from app_bi_user
where operator != '中国电信'
group by  operator;
User: ADMIN
Success: true
Duration: 0.077
Project: BI
Realization Names: [CUBE[name=BI_user]]
Cuboid Ids: [1025]
Total scan count: 1458
Total scan bytes: 61836
Result row count: 4
Accept Partial: true
Is Partial Result: false
Hit Exception Cache: false
Storage cache used: false
Is Query Push-Down: false
Is Prepare: false
Trace URL: null
Message: null
==========================[QUERY]===============================

2018-10-16 15:50:09,975 INFO  [http-nio-7070-exec-19] service.QueryService:350 : Check query
permission in 0 ms.
2018-10-16 15:50:09,975 DEBUG [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] util.CheckUtil:35
: query cache disabled in KylinConfig
2018-10-16 15:50:09,976 INFO  [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] service.QueryService:386
: Using project: BI
2018-10-16 15:50:09,976 INFO  [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] service.QueryService:387
: The original query:  select
operator,
count(*)
from app_bi_user
group by  operator;
2018-10-16 15:50:09,978 INFO  [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] service.QueryService:563
: The corrected query: select
operator,
count(*)
from app_bi_user
group by  operator
LIMIT 50000
2018-10-16 15:50:09,983 INFO  [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] routing.QueryRouter:58
: Find candidates by table APP.APP_BI_USER and project=BI : CUBE[name=BI_user]
2018-10-16 15:50:09,983 INFO  [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] routing.QueryRouter:51
: Applying rule: class org.apache.kylin.query.routing.rules.RemoveBlackoutRealizationsRule,
realizations before: [CUBE[name=BI_user]], realizations after: [CUBE[name=BI_user]]
2018-10-16 15:50:09,983 INFO  [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] routing.QueryRouter:51
: Applying rule: class org.apache.kylin.query.routing.rules.RemoveUncapableRealizationsRule,
realizations before: [CUBE[name=BI_user]], realizations after: [CUBE[name=BI_user]]
2018-10-16 15:50:09,984 INFO  [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] rules.RealizationSortRule:40
: CUBE[name=BI_user] priority 1 cost 162. 
2018-10-16 15:50:09,984 INFO  [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] routing.QueryRouter:51
: Applying rule: class org.apache.kylin.query.routing.rules.RealizationSortRule, realizations
before: [CUBE[name=BI_user]], realizations after: [CUBE[name=BI_user]]
2018-10-16 15:50:09,984 INFO  [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] routing.QueryRouter:75
: The realizations remaining: [CUBE[name=BI_user]],and the final chosen one for current olap
context 0 is CUBE[name=BI_user]
2018-10-16 15:50:10,028 DEBUG [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] enumerator.OLAPEnumerator:105
: query storage...
2018-10-16 15:50:10,028 DEBUG [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] gtrecord.GTCubeStorageQueryBase:333
: Need storage aggregation
2018-10-16 15:50:10,028 INFO  [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] gtrecord.GTCubeStorageQueryBase:566
: exactAggregation is false because need storage aggregation
2018-10-16 15:50:10,029 INFO  [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] gtrecord.GTCubeStorageQueryBase:311
: Filter column set for query: []
2018-10-16 15:50:10,029 INFO  [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] gtrecord.GTCubeStorageQueryBase:320
: Filter mask is: 0
2018-10-16 15:50:10,029 DEBUG [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] gtrecord.GTCubeStorageQueryBase:435
: storageLimitLevel set to LIMIT_ON_RETURN_SIZE because groupD is not clustered at head, groupsD:
[APP.APP_BI_USER.OPERATOR] with cuboid columns: [APP.APP_BI_USER.REGIST_TIME, APP.APP_BI_USER.OPERATOR]
2018-10-16 15:50:10,029 INFO  [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] storage.StorageContext:167
: Enabling limit push down: 50000 at level: LIMIT_ON_RETURN_SIZE
2018-10-16 15:50:10,030 INFO  [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] gtrecord.GTCubeStorageQueryBase:191
: Cuboid identified: cube=BI_user, cuboidId=1025, groupsD=[APP.APP_BI_USER.OPERATOR], filterD=[],
limitPushdown=50000, limitLevel=LIMIT_ON_RETURN_SIZE, storageAggr=true
2018-10-16 15:50:10,030 DEBUG [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] common.SegmentPruner:94
: Pruner passed on segment BI_user[FULL_BUILD]
2018-10-16 15:50:10,030 INFO  [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] gtrecord.CubeSegmentScanner:61
: Init CubeSegmentScanner for segment FULL_BUILD
2018-10-16 15:50:10,032 DEBUG [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] v2.CubeHBaseRPC:315
: hbase.rpc.timeout = 60000 ms, use 54000 ms as timeout for coprocessor
2018-10-16 15:50:10,032 INFO  [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] v2.CubeHBaseEndpointRPC:141
: Serialized scanRequestBytes 474 bytes, rawScanBytesString 44 bytes
2018-10-16 15:50:10,033 INFO  [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] v2.CubeHBaseEndpointRPC:143
: The scan 5f252d05 for segment BI_user[FULL_BUILD] is as below with 1 separate raw scans,
shard part of start/end key is set to 0
2018-10-16 15:50:10,033 INFO  [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] v2.CubeHBaseRPC:288
: Visiting hbase table KYLIN_LCGEU7PJ2E: cuboid require post aggregation, from 1 to 1025 Start:
\x00\x00\x00\x00\x00\x00\x00\x00\x04\x01\x00\x00\x00 (\x00\x00\x00\x00\x00\x00\x00\x00\x04\x01\x00\x00\x00)
Stop:  \x00\x00\x00\x00\x00\x00\x00\x00\x04\x01\xFF\xFF\xFF\x00 (\x00\x00\x00\x00\x00\x00\x00\x00\x04\x01\xFF\xFF\xFF\x00),
No Fuzzy Key
2018-10-16 15:50:10,033 DEBUG [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] v2.CubeHBaseEndpointRPC:148
: Submitting rpc to 1 shards starting from shard 0, scan range count 1
2018-10-16 15:50:10,034 INFO  [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] common.KylinConfig:332
: Loading kylin-defaults.properties from file:/usr/local/kylin/tomcat/webapps/kylin/WEB-INF/lib/kylin-core-common-2.5.0.jar!/kylin-defaults.properties
2018-10-16 15:50:10,042 DEBUG [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] common.KylinConfig:291
: KYLIN_CONF property was not set, will seek KYLIN_HOME env variable
2018-10-16 15:50:10,042 INFO  [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] gtrecord.SequentialCubeTupleIterator:73
: Using SortedIteratorMergerWithLimit to merge segment results
2018-10-16 15:50:10,043 DEBUG [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] enumerator.OLAPEnumerator:117
: return TupleIterator...
2018-10-16 15:50:10,055 INFO  [kylin-coproc--pool6-t1370] v2.CubeHBaseEndpointRPC:203 : <sub-thread
for Query 76caaa08-bb31-f06b-d1ac-f575a983b06f GTScanRequest 5f252d05>Endpoint RPC returned
from HTable KYLIN_LCGEU7PJ2E Shard \x4B\x59\x4C\x49\x4E\x5F\x4C\x43\x47\x45\x55\x37\x50\x4A\x32\x45\x2C\x2C\x31\x35\x33\x39\x36\x37\x30\x34\x30\x34\x30\x31\x30\x2E\x33\x35\x39\x32\x32\x36\x66\x38\x31\x34\x34\x64\x64\x61\x36\x33\x35\x35\x61\x37\x35\x37\x36\x64\x30\x39\x65\x36\x37\x61\x61\x62\x2E
on host: prod-zeus-worker06.jdfq.com.Total scanned row: 1458. Total scanned bytes: 61836.
Total filtered row: 0. Total aggred row: 1453. Time elapsed in EP: 7(ms). Server CPU usage:
6.69361274875782E-4, server physical mem left: 6.8800565248E10, server swap mem left:4.2949632E9.Etc
message: start latency: 15@0,agg done@6,compress done@6,server stats done@7, debugGitTag:ac017f4f24534ccf44c65229e68b513bc7bd2755;.Normal
Complete: true.Compressed row size: 32
2018-10-16 15:50:10,055 DEBUG [kylin-coproc--pool6-t1370] util.CompressionUtils:67 : Original:
32 bytes. Decompressed: 23 bytes. Time: 0
2018-10-16 15:50:10,055 DEBUG [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] gtrecord.SortMergedPartitionResultIterator:76
: Using SortMergedPartitionResultIterator to merge 1 partition results out of 1 partitions
2018-10-16 15:50:10,057 INFO  [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] service.QueryService:1050
: Processed rows for each storageContext: 5 
2018-10-16 15:50:10,058 INFO  [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] service.QueryService:455
: Stats of SQL response: isException: false, duration: 83, total scan count 1458
2018-10-16 15:50:10,058 DEBUG [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] util.CheckUtil:35
: query cache is disabled
2018-10-16 15:50:10,058 INFO  [Query 76caaa08-bb31-f06b-d1ac-f575a983b06f-72408] service.QueryService:344
: 
==========================[QUERY]===============================
Query Id: 76caaa08-bb31-f06b-d1ac-f575a983b06f
SQL: select
operator,
count(*)
from app_bi_user
group by  operator;
User: ADMIN
Success: true
Duration: 0.083
Project: BI
Realization Names: [CUBE[name=BI_user]]
Cuboid Ids: [1025]
Total scan count: 1458
Total scan bytes: 61836
Result row count: 5
Accept Partial: true
Is Partial Result: false
Hit Exception Cache: false
Storage cache used: false
Is Query Push-Down: false
Is Prepare: false
Trace URL: null
Message: null
==========================[QUERY]===============================

2018-10-16 15:50:12,350 INFO  [FetcherRunner 1802655599-356] threadpool.DefaultFetcherRunner:96
: Job Fetcher: 0 should running, 0 actual running, 0 stopped, 0 ready, 7911 already succeed,
0 error, 100 discarded, 0 others
2018-10-16 15:50:12,415 INFO  [http-nio-7070-exec-14] common.KylinConfig:332 : Loading kylin-defaults.properties
from file:/usr/local/kylin/tomcat/webapps/kylin/WEB-INF/lib/kylin-core-common-2.5.0.jar!/kylin-defaults.properties
2018-10-16 15:50:12,425 DEBUG [http-nio-7070-exec-14] common.KylinConfig:291 : KYLIN_CONF
property was not set, will seek KYLIN_HOME env variable
 


------------------
郭锋
百乘金科/金蛋分期/技术部/数据开发组
北京市朝阳区北辰世纪中心B座17层
 
 
 
------------------ Original ------------------
From:  "Na Zhai"<na.zhai@kyligence.io>;
Date:  Tue, Oct 16, 2018 03:17 PM
To:  "user@kylin.apache.org"<user@kylin.apache.org>; 
Subject:  答复: Re:答复: 【kylin2.5.0】中文查询结果问题
 
Hi,郭锋
Do you use apache-kylin-2.5.0-bin-cdh60.tar.gz? And can you provide the log information?
 
 
发送自 Windows 10 版邮件应用
 


发件人: 郭锋 <guofeng@bcpeople.com>
发送时间: Tuesday, October 16, 2018 2:41:52 PM
收件人: user
主题: Re:答复: 【kylin2.5.0】中文查询结果问题 
 
Hi,
     The three scenarios  as follows:
    
       1、when i use operator='中国电信',it return 0;         2、when i use operator
!='中国电信',it return correctly;        3、when i don't user where clause, it return
correctly;

             
                                                                      









------------------
郭锋
百乘金科/金蛋分期/技术部/数据开发组
北京市朝阳区北辰世纪中心B座17层
 
 
 
------------------ Original ------------------
From:  "郭锋"<guofeng@bcpeople.com>;
Date:  Tue, Oct 16, 2018 11:41 AM
To:  "user"<user@kylin.apache.org>; 
Subject:  Re:答复: 【kylin2.5.0】中文查询结果问题
 
HI,  
      图片已添加到邮件附件

      谢谢!





------------------
郭锋
百乘金科/金蛋分期/技术部/数据开发组
北京市朝阳区北辰世纪中心B座17层
 
 
 
------------------ Original ------------------
From:  "Na Zhai"<na.zhai@kyligence.io>;
Date:  Tue, Oct 16, 2018 11:16 AM
To:  "user@kylin.apache.org"<user@kylin.apache.org>; 
Subject:  答复: 【kylin2.5.0】中文查询结果问题
 
Hi, guofeng
the picture is not displayed, can you resend them? Does it happen accidentally?
 
发送自 Windows 10 版邮件应用
 
发件人: 郭锋
发送时间: 2018年10月15日 19:41
收件人: user
主题: 【kylin2.5.0】中文查询结果问题
 
hi all: 
       kylin 查询条件中包含中文,返回的查询结果有问题:
       现象:在where语句中用 “=” 返回结果为0;
                在where语句中用 “like”时,返回正确结果;
 
       版本信息:Kylin的版本为2.5.0 , 部署在cdh6.0.0 上
       请问,是否遇到过这种情况,有什么结果方法,谢谢!
       截图如下
 
 
Mime
View raw message