kylin-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Billy Liu (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (KYLIN-3284) Refactor all OLAPRel computeSelfCost
Date Sun, 18 Mar 2018 18:28:00 GMT

     [ https://issues.apache.org/jira/browse/KYLIN-3284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Billy Liu resolved KYLIN-3284.
------------------------------
       Resolution: Fixed
    Fix Version/s: v2.4.0

> Refactor all OLAPRel computeSelfCost
> ------------------------------------
>
>                 Key: KYLIN-3284
>                 URL: https://issues.apache.org/jira/browse/KYLIN-3284
>             Project: Kylin
>          Issue Type: Improvement
>          Components: Query Engine
>    Affects Versions: v2.3.0
>            Reporter: yiming.xu
>            Assignee: yiming.xu
>            Priority: Major
>             Fix For: v2.4.0
>
>
> for queries like:
>  select lstg_format_name from test_kylin_fact order by case 
>  when 1=1 then 
>  cal_dt 
>  ELSE 
>  seller_id 
>  end 
> the expected logical plan after volcano is:
> {code:java}
> OLAPToEnumerableConverter
>   OLAPLimitRel(ctx=[], fetch=[50000])
>     OLAPSortRel(sort0=[$1], dir0=[ASC], ctx=[])
>       OLAPProjectRel(LSTG_FORMAT_NAME=[$3], EXPR$1=[CASE(=(1, 1), $2, CAST($7):DATE)],
ctx=[])
>         OLAPTableScan(table=[[DEFAULT, TEST_KYLIN_FACT]], ctx=[], fields=[[0, 1, 2, 3,
4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26]])
> {code}
> however the actual is:
> {code:java}
>  EnumerableSort(sort0=[$1], dir0=[ASC])
>   OLAPToEnumerableConverter
>     OLAPSortRel(sort0=[$1], dir0=[ASC], ctx=[])
>       OLAPProjectRel(LSTG_FORMAT_NAME=[$3], EXPR$1=[CASE(=(1, 1), $2, CAST($7):DATE)],
ctx=[])
>         OLAPTableScan(table=[[DEFAULT, TEST_KYLIN_FACT]], ctx=[], fields=[[0, 1, 2, 3,
4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26]])
> {code}
> This is the RelOptPlanner:
> {code:java}
> Sets:
> Set#0, type: RecordType(BIGINT TRANS_ID, BIGINT ORDER_ID, DATE CAL_DT, VARCHAR(256) LSTG_FORMAT_NAME,
BIGINT LEAF_CATEG_ID, INTEGER LSTG_SITE_ID, SMALLINT SLR_SEGMENT_CD, INTEGER SELLER_ID, DECIMAL(19,
4) PRICE, INTEGER ITEM_COUNT, VARCHAR(256) TEST_COUNT_DISTINCT_BITMAP, DECIMAL(19, 4) DEAL_AMOUNT,
INTEGER DEAL_YEAR, VARCHAR(256) LEFTJOIN_BUYER_ID_AND_COUNTRY_NAME, VARCHAR(256) LEFTJOIN_SELLER_ID_AND_COUNTRY_NAME,
VARCHAR(256) LEFTJOIN_BUYER_COUNTRY_ABBR, VARCHAR(256) LEFTJOIN_SELLER_COUNTRY_ABBR, BIGINT
_KY_COUNT__, INTEGER _KY_SUM_TEST_KYLIN_FACT_ITEM_COUNT_, DECIMAL(19, 4) _KY_SUM_TEST_KYLIN_FACT_PRICE_,
DECIMAL(19, 4) _KY_MIN_TEST_KYLIN_FACT_PRICE_, DECIMAL(19, 4) _KY_MAX_TEST_KYLIN_FACT_PRICE_,
ANY _KY_COUNT_DISTINCT_TEST_KYLIN_FACT_SELLER_ID_, ANY _KY_COUNT_DISTINCT_TEST_KYLIN_FACT_LSTG_FORMAT_NAME_TEST_KYLIN_FACT_SELLER_ID_,
ANY _KY_COUNT_DISTINCT_TEST_KYLIN_FACT_TEST_COUNT_DISTINCT_BITMAP_, DECIMAL(19, 4) _KY_SUM_TEST_KYLIN_FACT_DEAL_AMOUNT_,
ANY _KY_PERCENTILE_APPROX_TEST_KYLIN_FACT_PRICE_)
> 	rel#140:Subset#0.OLAP.[], best=rel#131, importance=0.7290000000000001
> 		rel#131:OLAPTableScan.OLAP.[](table=[DEFAULT, TEST_KYLIN_FACT],ctx=,fields=[0, 1, 2,
3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26]),
rowcount=100.0, cumulative cost={5.0 rows, 5.050000000000001 cpu, 0.0 io}
> 		rel#179:OLAPSortRel.OLAP.[2](input=rel#140:Subset#0.OLAP.[],sort0=$2,dir0=ASC,ctx=),
rowcount=100.0, cumulative cost={2491.7919004335695 rows, 10.05 cpu, 0.0 io}
> 	rel#169:Subset#0.NONE.[2], best=null, importance=0.7290000000000001
> 		rel#167:LogicalSort.NONE.[2](input=rel#140:Subset#0.OLAP.[],sort0=$2,dir0=ASC), rowcount=100.0,
cumulative cost={inf}
> 	rel#171:Subset#0.ENUMERABLE.[2], best=rel#182, importance=0.81
> 		rel#181:EnumerableSort.ENUMERABLE.[2](input=rel#177:Subset#0.ENUMERABLE.[],sort0=$2,dir0=ASC),
rowcount=100.0, cumulative cost={1.0E100 rows, 110.05 cpu, 0.0 io}
> 		rel#182:OLAPToEnumerableConverter.ENUMERABLE.[2](input=rel#180:Subset#0.OLAP.[2]),
rowcount=100.0, cumulative cost={1.0E100 rows, 10.05 cpu, 0.0 io}
> 	rel#173:Subset#0.OLAP.[1], best=null, importance=0.6561
> 	rel#177:Subset#0.ENUMERABLE.[], best=rel#182, importance=0.7290000000000001
> 		rel#181:EnumerableSort.ENUMERABLE.[2](input=rel#177:Subset#0.ENUMERABLE.[],sort0=$2,dir0=ASC),
rowcount=100.0, cumulative cost={1.0E100 rows, 110.05 cpu, 0.0 io}
> 		rel#182:OLAPToEnumerableConverter.ENUMERABLE.[2](input=rel#180:Subset#0.OLAP.[2]),
rowcount=100.0, cumulative cost={1.0E100 rows, 10.05 cpu, 0.0 io}
> 		rel#185:OLAPToEnumerableConverter.ENUMERABLE.[](input=rel#140:Subset#0.OLAP.[]), rowcount=100.0,
cumulative cost={1.0E100 rows, 5.050000000000001 cpu, 0.0 io}
> 	rel#180:Subset#0.OLAP.[2], best=rel#179, importance=0.6561
> 		rel#179:OLAPSortRel.OLAP.[2](input=rel#140:Subset#0.OLAP.[],sort0=$2,dir0=ASC,ctx=),
rowcount=100.0, cumulative cost={2491.7919004335695 rows, 10.05 cpu, 0.0 io}
> Set#1, type: RecordType(VARCHAR(256) LSTG_FORMAT_NAME, DATE EXPR$1)
> 	rel#142:Subset#1.NONE.[], best=null, importance=0.81
> 		rel#141:LogicalProject.NONE.[](input=rel#140:Subset#0.OLAP.[],LSTG_FORMAT_NAME=$3,EXPR$1=CASE(=(1,
1), $2, CAST($7):DATE)), importance=0.0, rowcount=100.0, cumulative cost={inf}
> 		rel#143:LogicalSort.NONE.[1](input=rel#142:Subset#1.NONE.[],sort0=$1,dir0=ASC), rowcount=100.0,
cumulative cost={inf}
> 		rel#163:LogicalProject.NONE.[](input=rel#140:Subset#0.OLAP.[],LSTG_FORMAT_NAME=$3,EXPR$1=$2),
rowcount=100.0, cumulative cost={inf}
> 		rel#170:LogicalProject.NONE.[1](input=rel#169:Subset#0.NONE.[2],LSTG_FORMAT_NAME=$3,EXPR$1=$2),
rowcount=100.0, cumulative cost={inf}
> 	rel#149:Subset#1.ENUMERABLE.[], best=rel#157, importance=0.9
> 		rel#148:AbstractConverter.ENUMERABLE.[1](input=rel#152:Subset#1.NONE.[1],convention=ENUMERABLE,sort=[1]),
rowcount=100.0, cumulative cost={inf}
> 		rel#150:EnumerableSort.ENUMERABLE.[1](input=rel#149:Subset#1.ENUMERABLE.[],sort0=$1,dir0=ASC),
rowcount=100.0, cumulative cost={1.0E100 rows, 130.05 cpu, 0.0 io}
> 		rel#153:AbstractConverter.ENUMERABLE.[1](input=rel#149:Subset#1.ENUMERABLE.[],convention=ENUMERABLE,sort=[1]),
rowcount=100.0, cumulative cost={inf}
> 		rel#157:OLAPToEnumerableConverter.ENUMERABLE.[1](input=rel#156:Subset#1.OLAP.[1]),
rowcount=100.0, cumulative cost={1.0E100 rows, 30.05 cpu, 0.0 io}
> 		rel#162:OLAPToEnumerableConverter.ENUMERABLE.[](input=rel#154:Subset#1.OLAP.[]), rowcount=100.0,
cumulative cost={1.0E100 rows, 25.05 cpu, 0.0 io}
> 		rel#172:EnumerableProject.ENUMERABLE.[[1]](input=rel#171:Subset#0.ENUMERABLE.[2],LSTG_FORMAT_NAME=$3,EXPR$1=$2),
rowcount=100.0, cumulative cost={1.0E100 rows, 210.05 cpu, 0.0 io}
> 		rel#178:EnumerableProject.ENUMERABLE.[](input=rel#177:Subset#0.ENUMERABLE.[],LSTG_FORMAT_NAME=$3,EXPR$1=$2),
rowcount=100.0, cumulative cost={1.0E100 rows, 210.05 cpu, 0.0 io}
> 	rel#151:Subset#1.ENUMERABLE.[1], best=rel#150, importance=1.0
> 		rel#148:AbstractConverter.ENUMERABLE.[1](input=rel#152:Subset#1.NONE.[1],convention=ENUMERABLE,sort=[1]),
rowcount=100.0, cumulative cost={inf}
> 		rel#150:EnumerableSort.ENUMERABLE.[1](input=rel#149:Subset#1.ENUMERABLE.[],sort0=$1,dir0=ASC),
rowcount=100.0, cumulative cost={1.0E100 rows, 130.05 cpu, 0.0 io}
> 		rel#153:AbstractConverter.ENUMERABLE.[1](input=rel#149:Subset#1.ENUMERABLE.[],convention=ENUMERABLE,sort=[1]),
rowcount=100.0, cumulative cost={inf}
> 		rel#157:OLAPToEnumerableConverter.ENUMERABLE.[1](input=rel#156:Subset#1.OLAP.[1]),
rowcount=100.0, cumulative cost={1.0E100 rows, 30.05 cpu, 0.0 io}
> 		rel#172:EnumerableProject.ENUMERABLE.[[1]](input=rel#171:Subset#0.ENUMERABLE.[2],LSTG_FORMAT_NAME=$3,EXPR$1=$2),
rowcount=100.0, cumulative cost={1.0E100 rows, 210.05 cpu, 0.0 io}
> 	rel#152:Subset#1.NONE.[1], best=null, importance=0.9
> 		rel#143:LogicalSort.NONE.[1](input=rel#142:Subset#1.NONE.[],sort0=$1,dir0=ASC), rowcount=100.0,
cumulative cost={inf}
> 		rel#170:LogicalProject.NONE.[1](input=rel#169:Subset#0.NONE.[2],LSTG_FORMAT_NAME=$3,EXPR$1=$2),
rowcount=100.0, cumulative cost={inf}
> 	rel#154:Subset#1.OLAP.[], best=rel#160, importance=0.7290000000000001
> 		rel#155:OLAPSortRel.OLAP.[1](input=rel#154:Subset#1.OLAP.[],sort0=$1,dir0=ASC,ctx=),
rowcount=100.0, cumulative cost={199.2068074395237 rows, 30.05 cpu, 0.0 io}
> 		rel#160:OLAPProjectRel.OLAP.[](input=rel#140:Subset#0.OLAP.[],LSTG_FORMAT_NAME=$3,EXPR$1=CASE(=(1,
1), $2, CAST($7):DATE),ctx=), rowcount=100.0, cumulative cost={15.0 rows, 25.05 cpu, 0.0 io}
> 		rel#164:OLAPProjectRel.OLAP.[](input=rel#140:Subset#0.OLAP.[],LSTG_FORMAT_NAME=$3,EXPR$1=$2,ctx=),
rowcount=100.0, cumulative cost={15.0 rows, 25.05 cpu, 0.0 io}
> 		rel#174:OLAPProjectRel.OLAP.[1](input=rel#173:Subset#0.OLAP.[1],LSTG_FORMAT_NAME=$3,EXPR$1=$2,ctx=),
rowcount=100.0, cumulative cost={inf}
> 	rel#156:Subset#1.OLAP.[1], best=rel#155, importance=0.81
> 		rel#155:OLAPSortRel.OLAP.[1](input=rel#154:Subset#1.OLAP.[],sort0=$1,dir0=ASC,ctx=),
rowcount=100.0, cumulative cost={199.2068074395237 rows, 30.05 cpu, 0.0 io}
> 		rel#174:OLAPProjectRel.OLAP.[1](input=rel#173:Subset#0.OLAP.[1],LSTG_FORMAT_NAME=$3,EXPR$1=$2,ctx=),
rowcount=100.0, cumulative cost={inf}
> {code}
> The best root rel is  rel#151:Subset#1.ENUMERABLE.[1] : rel#150:EnumerableSort.ENUMERABLE
,because it cost equal rel#157:OLAPToEnumerableConverter.ENUMERABLE. The OLAPToEnumerableConverter
cost is so big that the details are neglected.We should refactor the cost system.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message