kylin-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "KANG-SEN LU (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KYLIN-3636) in kylin 2.4.1 and 2.5.1 topn aggregation query caused exception
Date Thu, 15 Nov 2018 13:36:00 GMT

    [ https://issues.apache.org/jira/browse/KYLIN-3636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688068#comment-16688068
] 

KANG-SEN LU commented on KYLIN-3636:
------------------------------------

Hi, Shaofeng:

 

I would like to add that there is NO bug in the kylin code, one may argue. Because the reason
I faced those problem related to TOPN aggregation function is caused by "improper documentation
about how to configure TOPN metric". So if we can improve the usage documentation, there will
be more happy users.

> in kylin 2.4.1 and 2.5.1 topn aggregation query caused exception
> ----------------------------------------------------------------
>
>                 Key: KYLIN-3636
>                 URL: https://issues.apache.org/jira/browse/KYLIN-3636
>             Project: Kylin
>          Issue Type: Bug
>          Components: Query Engine
>    Affects Versions: v2.4.1
>            Reporter: KANG-SEN LU
>            Assignee: Shaofeng SHI
>            Priority: Major
>             Fix For: v2.6.0
>
>
> Hi, ShaoFeng:
>  
> Thanks for the reply. I missed this email and not responded earlier, I am sorry.
>  
> I tried to reproduce this problem with the sample database, and it did not happen.
>  
> So I am hoping by collecting enough "clue", someone can figure out why this problem occurred.
>  
> --------------------------
> I issued the following query at the sample project to exercise the topn aggregation:
>  
> select seller_id, SUM(price) as total from kylin_sales group by seller_id order by total
limit 5;
>  
> With my own added debugging, I saw the following log in the kylin.log: (the query worked
OK).
>  
> 2018-10-16 16:18:19,963 INFO  [Query a747f16f-4b12-cc97-08d2-9b45c27a529f-90] model.FunctionDesc:59
: KSL22222, getRewriteFieldName=_KY_SUM_KYLIN_SALES_PRICE_
> 2018-10-16 16:18:19,963 INFO  [kylin-coproc--pool12-t1] v2.CubeHBaseEndpointRPC:217
: Query-a747f16f-4b12-cc97-08d2-9b45c27a529f: send request to the init region server anovadata4.anovadata.local
on table ANOVA_KYLIN_25X_K758MEAWJG
> 2018-10-16 16:18:19,963 INFO  [Query a747f16f-4b12-cc97-08d2-9b45c27a529f-90] topn.TopNMeasureType:399
: KSL888: in TopNMeasureType.java, sumFieldName= _KY_SUM_KYLIN_SALES_PRICE_
>  
>  
> When I was executing my project query, I issued the following select statement:
>  
> SELECT  ZETTICSDW.A_VL_HOURLY_V.IMSIID \"ZETTICSDW_A_VL_HOURLY_V_IMSIID\", SUM(ZETTICSDW.A_VL_HOURLY_V.SIG_EVENT_COUNT)
\"vl_aggs_model___USERS_BY_ERROR_3XX\"  FROM  ZETTICSDW.A_VL_HOURLY_V inner JOIN ZETTICSDW.T_VL_TRANSACTION_RULE_V
ON ( ZETTICSDW.A_VL_HOURLY_V.CAUSE_CODE_KEY = ZETTICSDW.T_VL_TRANSACTION_RULE_V.CAUSE_CODE_KEY
AND ZETTICSDW.A_VL_HOURLY_V.REASON_CODE_KEY = ZETTICSDW.T_VL_TRANSACTION_RULE_V.REASON_CODE_KEY
AND ZETTICSDW.A_VL_HOURLY_V.TRANSACTION_TYPE_KEY = ZETTICSDW.T_VL_TRANSACTION_RULE_V.TRANSACTION_TYPE_KEY) 
> WHERE  ((ZETTICSDW.A_VL_HOURLY_V.THEDATE = '20180209') AND ((ZETTICSDW.A_VL_HOURLY_V.THEHOUR
>= '02') AND (ZETTICSDW.A_VL_HOURLY_V.THEHOUR <= '03'))) AND ZETTICSDW.T_VL_TRANSACTION_RULE_V.DISPLAY_STRING
LIKE '%+3%'  
> GROUP BY  ZETTICSDW.A_VL_HOURLY_V.IMSIID  
> ORDER BY  \"vl_aggs_model___USERS_BY_ERROR_3XX\"
> LIMIT 25
>  
> An exception occurred within the method "private ColumnRowType buildColumnRowType()"
of "query/src/main/java/org/apache/kylin/query/relnode/OLAPTableScan.java".
>  
>         if (columns.size() != rowType.getFieldCount()) {
>             throw new IllegalStateException("RowType=" + rowType.getFieldCount()
+ ", ColumnRowType=" + columns.size());
>         }
>  
> It printed "RowType=133, ColumnRowType=132".
> The RowType list contains one extra column name: "ANY _KY_SUM_1_3a1aedef_SIG_EVENT_COUNT_".
>  
>  
> I think this has something to do with the bug fix "KYLIN-3359 Support sum(expression)
if possible".
>  
> After this bug fix was submitted, I noticed that a lot of column name was added into
rowType like "_KY_SUM_XXX".
>  
> This strange column name "_KY_SUM_1_3a1aedef_SIG_EVENT_COUNT_" is very similar.
>  
> I also found that this extra column name, only existed in RowType but not in ColumnRowType,
was added in the method "public void implementRewrite(RewriteImplementor implementor)" within
"OLAPJoinRel.java".
>  
> With my own debug statement, I saw this debug text in kylin.log:
>  
> relnode.OLAPJoinRel:362 : KSL54: newField= #132: _KY_SUM_1_6735969a_SIG_EVENT_COUNT_
ANY
>  
>  
> I hope someone with a deeper technical knowledge in kylin query engine can figure out
what was causing the problem I have seen.
>  
> Thanks again.
>  
> Kang-sen
>  
> *From:* ShaoFeng Shi [[mailto:shaofengshi@apache.org]] 
>  *Sent:* Friday, October 05, 2018 9:59 PM
>  *To:* user <[user@kylin.apache.org|mailto:user@kylin.apache.org]>
>  *Subject:* Re: any body see topn in kylin 2.5.1 working?
>  
> Hi Kang-Sen,
>  
> Didn't see this; Can you reproduce the problem with the sample cube? 
>  
> Kang-Sen Lu <[klu@anovadata.com|mailto:klu@anovadata.com]> 于2018年10月6日周六
上午12:21写道:
> I have been running kylin 2.2.1 with topn aggregation OK.
>  
> Recently, I upgraded to kylin 2.4.1 and 2.5.1, in both releases I am having query problem
related to topn support.
>  
> The problem is an exception occurred with the following error message in kylin.log:
>  
> Caused by: java.lang.IllegalStateException: RowType=133, ColumnRowType=132
>         at org.apache.kylin.query.relnode.OLAPJoinRel.buildColumnRowType(OLAPJoinRel.java:223)
>         at org.apache.kylin.query.relnode.OLAPJoinRel.implementRewrite(OLAPJoinRel.java:359)
>         at org.apache.kylin.query.relnode.OLAPRel$RewriteImplementor.visitChild(OLAPRel.java:174)
>         at org.apache.kylin.query.relnode.OLAPFilterRel.implementRewrite(OLAPFilterRel.java:163)
>         at org.apache.kylin.query.relnode.OLAPRel$RewriteImplementor.visitChild(OLAPRel.java:174)
>         at org.apache.kylin.query.relnode.OLAPProjectRel.implementRewrite(OLAPProjectRel.java:228)
>         at org.apache.kylin.query.relnode.OLAPRel$RewriteImplementor.visitChild(OLAPRel.java:174)
>         at org.apache.kylin.query.relnode.OLAPAggregateRel.implementRewrite(OLAPAggregateRel.java:402)
>         at org.apache.kylin.query.relnode.OLAPRel$RewriteImplementor.visitChild(OLAPRel.java:174)
>         at org.apache.kylin.query.relnode.OLAPSortRel.implementRewrite(OLAPSortRel.java:86)
>         at org.apache.kylin.query.relnode.OLAPRel$RewriteImplementor.visitChild(OLAPRel.java:174)
>         at org.apache.kylin.query.relnode.OLAPLimitRel.implementRewrite(OLAPLimitRel.java:109)
>         at org.apache.kylin.query.relnode.OLAPRel$RewriteImplementor.visitChild(OLAPRel.java:174)
>         at org.apache.kylin.query.relnode.OLAPToEnumerableConverter.implement(OLAPToEnumerableConverter.java:100)
>         at org.apache.calcite.adapter.enumerable.EnumerableRelImplementor.implementRoot(EnumerableRelImplementor.java:108)
>         at org.apache.calcite.adapter.enumerable.EnumerableInterpretable.toBindable(EnumerableInterpretable.java:92)
>         at org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.implement(CalcitePrepareImpl.java:1281)
>         at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:331)
>         at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:228)
>         at org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:801)
>         at org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:656)
>         at org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:618)
>         at org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:214)
>         at org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:603)
>         at org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:638)
>         at org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:149)
>         ... 84 more
>  
>  
> --
> Best regards,
>  
> Shaofeng Shi 史少锋
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message