kylin-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "KANG-SEN LU (JIRA)" <j...@apache.org>
Subject [jira] [Created] (KYLIN-3636) in kylin 2.4.1 and 2.5.1 topn aggregation query caused exception
Date Thu, 18 Oct 2018 12:54:00 GMT
KANG-SEN LU created KYLIN-3636:
----------------------------------

             Summary: in kylin 2.4.1 and 2.5.1 topn aggregation query caused exception
                 Key: KYLIN-3636
                 URL: https://issues.apache.org/jira/browse/KYLIN-3636
             Project: Kylin
          Issue Type: Bug
          Components: Query Engine
    Affects Versions: v2.4.1
            Reporter: KANG-SEN LU


Hi, ShaoFeng:

 

Thanks for the reply. I missed this email and not responded earlier, I am sorry.

 

I tried to reproduce this problem with the sample database, and it did not happen.

 

So I am hoping by collecting enough "clue", someone can figure out why this problem occurred.

 

--------------------------

I issued the following query at the sample project to exercise the topn aggregation:

 

select seller_id, SUM(price) as total from kylin_sales group by seller_id order by total limit
5;

 

With my own added debugging, I saw the following log in the kylin.log: (the query worked OK).

 

2018-10-16 16:18:19,963 INFO  [Query a747f16f-4b12-cc97-08d2-9b45c27a529f-90] model.FunctionDesc:59
: KSL22222, getRewriteFieldName=_KY_SUM_KYLIN_SALES_PRICE_

2018-10-16 16:18:19,963 INFO  [kylin-coproc--pool12-t1] v2.CubeHBaseEndpointRPC:217 : Query-a747f16f-4b12-cc97-08d2-9b45c27a529f:
send request to the init region server anovadata4.anovadata.local on table ANOVA_KYLIN_25X_K758MEAWJG

2018-10-16 16:18:19,963 INFO  [Query a747f16f-4b12-cc97-08d2-9b45c27a529f-90] topn.TopNMeasureType:399
: KSL888: in TopNMeasureType.java, sumFieldName= _KY_SUM_KYLIN_SALES_PRICE_

 

 

When I was executing my project query, I issued the following select statement:

 

SELECT  ZETTICSDW.A_VL_HOURLY_V.IMSIID \"ZETTICSDW_A_VL_HOURLY_V_IMSIID\", SUM(ZETTICSDW.A_VL_HOURLY_V.SIG_EVENT_COUNT)
\"vl_aggs_model___USERS_BY_ERROR_3XX\"  FROM  ZETTICSDW.A_VL_HOURLY_V inner JOIN ZETTICSDW.T_VL_TRANSACTION_RULE_V
ON ( ZETTICSDW.A_VL_HOURLY_V.CAUSE_CODE_KEY = ZETTICSDW.T_VL_TRANSACTION_RULE_V.CAUSE_CODE_KEY
AND ZETTICSDW.A_VL_HOURLY_V.REASON_CODE_KEY = ZETTICSDW.T_VL_TRANSACTION_RULE_V.REASON_CODE_KEY
AND ZETTICSDW.A_VL_HOURLY_V.TRANSACTION_TYPE_KEY = ZETTICSDW.T_VL_TRANSACTION_RULE_V.TRANSACTION_TYPE_KEY) 

WHERE  ((ZETTICSDW.A_VL_HOURLY_V.THEDATE = '20180209') AND ((ZETTICSDW.A_VL_HOURLY_V.THEHOUR
>= '02') AND (ZETTICSDW.A_VL_HOURLY_V.THEHOUR <= '03'))) AND ZETTICSDW.T_VL_TRANSACTION_RULE_V.DISPLAY_STRING
LIKE '%+3%'  

GROUP BY  ZETTICSDW.A_VL_HOURLY_V.IMSIID  

ORDER BY  \"vl_aggs_model___USERS_BY_ERROR_3XX\"

LIMIT 25

 

An exception occurred within the method "private ColumnRowType buildColumnRowType()" of "query/src/main/java/org/apache/kylin/query/relnode/OLAPTableScan.java".

 

        if (columns.size() != rowType.getFieldCount()) {

            throw new IllegalStateException("RowType=" + rowType.getFieldCount()
+ ", ColumnRowType=" + columns.size());

        }

 

It printed "RowType=133, ColumnRowType=132".

The RowType list contains one extra column name: "ANY _KY_SUM_1_3a1aedef_SIG_EVENT_COUNT_".

 

 

I think this has something to do with the bug fix "KYLIN-3359 Support sum(expression) if possible".

 

After this bug fix was submitted, I noticed that a lot of column name was added into rowType
like "_KY_SUM_XXX".

 

This strange column name "_KY_SUM_1_3a1aedef_SIG_EVENT_COUNT_" is very similar.

 

I also found that this extra column name, only existed in RowType but not in ColumnRowType,
was added in the method "public void implementRewrite(RewriteImplementor implementor)" within
"OLAPJoinRel.java".

 

With my own debug statement, I saw this debug text in kylin.log:

 

relnode.OLAPJoinRel:362 : KSL54: newField= #132: _KY_SUM_1_6735969a_SIG_EVENT_COUNT_ ANY

 

 

I hope someone with a deeper technical knowledge in kylin query engine can figure out what
was causing the problem I have seen.

 

Thanks again.

 

Kang-sen

 

*From:* ShaoFeng Shi [[mailto:shaofengshi@apache.org]] 
 *Sent:* Friday, October 05, 2018 9:59 PM
 *To:* user <[user@kylin.apache.org|mailto:user@kylin.apache.org]>
 *Subject:* Re: any body see topn in kylin 2.5.1 working?

 

Hi Kang-Sen,

 

Didn't see this; Can you reproduce the problem with the sample cube? 

 

Kang-Sen Lu <[klu@anovadata.com|mailto:klu@anovadata.com]> 于2018年10月6日周六
上午12:21写道:

I have been running kylin 2.2.1 with topn aggregation OK.

 

Recently, I upgraded to kylin 2.4.1 and 2.5.1, in both releases I am having query problem
related to topn support.

 

The problem is an exception occurred with the following error message in kylin.log:

 

Caused by: java.lang.IllegalStateException: RowType=133, ColumnRowType=132

        at org.apache.kylin.query.relnode.OLAPJoinRel.buildColumnRowType(OLAPJoinRel.java:223)

        at org.apache.kylin.query.relnode.OLAPJoinRel.implementRewrite(OLAPJoinRel.java:359)

        at org.apache.kylin.query.relnode.OLAPRel$RewriteImplementor.visitChild(OLAPRel.java:174)

        at org.apache.kylin.query.relnode.OLAPFilterRel.implementRewrite(OLAPFilterRel.java:163)

        at org.apache.kylin.query.relnode.OLAPRel$RewriteImplementor.visitChild(OLAPRel.java:174)

        at org.apache.kylin.query.relnode.OLAPProjectRel.implementRewrite(OLAPProjectRel.java:228)

        at org.apache.kylin.query.relnode.OLAPRel$RewriteImplementor.visitChild(OLAPRel.java:174)

        at org.apache.kylin.query.relnode.OLAPAggregateRel.implementRewrite(OLAPAggregateRel.java:402)

        at org.apache.kylin.query.relnode.OLAPRel$RewriteImplementor.visitChild(OLAPRel.java:174)

        at org.apache.kylin.query.relnode.OLAPSortRel.implementRewrite(OLAPSortRel.java:86)

        at org.apache.kylin.query.relnode.OLAPRel$RewriteImplementor.visitChild(OLAPRel.java:174)

        at org.apache.kylin.query.relnode.OLAPLimitRel.implementRewrite(OLAPLimitRel.java:109)

        at org.apache.kylin.query.relnode.OLAPRel$RewriteImplementor.visitChild(OLAPRel.java:174)

        at org.apache.kylin.query.relnode.OLAPToEnumerableConverter.implement(OLAPToEnumerableConverter.java:100)

        at org.apache.calcite.adapter.enumerable.EnumerableRelImplementor.implementRoot(EnumerableRelImplementor.java:108)

        at org.apache.calcite.adapter.enumerable.EnumerableInterpretable.toBindable(EnumerableInterpretable.java:92)

        at org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.implement(CalcitePrepareImpl.java:1281)

        at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:331)

        at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:228)

        at org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:801)

        at org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:656)

        at org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:618)

        at org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:214)

        at org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:603)

        at org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:638)

        at org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:149)

        ... 84 more

 

 

--

Best regards,

 

Shaofeng Shi 史少锋

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message