From issues-return-22033-archive-asf-public=cust-asf.ponee.io@kylin.apache.org Tue Nov 27 07:20:04 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id C45A8180670 for ; Tue, 27 Nov 2018 07:20:03 +0100 (CET) Received: (qmail 21369 invoked by uid 500); 27 Nov 2018 06:20:02 -0000 Mailing-List: contact issues-help@kylin.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@kylin.apache.org Delivered-To: mailing list issues@kylin.apache.org Received: (qmail 21349 invoked by uid 99); 27 Nov 2018 06:20:02 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 27 Nov 2018 06:20:02 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 60C6EC6330 for ; Tue, 27 Nov 2018 06:20:02 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -109.501 X-Spam-Level: X-Spam-Status: No, score=-109.501 tagged_above=-999 required=6.31 tests=[ENV_AND_HDR_SPF_MATCH=-0.5, KAM_ASCII_DIVIDERS=0.8, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, USER_IN_DEF_SPF_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id ju8IPcKcyUve for ; Tue, 27 Nov 2018 06:20:01 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id E9F655F612 for ; Tue, 27 Nov 2018 06:20:00 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 75D0AE092E for ; Tue, 27 Nov 2018 06:20:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 3A2852135D for ; Tue, 27 Nov 2018 06:20:00 +0000 (UTC) Date: Tue, 27 Nov 2018 06:20:00 +0000 (UTC) From: "Shaofeng SHI (JIRA)" To: issues@kylin.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (KYLIN-3636) Wrong "storage_type" in CubeDesc causing cube building error MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/KYLIN-3636?page=3Dcom.atlassian= .jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D1669= 9970#comment-16699970 ]=20 Shaofeng SHI commented on KYLIN-3636: ------------------------------------- I'm not sure what's the root cause in your side, but I see the default valu= e in CubeDesc.java is wrong. I=C2=A0updated that, hope it can solve the pro= blem. > Wrong "storage_type" in CubeDesc causing cube building error > ------------------------------------------------------------ > > Key: KYLIN-3636 > URL: https://issues.apache.org/jira/browse/KYLIN-3636 > Project: Kylin > Issue Type: Bug > Components: Query Engine > Affects Versions: v2.4.1 > Reporter: KANG-SEN LU > Assignee: Shaofeng SHI > Priority: Major > Fix For: v2.5.2 > > > Hi, ShaoFeng: > =C2=A0 > Thanks for the reply. I missed this email and not responded earlier, I am= sorry. > =C2=A0 > I tried to reproduce this problem with the sample database, and it did no= t happen. > =C2=A0 > So I am hoping by collecting enough "clue", someone can figure out why th= is problem occurred. > =C2=A0 > -------------------------- > I issued the following query at the sample project to exercise the topn a= ggregation: > =C2=A0 > select seller_id, SUM(price) as total from kylin_sales group by seller_id= order by total limit 5; > =C2=A0 > With my own added debugging, I saw the following log in the kylin.log: (t= he query worked OK). > =C2=A0 > 2018-10-16 16:18:19,963 INFO=C2=A0 [Query a747f16f-4b12-cc97-08d2-9b45c27= a529f-90] model.FunctionDesc:59 : KSL22222, getRewriteFieldName=3D_KY_SUM_K= YLIN_SALES_PRICE_ > 2018-10-16 16:18:19,963 INFO=C2=A0 [kylin-coproc--pool12-t1] v2.CubeHBase= EndpointRPC:217 : Query-a747f16f-4b12-cc97-08d2-9b45c27a529f: send request = to the init region server anovadata4.anovadata.local on table ANOVA_KYLIN_2= 5X_K758MEAWJG > 2018-10-16 16:18:19,963 INFO=C2=A0 [Query a747f16f-4b12-cc97-08d2-9b45c27= a529f-90] topn.TopNMeasureType:399 : KSL888: in TopNMeasureType.java, sumFi= eldName=3D _KY_SUM_KYLIN_SALES_PRICE_ > =C2=A0 > =C2=A0 > When I was executing my project query, I issued the following select stat= ement: > =C2=A0 > SELECT=C2=A0 ZETTICSDW.A_VL_HOURLY_V.IMSIID \"ZETTICSDW_A_VL_HOURLY_V_IMS= IID\", SUM(ZETTICSDW.A_VL_HOURLY_V.SIG_EVENT_COUNT) \"vl_aggs_model___USERS= _BY_ERROR_3XX\"=C2=A0 FROM=C2=A0 ZETTICSDW.A_VL_HOURLY_V inner JOIN ZETTICS= DW.T_VL_TRANSACTION_RULE_V ON ( ZETTICSDW.A_VL_HOURLY_V.CAUSE_CODE_KEY =3D = ZETTICSDW.T_VL_TRANSACTION_RULE_V.CAUSE_CODE_KEY AND ZETTICSDW.A_VL_HOURLY_= V.REASON_CODE_KEY =3D ZETTICSDW.T_VL_TRANSACTION_RULE_V.REASON_CODE_KEY AND= ZETTICSDW.A_VL_HOURLY_V.TRANSACTION_TYPE_KEY =3D ZETTICSDW.T_VL_TRANSACTIO= N_RULE_V.TRANSACTION_TYPE_KEY)=C2=A0 > WHERE=C2=A0 ((ZETTICSDW.A_VL_HOURLY_V.THEDATE =3D '20180209') AND ((ZETTI= CSDW.A_VL_HOURLY_V.THEHOUR >=3D '02') AND (ZETTICSDW.A_VL_HOURLY_V.THEHOUR = <=3D '03'))) AND ZETTICSDW.T_VL_TRANSACTION_RULE_V.DISPLAY_STRING LIKE '%+3= %'=C2=A0=C2=A0 > GROUP BY=C2=A0 ZETTICSDW.A_VL_HOURLY_V.IMSIID=C2=A0=C2=A0 > ORDER BY=C2=A0 \"vl_aggs_model___USERS_BY_ERROR_3XX\" > LIMIT 25 > =C2=A0 > An exception occurred within the method "private ColumnRowType buildColum= nRowType()" of "query/src/main/java/org/apache/kylin/query/relnode/OLAPTabl= eScan.java". > =C2=A0 > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0if (columns.size() !=3D r= owType.getFieldCount()) { > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 throw = new IllegalStateException("RowType=3D" + rowType.getFieldCount() + ", Colum= nRowType=3D" + columns.size()); > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 } > =C2=A0 > It printed "RowType=3D133, ColumnRowType=3D132". > The RowType list contains one extra column name: "ANY _KY_SUM_1_3a1aedef_= SIG_EVENT_COUNT_". > =C2=A0 > =C2=A0 > I think this has something to do with the bug fix "KYLIN-3359 Support sum= (expression) if possible". > =C2=A0 > After this bug fix was submitted, I noticed that a lot of column name was= added into rowType like "_KY_SUM_XXX". > =C2=A0 > This strange column name "_KY_SUM_1_3a1aedef_SIG_EVENT_COUNT_" is very si= milar. > =C2=A0 > I also found that this extra column name, only existed in RowType but not= in ColumnRowType, was added in the method "public void implementRewrite(Re= writeImplementor implementor)" within "OLAPJoinRel.java". > =C2=A0 > With my own debug statement, I saw this debug text in kylin.log: > =C2=A0 > relnode.OLAPJoinRel:362 : KSL54: newField=3D #132: _KY_SUM_1_6735969a_SIG= _EVENT_COUNT_ ANY > =C2=A0 > =C2=A0 > I hope someone with a deeper technical knowledge in kylin query engine ca= n figure out what was causing the problem I have seen. > =C2=A0 > Thanks again. > =C2=A0 > Kang-sen > =C2=A0 > *From:* ShaoFeng Shi [[mailto:shaofengshi@apache.org]]=20 > *Sent:* Friday, October 05, 2018 9:59 PM > *To:* user <[user@kylin.apache.org|mailto:user@kylin.apache.org]> > *Subject:* Re: any body see topn in kylin 2.5.1 working? > =C2=A0 > Hi Kang-Sen, > =C2=A0 > Didn't see this; Can you reproduce the problem with the sample cube?=C2= =A0 > =C2=A0 > Kang-Sen Lu <[klu@anovadata.com|mailto:klu@anovadata.com]> =E4=BA=8E2018= =E5=B9=B410=E6=9C=886=E6=97=A5=E5=91=A8=E5=85=AD =E4=B8=8A=E5=8D=8812:21=E5= =86=99=E9=81=93=EF=BC=9A > I have been running kylin 2.2.1 with topn aggregation OK. > =C2=A0 > Recently, I upgraded to kylin 2.4.1 and 2.5.1, in both releases I am havi= ng query problem related to topn support. > =C2=A0 > The problem is an exception occurred with the following error message in = kylin.log: > =C2=A0 > Caused by: java.lang.IllegalStateException: RowType=3D133, ColumnRowType= =3D132 > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.kylin.query.reln= ode.OLAPJoinRel.buildColumnRowType(OLAPJoinRel.java:223) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.kylin.query.reln= ode.OLAPJoinRel.implementRewrite(OLAPJoinRel.java:359) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.kylin.query.reln= ode.OLAPRel$RewriteImplementor.visitChild(OLAPRel.java:174) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.kylin.query.reln= ode.OLAPFilterRel.implementRewrite(OLAPFilterRel.java:163) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.kylin.query.reln= ode.OLAPRel$RewriteImplementor.visitChild(OLAPRel.java:174) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.kylin.query.reln= ode.OLAPProjectRel.implementRewrite(OLAPProjectRel.java:228) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.kylin.query.reln= ode.OLAPRel$RewriteImplementor.visitChild(OLAPRel.java:174) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.kylin.query.reln= ode.OLAPAggregateRel.implementRewrite(OLAPAggregateRel.java:402) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.kylin.query.reln= ode.OLAPRel$RewriteImplementor.visitChild(OLAPRel.java:174) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.kylin.query.reln= ode.OLAPSortRel.implementRewrite(OLAPSortRel.java:86) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.kylin.query.reln= ode.OLAPRel$RewriteImplementor.visitChild(OLAPRel.java:174) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.kylin.query.reln= ode.OLAPLimitRel.implementRewrite(OLAPLimitRel.java:109) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.kylin.query.reln= ode.OLAPRel$RewriteImplementor.visitChild(OLAPRel.java:174) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.kylin.query.reln= ode.OLAPToEnumerableConverter.implement(OLAPToEnumerableConverter.java:100) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.calcite.adapter.= enumerable.EnumerableRelImplementor.implementRoot(EnumerableRelImplementor.= java:108) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.calcite.adapter.= enumerable.EnumerableInterpretable.toBindable(EnumerableInterpretable.java:= 92) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.calcite.prepare.= CalcitePrepareImpl$CalcitePreparingStmt.implement(CalcitePrepareImpl.java:1= 281) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.calcite.prepare.= Prepare.prepareSql(Prepare.java:331) > =C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0=C2=A0=C2=A0at org.apache.calcite.prepare.= Prepare.prepareSql(Prepare.java:228) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.calcite.prepare.= CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:801) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.calcite.prepare.= CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:656) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.calcite.prepare.= CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:618) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.calcite.jdbc.Cal= citeConnectionImpl.parseQuery(CalciteConnectionImpl.java:214) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.calcite.jdbc.Cal= citeMetaImpl.prepareAndExecute(CalciteMetaImpl.java:603) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.calcite.avatica.= AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:638) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at org.apache.calcite.avatica.= AvaticaStatement.executeInternal(AvaticaStatement.java:149) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ... 84 more > =C2=A0 > =C2=A0 > -- > Best regards, > =C2=A0 > Shaofeng Shi =E5=8F=B2=E5=B0=91=E9=94=8B > =C2=A0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)