Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 8C873200D57 for ; Mon, 27 Nov 2017 06:41:08 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 8B168160BFF; Mon, 27 Nov 2017 05:41:08 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 8454E160C00 for ; Mon, 27 Nov 2017 06:41:07 +0100 (CET) Received: (qmail 90290 invoked by uid 500); 27 Nov 2017 05:41:06 -0000 Mailing-List: contact user-help@kylin.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@kylin.apache.org Delivered-To: mailing list user@kylin.apache.org Received: (qmail 90270 invoked by uid 99); 27 Nov 2017 05:41:06 -0000 Received: from mail-relay.apache.org (HELO mail-relay.apache.org) (140.211.11.15) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 27 Nov 2017 05:41:06 +0000 Received: from mail-pg0-f54.google.com (mail-pg0-f54.google.com [74.125.83.54]) by mail-relay.apache.org (ASF Mail Server at mail-relay.apache.org) with ESMTPSA id 95DF01A0044; Mon, 27 Nov 2017 05:41:05 +0000 (UTC) Received: by mail-pg0-f54.google.com with SMTP id r12so18201144pgu.10; Sun, 26 Nov 2017 21:41:04 -0800 (PST) X-Gm-Message-State: AJaThX750CDZxmrULNMi4sDsPny2drWdPySpI6kFebfDV4+zLE5m//EK vcmpK1Q61nyukg0IbQa5/7Z8YuJtDRmm3vw6uXM= X-Google-Smtp-Source: AGs4zMYjngSwVrRBITUubnEAM4/Enfgr6HFWJhEPSkLeIHw3LNsm651pjMPLSzJc9GFcOF6pn14pbwH6VceAFy3Lfjk= X-Received: by 10.98.224.200 with SMTP id d69mr20135481pfm.100.1511761263758; Sun, 26 Nov 2017 21:41:03 -0800 (PST) MIME-Version: 1.0 Received: by 10.100.160.169 with HTTP; Sun, 26 Nov 2017 21:40:23 -0800 (PST) In-Reply-To: References: From: ShaoFeng Shi Date: Mon, 27 Nov 2017 13:40:23 +0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [Discuss] Disable/hide "RAW" measure in Kylin web GUI To: dev , user Content-Type: multipart/alternative; boundary="001a11370ab01c94e0055ef059bb" archived-at: Mon, 27 Nov 2017 05:41:08 -0000 --001a11370ab01c94e0055ef059bb Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Last year I raised this discussion but didn't have follow-up action. Now we see there are still some new users misuse this feature, and then face performance and maintenance issues. In Kylin 2.1, the new "query pushdown" feature can forward the Cube unmatched queries to alternative query engines like Hive / SparkSQL. The raw data query is just such a scenario. So I think it is time to disable the RAW measure on Kylin now. JIRA created for it: https://issues.apache.org/jira/browse/KYLIN-3062 Please comment if you see any issue. 2016-12-19 22:03 GMT+08:00 Billy Liu : > The experimental mode is system wide feature toggle. I think case by case > is more flexible. Most new features could have toggles, default are off. > > 2016-12-19 21:40 GMT+08:00 Luke Han : > > > Beta or Experimental will also bring confusing for most of users. > > > > Maybe we could have something called "expert" or "experimental" model i= n > > system configuration. > > > > User will not see such content since they will be hidden by default but > > admin could set it to true if they confident to enable such features. > > > > How do you think? > > > > > > Best Regards! > > --------------------- > > > > Luke Han > > > > On Mon, Dec 19, 2016 at 11:24 AM, Xiaoyu Wang > wrote: > > > > > I=E2=80=99m sorry for too long not maintain it! > > > > > > I agree with liyang to give a label to the RAW measure =E2=80=9CBeta= =E2=80=9D or > others. > > > > > > I will improve it when I have time! > > > > > > 2016-12-19 10:21 GMT+08:00 Li Yang : > > > > > > > Or display the Raw with "Beta" or "Experimental" to warn user that = it > > is > > > > not a mature feature? > > > > > > > > On Fri, Dec 16, 2016 at 12:30 PM, =E5=BA=B7=E5=87=AF=E6=A3=AE wrote: > > > > > > > > > +1. > > > > > But the "RAW" measure is still some useful, we could improve it > next > > > year > > > > > when we have time. > > > > > > > > > > > > > > > ------------------ =E5=8E=9F=E5=A7=8B=E9=82=AE=E4=BB=B6 ---------= --------- > > > > > =E5=8F=91=E4=BB=B6=E4=BA=BA: "ShaoFeng Shi";; > > > > > =E5=8F=91=E9=80=81=E6=97=B6=E9=97=B4: 2016=E5=B9=B412=E6=9C=8815= =E6=97=A5(=E6=98=9F=E6=9C=9F=E5=9B=9B) =E4=B8=AD=E5=8D=8812:05 > > > > > =E6=94=B6=E4=BB=B6=E4=BA=BA: "dev"; > > > > > > > > > > =E4=B8=BB=E9=A2=98: [Discuss] Disable/hide "RAW" measure in Kylin= web GUI > > > > > > > > > > > > > > > > > > > > Hello developers and users, > > > > > > > > > > I have a proposal want to disucss here, which is about the "RAW" > > > measure > > > > in > > > > > Kylin. > > > > > > > > > > The RAW measure was developed to solve the requirement for gettin= g > > raw > > > > data > > > > > when user drill down from high level to low levels. It's > performance > > > > would > > > > > be much better than fetching from source like Hive, so some users > > like > > > > it. > > > > > This blog introduces it: > > > > > https://kylin.apache.org/blog/2016/05/29/raw-measure-in-kylin/ > > > > > > > > > > While it has some limitations: > > > > > 1) always use dictionary encoding, which means it couldn't suppor= t > > UHC; > > > > > since raw columns usually be transaction IDs, numbers, etc, the > > > building > > > > > cost is much higher than ordinary dimensions; > > > > > 2) the raw messages for a dimension combination are persisted in > one > > > big > > > > > cell,; When too many rows dumped in one cell, will get > BufferOverflow > > > > > error. This couldn't be predicted, so for an modeler or analyst h= e > > > > doesn't > > > > > know whether this feature could work when he creates the cube. > > > > > 3) seems no people maintains it; > > > > > > > > > > Based on above, I propose hiding this measure on Web GUI by > default, > > so > > > > > avoiding confusing user; If someone hear and want to use this, he > > still > > > > can > > > > > enable it by simply set in kylin.properties. > > > > > > > > > > Any input is welcomed. > > > > > > > > > > -- > > > > > Best regards, > > > > > > > > > > Shaofeng Shi =E5=8F=B2=E5=B0=91=E9=94=8B > > > > > > > > > > > > > > > --=20 Best regards, Shaofeng Shi =E5=8F=B2=E5=B0=91=E9=94=8B --001a11370ab01c94e0055ef059bb Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Last year I raised this discussion but didn't have fol= low-up action.

Now we see there are still some new users= misuse this feature, and then face performance and maintenance issues.

In Kylin 2.1, the new "query pushdown" feat= ure can forward the Cube unmatched queries to alternative query engines lik= e Hive / SparkSQL. The raw data query is just such a scenario.=C2=A0
<= div>
So I think it is time to disable the RAW measure on Kyli= n now.=C2=A0 JIRA created for it:=C2=A0https://issues.apache.org/jira/browse/KYLIN-3062


2016-12-19 22:03= GMT+08:00 Billy Liu <billyliu@apache.org>:
The experimental mode is system wide feature toggle. I = think case by case
is more flexible. Most new features could have toggles, default are off.

2016-12-19 21:40 GMT+08:00 Luke Han <luke.hq@gmail.com>:

> Beta or Experimental will also bring confusing for most of users.
>
> Maybe we could have something called "expert" or "exper= imental" model in
> system configuration.
>
> User will not see such content since they will be hidden by default bu= t
> admin could set it to true if they confident to enable such features.<= br> >
> How do you think?
>
>
> Best Regards!
> ---------------------
>
> Luke Han
>
> On Mon, Dec 19, 2016 at 11:24 AM, Xiaoyu Wang <wangxy.jd@gmail.com> wrote:
>
> > I=E2=80=99m sorry for too long not maintain it!
> >
> > I agree with liyang to give a label to the RAW measure =E2=80=9CB= eta=E2=80=9D or others.
> >
> > I will improve it when I have time!
> >
> > 2016-12-19 10:21 GMT+08:00 Li Yang <liyang@apache.org>:
> >
> > > Or display the Raw with "Beta" or "Experiment= al" to warn user that it
> is
> > > not a mature feature?
> > >
> > > On Fri, Dec 16, 2016 at 12:30 PM, =E5=BA=B7=E5=87=AF=E6=A3= =AE <kangkaisen@qq.com> wrot= e:
> > >
> > > > +1.
> > > > But the "RAW" measure is still some useful, w= e could improve it next
> > year
> > > > when we have time.
> > > >
> > > >
> > > > ------------------ =E5=8E=9F=E5=A7=8B=E9=82=AE=E4=BB=B6= ------------------
> > > > =E5=8F=91=E4=BB=B6=E4=BA=BA: "ShaoFeng Shi";&= lt;shaofengshi@apache.org>= ;
> > > > =E5=8F=91=E9=80=81=E6=97=B6=E9=97=B4: 2016=E5=B9=B412= =E6=9C=8815=E6=97=A5(=E6=98=9F=E6=9C=9F=E5=9B=9B) =E4=B8=AD=E5=8D=8812:05 > > > > =E6=94=B6=E4=BB=B6=E4=BA=BA: "dev"<dev@kylin.apache.org>;
> > > >
> > > > =E4=B8=BB=E9=A2=98: [Discuss] Disable/hide "RAW&qu= ot; measure in Kylin web GUI
> > > >
> > > >
> > > >
> > > > Hello developers and users,
> > > >
> > > > I have a proposal want to disucss here, which is about = the "RAW"
> > measure
> > > in
> > > > Kylin.
> > > >
> > > > The RAW measure was developed to solve the requirement = for getting
> raw
> > > data
> > > > when user drill down from high level to low levels. It&= #39;s performance
> > > would
> > > > be much better than fetching from source like Hive, so = some users
> like
> > > it.
> > > > This blog introduces it:
> > > > https://kylin.apac= he.org/blog/2016/05/29/raw-measure-in-kylin/
> > > >
> > > > While it has some limitations:
> > > > 1) always use dictionary encoding, which means it could= n't support
> UHC;
> > > > since raw columns usually be transaction IDs, numbers, = etc, the
> > building
> > > > cost is much higher than ordinary dimensions;
> > > > 2) the raw messages for a dimension combination are per= sisted in one
> > big
> > > > cell,; When too many rows dumped in one cell, will get = BufferOverflow
> > > > error. This couldn't be predicted, so for an modele= r or analyst he
> > > doesn't
> > > > know whether this feature could work when he creates th= e cube.
> > > > 3) seems no people maintains it;
> > > >
> > > > Based on above, I propose hiding this measure on Web GU= I by default,
> so
> > > > avoiding confusing user; If someone hear and want to us= e this, he
> still
> > > can
> > > > enable it by simply set in kylin.properties.
> > > >
> > > > Any input is welcomed.
> > > >
> > > > --
> > > > Best regards,
> > > >
> > > > Shaofeng Shi =E5=8F=B2=E5=B0=91=E9=94=8B
> > > >
> > >
> >
>



--
=
Best regards,

Shaofeng Sh= i =E5=8F=B2=E5=B0=91=E9=94=8B

--001a11370ab01c94e0055ef059bb--