From user-return-3555-archive-asf-public=cust-asf.ponee.io@kylin.apache.org Fri Sep 28 16:27:38 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id AB71E180627 for ; Fri, 28 Sep 2018 16:27:37 +0200 (CEST) Received: (qmail 33801 invoked by uid 500); 28 Sep 2018 14:27:36 -0000 Mailing-List: contact user-help@kylin.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@kylin.apache.org Delivered-To: mailing list user@kylin.apache.org Received: (qmail 33792 invoked by uid 99); 28 Sep 2018 14:27:36 -0000 Received: from mail-relay.apache.org (HELO mailrelay2-lw-us.apache.org) (207.244.88.137) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 28 Sep 2018 14:27:36 +0000 Received: from mail-it1-f175.google.com (mail-it1-f175.google.com [209.85.166.175]) by mailrelay2-lw-us.apache.org (ASF Mail Server at mailrelay2-lw-us.apache.org) with ESMTPSA id DE9632392 for ; Fri, 28 Sep 2018 14:27:35 +0000 (UTC) Received: by mail-it1-f175.google.com with SMTP id 74-v6so2686340itw.1 for ; Fri, 28 Sep 2018 07:27:35 -0700 (PDT) X-Gm-Message-State: ABuFfoipaU0qgY7MIsz0n4u+m7RbXvlzxrzsdS9WRqbY1WnSN7HUHhut I1+hm3YgOXOB9JfF7Lo1Z9wMU7OsT4aie32RBrU= X-Google-Smtp-Source: ACcGV62pWnSZ0H3eRbyrEobpwCUdId/gSWYYnnntJvLHFwMYjPffB5GfP22TQDCMYlb5AvFiAWs1C6g1Ow3j6bozfho= X-Received: by 2002:a02:1103:: with SMTP id 3-v6mr13228569jaf.144.1538144855453; Fri, 28 Sep 2018 07:27:35 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: ShaoFeng Shi Date: Fri, 28 Sep 2018 22:26:58 +0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: How to build necessary cuboids rather than building all cuboidswhenadding or removing one dimension To: user Content-Type: multipart/alternative; boundary="000000000000b8f46d0576ef41b8" --000000000000b8f46d0576ef41b8 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable The Cube planner can run an "Optimize job" to add/remove new/deprecated cuboids. As it won't change the Cube's structure (dimension/measures), so it can run on an existing Cube. If you want to add/remove dimension or measures, usually a new cube need be created. If you want to reuse the existing cube's data, in the new cube, you can just specify the new cuboids in some way. Let's give an example; Say now I have a Cube A, which has two dimensions: D1, D2; It would have 3 cuboids (Kylin skips 0 cuboid): [D1, D2], [D1], [D2] Now I want to add new dimension D3, then I create Cube B with D1, D2, and D3; By default, it would have 7 cuboids. If you want to just build the new cuboids, in B you can define D3 as "mandatory" cuboids, then Cube B only build the cuboids which have D3 presented: [D1, D2, D3], [D1, D3], [D2, D3], [D3] For queries that have no D3, Kylin will select cube A to answer. Others to cube B. But you need to ensure cube A and B has the same range of data be fed to ensure the data consistency. Just my two cents. Chao Long =E4=BA=8E2018=E5=B9=B49=E6=9C=8828=E6=97=A5=E5= =91=A8=E4=BA=94 =E4=B8=8B=E5=8D=883:43=E5=86=99=E9=81=93=EF=BC=9A > May i know your usage scenerio? As far as I know, if a cube has already > been built, then the structure of the cube can not be modified, ie., you > can't add or remove dimension. But if it has not been built yet, the > structure can be modified. For now, it is not workable to build the > modified cuboid only. > > > ------------------ =E5=8E=9F=E5=A7=8B=E9=82=AE=E4=BB=B6 -----------------= - > *=E5=8F=91=E4=BB=B6=E4=BA=BA:* "you Zhuang"; > *=E5=8F=91=E9=80=81=E6=97=B6=E9=97=B4:* 2018=E5=B9=B49=E6=9C=8827=E6=97= =A5(=E6=98=9F=E6=9C=9F=E5=9B=9B) =E4=B8=8B=E5=8D=883:15 > *=E6=94=B6=E4=BB=B6=E4=BA=BA:* "user"; > *=E4=B8=BB=E9=A2=98:* Re: How to build necessary cuboids rather than buil= ding all > cuboidswhenadding or removing one dimension > > Hybrid model seems difficult to be managed cohesively , is there a way to > modify the cube , and then build modified cuboids only? > > On 26 Sep 2018, at 3:42 PM, Chao Long wrote: > > Hybrid is to consolidate history cube and new cube. And once a cube is > built, its structure couldn=E2=80=99t be modified. To add/remove dimensio= n, you > need to clone a new cube, then add/remove in the new cube. And if you can > accept the absence of new dimensions for historical data, you can build t= he > new cube since the end time of the old cube. And then create a hybrid mod= el > over the old and new cube. > > > ------------------ Original ------------------ > *From: * "you Zhuang"; > *Date: * Tue, Sep 25, 2018 10:39 AM > *To: * "user"; > *Subject: * How to build necessary cuboids rather than building all > cuboids whenadding or removing one dimension > > Hybrid cube does not affect all segments, and not cover removing > situation. So how to build the related adding or removing dimension cuboi= ds > only? > > > --=20 Best regards, Shaofeng Shi =E5=8F=B2=E5=B0=91=E9=94=8B --000000000000b8f46d0576ef41b8 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
The Cube planner can run an "Optimize job" to ad= d/remove new/deprecated cuboids. As it won't change the Cube's stru= cture (dimension/measures), so it can run on an existing Cube.

If you want to add/remove dimension or measures, usually a new cube = need be created. If you want to reuse the=C2=A0existing cube's data, in= the new cube, you can just specify the new cuboids in some way.
=
Let's give an example; Say now I have a Cube A, which ha= s two dimensions: D1, D2;

It would have 3 cuboids = (Kylin skips 0 cuboid):
[D1, D2], [D1], [D2]

=
Now I want to add new dimension D3, then I create Cube B with D1, D2, = and D3; By default, it would have 7 cuboids. If you want to just build the = new cuboids, in B you can define D3 as "mandatory" cuboids, then = Cube B only build the cuboids which have=C2=A0D3 presented:
[D1, = D2, D3], [D1, D3], [D2, D3], [D3]

For queries that= have=C2=A0no D3, Kylin will select cube A to answer. Others to cube B. But= you need to ensure cube A and B has the same range of data be fed to ensur= e the data consistency.

Just my two cents.

=C2=A0

Chao Long <wayne.l@qq.com= > =E4=BA=8E2018=E5=B9=B49=E6=9C=8828=E6=97=A5=E5=91=A8=E4=BA=94 =E4=B8= =8B=E5=8D=883:43=E5=86=99=E9=81=93=EF=BC=9A
May i know your usage scenerio? As far as I know,=C2=A0if a c= ube has already been built, then the structure of the cube can not be modif= ied, ie., you can't add or remove dimension. But if it has not been bui= lt yet, the structure can be modified. For now, it is not workable to build= the modified cuboid only.


--------= ----------=C2=A0=E5=8E=9F=E5=A7=8B=E9=82=AE=E4=BB=B6=C2=A0-----------------= -
=E5=8F=91=E4=BB=B6=E4=BA=BA:=C2=A0"you Zhuang"<zhuangzixiaoyou@gmail.= com>;
=E5=8F=91=E9=80=81=E6=97=B6=E9=97=B4:=C2=A020= 18=E5=B9=B49=E6=9C=8827=E6=97=A5(=E6=98=9F=E6=9C=9F=E5=9B=9B) =E4=B8=8B=E5= =8D=883:15
=E6=94=B6=E4=BB=B6=E4=BA=BA:=C2=A0"user&qu= ot;<user@kyli= n.apache.org>;
=E4=B8=BB=E9=A2=98:=C2=A0= Re: How to build necessary cuboids rather than building all cuboidswhenaddi= ng or removing one dimension

Hybrid model seems d= ifficult to be managed cohesively , is there a way to modify the cube , and= then build modified cuboids only?

On 26 Sep 2018, at 3:42 PM, Chao Long <wayne.l@qq.com> wrote:

Hybrid is to consolida= te history cube and new cube. And once a cube is built, its structure could= n=E2=80=99t be modified. To add/remove dimension, you need to clone a new c= ube, then add/remove in the new cube. And if you can accept the absence of = new dimensions for historical data, you can build the new cube since the en= d time of the old cube. And then create a hybrid model over the old and new= cube.


------------------ Original = ------------------
From: =C2=A0"you Zhuang"<zhuangzixiaoyou@gmail.com>;
= Subject: =C2=A0How to build necessary cuboids rather than building all = cuboids whenadding or removing one dimension

Hyb= rid cube does not affect all segments, and not cover removing situation. So= how to build the related adding or removing dimension cuboids only?
<= /div>



--
Best regards,
Shaofeng Shi =E5=8F=B2=E5=B0=91=E9=94=8B

--000000000000b8f46d0576ef41b8--