Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 576D7200C04 for ; Tue, 24 Jan 2017 10:01:37 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 56251160B6A; Tue, 24 Jan 2017 09:01:37 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 9D231160B4B for ; Tue, 24 Jan 2017 10:01:36 +0100 (CET) Received: (qmail 60249 invoked by uid 500); 24 Jan 2017 09:01:34 -0000 Mailing-List: contact user-help@kylin.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@kylin.apache.org Delivered-To: mailing list user@kylin.apache.org Received: (qmail 60186 invoked by uid 99); 24 Jan 2017 09:01:34 -0000 Received: from mail-relay.apache.org (HELO mail-relay.apache.org) (140.211.11.15) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 24 Jan 2017 09:01:34 +0000 Received: from mail-yb0-f181.google.com (mail-yb0-f181.google.com [209.85.213.181]) by mail-relay.apache.org (ASF Mail Server at mail-relay.apache.org) with ESMTPSA id 204451A0143 for ; Tue, 24 Jan 2017 09:01:34 +0000 (UTC) Received: by mail-yb0-f181.google.com with SMTP id w194so95688181ybe.0 for ; Tue, 24 Jan 2017 01:01:34 -0800 (PST) X-Gm-Message-State: AIkVDXIIuBHSSLsWhkGkSu1fLaG1o6yDrThmPnUl/kyEMh2zjW8iBTPfS8Nv/t1ghYaFcDwRnFXybdZGAluHyw== X-Received: by 10.37.177.4 with SMTP id g4mr10051057ybj.155.1485248493278; Tue, 24 Jan 2017 01:01:33 -0800 (PST) MIME-Version: 1.0 Received: by 10.13.199.1 with HTTP; Tue, 24 Jan 2017 01:00:52 -0800 (PST) In-Reply-To: <435EFE78CE899744B5D765672BE65977AD0D9B@CYY-MAILBOX04.tcent.cn> References: <435EFE78CE899744B5D765672BE65977AD0D9B@CYY-MAILBOX04.tcent.cn> From: ShaoFeng Shi Date: Tue, 24 Jan 2017 17:00:52 +0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: more than 1 append dict for globalDict To: user Content-Type: multipart/alternative; boundary=f403045f337cd878510546d35c5f archived-at: Tue, 24 Jan 2017 09:01:37 -0000 --f403045f337cd878510546d35c5f Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hi zhangda, Do you have multiple segments (which has the count distinct measure) building concurrently? In 1.6.0 and before there is a concurrent bug I think; Yerui Sun fixed it in https://issues.apache.org/jira/browse/KYLIN-2192 So, please check whether it is this problem first; If yes, you need add some control (lock) to avoid concurrent build for this cube. 2017-01-24 16:33 GMT+08:00 =E5=B8=82=E5=9C=BA=E4=B8=AD=E5=BF=83-ZHANGDA3269= 8 : > Hi there, > > > > I have a cube computing for some UV statistics. Since it requires global > count distinct operation, in the advanced dictionaries setting I put UV k= ey > column=E2=80=99s builder class as =E2=80=98GlobalDictionaryBuilder=E2=80= =99. > > Everyday build went on without any problem until yesterday there was a > =E2=80=98GlobalDict xxxx should have 0 or 1 append dict but 2"=E2=80=99 e= xception. I > checked the =E2=80=98kylin_metadata=E2=80=99 table in the hbase , and saw= there were indeed > 2 dict > > '/dict/APPLYDATA_DSDCADMIN.FLY_ZHUANGHUALV_PROC1/ > LOGINKEY/ae5a65ca-022f-4c81-89f2-7cacb2789888.dict' > > /dict/APPLYDATA_DSDCADMIN.FLY_ZHUANGHUALV_PROC1/LOGINKEY/ > cedce514-657d-4032-a591-7f01d984df65.dict' > > Fyi, APPLYDATA_DSDCADMIN.FLY_ZHUANGHUALV_PROC1 is the hive table name, > LOGINKEY is the count distinct key > > I=E2=80=99m not sure what had gone wrong, and I suppose global dict on 1 = key > should be unique. Has anyone ever encountered similar error before, can > share some ideas? Thanks! > --=20 Best regards, Shaofeng Shi =E5=8F=B2=E5=B0=91=E9=94=8B --f403045f337cd878510546d35c5f Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hi zhangda,

Do you have multiple segmen= ts (which has the count distinct measure) building concurrently? In 1.6.0 a= nd before there is a concurrent bug I think; Yerui Sun fixed it in https://issues.apache= .org/jira/browse/KYLIN-2192
=C2=A0
So, please c= heck whether it is this problem first; If yes, you need add some control (l= ock) to avoid concurrent build for this cube.

2017-01-24 16:33 GMT+08:00 =E5=B8= =82=E5=9C=BA=E4=B8=AD=E5=BF=83-ZHANGDA32698 <zd32698@ly.com>:=

Hi there,<= /p>

=C2=A0

I have a cube computing for som= e UV statistics. Since it requires global count distinct operation, in the = advanced dictionaries setting I put UV key column=E2=80=99s builder class a= s =E2=80=98GlobalDictionaryBuilder=E2=80=99.

Everyday build went on without = any problem until yesterday there was a =E2=80=98GlobalDict xxxx should hav= e 0 or 1 append dict but 2"=E2=80=99 exception. I checked the =E2=80= =98kylin_metadata=E2=80=99 table in the hbase , and saw there were indeed 2 dict

'/dict/APPLYDATA_DSDCADMIN.= FLY_ZHUANGHUALV_PROC1/LOGINKEY/ae5a65ca-022f-4c81-89f2-7cacb= 2789888.dict'

/dict/APPLYDATA_DSDCADMIN.FLY_<= wbr>ZHUANGHUALV_PROC1/LOGINKEY/cedce514-657d-4032-a591-7f01d984df= 65.dict'

Fyi, APPLYDATA_DSDCADMIN.FLY_ZHUANGHUALV_PROC1 is the hive table name, LOGINKEY is the count distinct= key

I=E2=80=99m not sure what had g= one wrong, and I suppose global dict on 1 key should be unique.=C2=A0 Has a= nyone ever encountered similar error before, can share some ideas? Thanks!<= u>




--
Best regards,

Shaofeng Shi =E5=8F=B2= =E5=B0=91=E9=94=8B

--f403045f337cd878510546d35c5f--