kylin-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ShaoFeng Shi <shaofeng...@apache.org>
Subject Re: Segment overlap issues when refreshing a cube segment
Date Wed, 22 Aug 2018 13:20:54 GMT
Yes, KYLIN-3311 introduces a lock mechanism when updating metadata, that
can avoid concurrent updates on the same entity, so to avoid two duplicated
segments be inserted.

2018-08-22 19:52 GMT+08:00 <roberto.tardio@stratebi.com>:

> Many thanks ShaoFeng!
>
>
>
> Yes, I could see the new button because we also have a test installation
> of Kylin 2.4 and we are thinking about to update kylin of the production
> environment from 2.2 to 2.4.
>
>
>
> As for the API, I was aware of its existence. However, in the case I'm
> telling you we can't delete it because the segment is in the middle of
> other segments and in NEW state, duplicated. That's why I was talking about
> the JIRA advocacy https://issues.apache.org/jira/browse/KYLIN-3311  .  Do
> you think that the error causing the segment overlap is related to this
> JIRA?
>
>
>
> Kind Regards,
>
>
>
> Roberto
>
>
>
> *From:* ShaoFeng Shi [mailto:shaofengshi@apache.org]
> *Sent:* martes, 21 de agosto de 2018 14:54
> *To:* user <user@kylin.apache.org>
> *Subject:* Re: Segment overlap issues when refreshing a cube segment
>
>
>
> Hi Roberto, in the latest Kylin (I'm running), on the web GUI, there is a
> "Delete segment" button, which allows you to delete a segment easily.
>
>
>
> You can try to use the Rest API to delete a segment, this API should have
> existed for a long time: https://kylin.apache.org/docs/howto/howto_use_
> restapi.html#delete-segment
>
>
>
> I
>
>
>
> 2018-08-21 6:15 GMT+08:00 <roberto.tardio@stratebi.com>:
>
> Hi,
>
>
>
> I am using Kylin 2.2, where I have several OLAP cubes in which I am
> constantly adding and refreshing segments. I am faced with a problem that
> has generated two segments for the same period of time and therefore an
> error of the "segments overlap" type. Kylin is also cluster mounted, with
> one machine for cube building and one for querying. Here is the log of the
> query machine that seems to reflect the error (cube segment building
> process was successfully completed):
>
>
>
> 2018-08-16 09:19:35,833 INFO  [http-bio-7070-exec-93] cube.CubeManager:799
> : Reloaded cube trafico_cubo_v8 being CUBE[name=trafico_cubo_v8] having 20
> segments
>
> 2018-08-16 09:19:35,833 DEBUG [http-bio-7070-exec-93]
> cachesync.Broadcaster:256 : BroadcastingUPDATE, project_data,
> My_Cube_Trafico
>
> 2018-08-16 09:19:35,834 INFO  [http-bio-7070-exec-93]
> service.CacheService:143 : removeOLAPDataSource is called for project
> My_Cube_Trafico
>
> 2018-08-16 09:19:35,834 INFO  [http-bio-7070-exec-93]
> service.CacheService:125 : cleaning cache for project My_Cube_Trafico
> (currently remove all entries)
>
> 2018-08-16 09:19:35,834 DEBUG [http-bio-7070-exec-93]
> cachesync.Broadcaster:290 : Done broadcastingUPDATE, project_data,
> My_Cube_Trafico
>
> 2018-08-16 09:19:35,834 DEBUG [http-bio-7070-exec-93]
> cachesync.Broadcaster:290 : Done broadcastingUPDATE, cube, trafico_cubo_v8
>
> 2018-08-16 09:19:36,817 INFO  [http-bio-7070-exec-104]
> cube.CubeManager:358 : Updating cube instance 'plataforma_cubo_mes_v2'
>
> *2018-08-16 09:19:36,817 ERROR [http-bio-7070-exec-104]
> controller.CubeController:337 : Segments overlap:
> plataforma_cubo_mes_v2[20180801000000_20180901000000] and
> plataforma_cubo_mes_v2[20180801000000_20180901000000]*
>
> 2018-08-16 09:19:36,818 ERROR [http-bio-7070-exec-104]
> controller.BasicController:57 :
>
> 2018-08-16 09:19:37,264 DEBUG [http-bio-7070-exec-93]
> cachesync.Broadcaster:256 : BroadcastingUPDATE, cube, trafico_cubo_v8
>
> 2018-08-16 09:19:37,265 INFO  [http-bio-7070-exec-93] cube.CubeManager:799
> : Reloaded cube trafico_cubo_v8 being CUBE[name=trafico_cubo_v8] having 20
> segments
>
>
>
> *Could this error be related to the JIRA entry
> https://issues.apache.org/jira/browse/KYLIN-3311
> <https://issues.apache.org/jira/browse/KYLIN-3311>?* In that case, I see
> that it has been fixed in version 2.4 of Kylin.
>
>
>
> Also, to fix the error, we had to stop Kylin, remove the duplicate segment
> from the metadata, restart Kylin and then refresh the segment. As far as I
> can read on the website https://issues.apache.org/jira/browse/KYLIN-2849
> , from version 2.3 onwards you can delete a segment in any position from
> the UI or the Kylin API after disabling the cube. *Is that correct?* This
> would help us mitigate the problem more quickly.
>
>
>
> Thanks in advance!
>
> *Roberto Tardío Olmos*
>
> *Head of Big Data Analytics*
>
> Avenida de Brasil, 17
> <https://maps.google.com/?q=Avenida+de+Brasil,+17&entry=gmail&source=g>,
> Planta 16.28020 Madrid
>
> Fijo: 91.788.34.10
>
>
> [image:
> http://www.stratebi.com/image/layout_set_logo?img_id=21615&t=1486381163544]
>
>
>
> http://bigdata.stratebi.com/
>
>
>
> http://www.stratebi.com
>
>
>
>
>
>
>
> --
>
> Best regards,
>
>
>
> Shaofeng Shi 史少锋
>
>
>



-- 
Best regards,

Shaofeng Shi 史少锋

Mime
View raw message