kylin-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "240902713" <>
Subject 回复: kylin doesn't drop intermediate lookup table after job finished
Date Thu, 01 Aug 2019 06:33:21 GMT
Thanks, ShaoFeng

------------------ 原始邮件 ------------------
发件人: "ShaoFeng Shi"<>;
发送时间: 2019年7月31日(星期三) 晚上10:12
收件人: "user"<>;
抄送: "240902713"<>;
主题: Re: kylin doesn't drop intermediate lookup table after job finished


Good feedback. I think it is a legacy issue. At the very beginning, the intermediate table
name for the lookup table in view is likely to be the same. So if a job dropped the table
on the finish, it may cause another job failed. As a temporary solution, we disabled the cleanup
for Hive view tables.

Now since KYLIN-3515 has solved the conflict problem, I think this cleanup can be uncommented.
JIRA is created for this: If you want to
contribute, welcome to raise a PR. Thanks!

Best regards,

Shaofeng Shi 史少锋
Apache Kylin PMC

Apache Kylin FAQ:
Join Kylin user mail group:
Join Kylin dev mail group:

Xiaoxiang Yu <> 于2019年7月31日周三 上午1:27写道:

    I think these tables could be cleaned by StorageCleanupJob. Please refer to
to clean these intermediate table.

Best wishes to you ! 
From :Xiaoxiang Yu

At 2019-07-31 15:42:27, "240902713" <> wrote:
 Hello :

I have a  question about why kylin doesn't drop the intermediate lookup table after job finished.

My kylin version: 2.6.0

I have a cube with a fact table join a lookup table  in hive, and both are hive view. I submit
a job one time per hour.

Kylin can drop the intermediate fact table , but doesn't drop the intermediate lookup table

I check the source code , and find out that , at '13 step: Hive Cleanup' ,  the source code
has been commented. as the 

following picture shows:

After several days, in hive, there are a lot of intermediate lookup tables which are not droped.

Why can this step result some concurrent issue ? 

Looking forward your reply.

Thanks !
View raw message