kylin-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "kangkaisen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KYLIN-3430) Global Dictionary Cleanup
Date Fri, 06 Jul 2018 10:14:00 GMT

    [ https://issues.apache.org/jira/browse/KYLIN-3430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16534657#comment-16534657
] 

kangkaisen commented on KYLIN-3430:
-----------------------------------

Hi, [~temple.zhou].

There is also a config "kylin.dictionary.append-version-ttl", the default value is three days.  which
means we will not delete the dir Within three days.

 

The Global Dictionary is not cube level, multiple cubes could use the same Global Dictionary. 
So we shouldn't delete Global Dictionary when we drop one cube.

 

But you remind me,  if the Global Dictionary related cube are all dropped, currently, we
will never delete the Global Dictionary in HDFS.  This is a issue indeed, thanks you very
much!  I thought wrong.

 

As for how to check a Global Dictionary whether has reference or not, I think we could get
the active Global Dictionary from the cube metadata, if all cube don't use a Global Dictionary,
we could think the Global Dictionary is useless and delete it.

Thanks you !

 

 

> Global Dictionary Cleanup
> -------------------------
>
>                 Key: KYLIN-3430
>                 URL: https://issues.apache.org/jira/browse/KYLIN-3430
>             Project: Kylin
>          Issue Type: Improvement
>          Components: Tools, Build and Test
>    Affects Versions: v2.1.0, v2.2.0, v2.3.0, v2.3.1, v2.4.0
>            Reporter: Temple Zhou
>            Assignee: Temple Zhou
>            Priority: Major
>         Attachments: KYLIN-3430.master.001.patch
>
>
> I had run "{{./bin/metastore.sh clean --delete true" to cleanup my Kylin metadata, but,
after that, the Global Dictionary still exists in my HDFS and the size of directory "/kylin_metadata/resources/GlobalDict/dict"
hasn't shrunk.}}
>  
> {{BTW: I'm very sure that there are redundant Global Dictionaries.}}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message