kylin-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Temple Zhou (JIRA)" <>
Subject [jira] [Commented] (KYLIN-3430) Global Dictionary Cleanup
Date Thu, 05 Jul 2018 03:20:00 GMT


Temple Zhou commented on KYLIN-3430:

Hi Shaofeng,

I have no objection to the 1) & 2), but the 3).

I do not check about whether a dictionary has reference or not, as I found that even the GD
was being used, there were still redundant GD.

hdfs@t1:~$ hdfs dfs -du -h /kylin/kylin_metadata/resources/GlobalDict/dict/DEFAULT.TEST_GLOBAL_DICT/USER_ID/
148  444  /kylin/kylin_metadata/resources/GlobalDict/dict/DEFAULT.TEST_GLOBAL_DICT/USER_ID/version_1530759906521
157  471  /kylin/kylin_metadata/resources/GlobalDict/dict/DEFAULT.TEST_GLOBAL_DICT/USER_ID/version_1530759913804
166  498  /kylin/kylin_metadata/resources/GlobalDict/dict/DEFAULT.TEST_GLOBAL_DICT/USER_ID/version_1530759915097

Besides, I analyzed the contents of GD and found that the newer GD seems to contain everything
in the older GD, so I think only the newest GD should be reserved, even the GD is being used
by active cube. I don't know if I express it clearly.

> Global Dictionary Cleanup
> -------------------------
>                 Key: KYLIN-3430
>                 URL:
>             Project: Kylin
>          Issue Type: Improvement
>          Components: Tools, Build and Test
>    Affects Versions: v2.1.0, v2.2.0, v2.3.0, v2.3.1, v2.4.0
>            Reporter: Temple Zhou
>            Assignee: Temple Zhou
>            Priority: Major
>         Attachments: KYLIN-3430.master.001.patch
> I had run "{{./bin/ clean --delete true" to cleanup my Kylin metadata, but,
after that, the Global Dictionary still exists in my HDFS and the size of directory "/kylin_metadata/resources/GlobalDict/dict"
hasn't shrunk.}}
> {{BTW: I'm very sure that there are redundant Global Dictionaries.}}

This message was sent by Atlassian JIRA

View raw message