kylin-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xiaoxiang Yu (Jira)" <j...@apache.org>
Subject [jira] [Assigned] (KYLIN-5019) Avoid building global dictionary from all data of fact table each time
Date Wed, 11 Aug 2021 02:10:00 GMT

     [ https://issues.apache.org/jira/browse/KYLIN-5019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Xiaoxiang Yu reassigned KYLIN-5019:
-----------------------------------

    Fix Version/s: v4.0.0
         Assignee: Shengjun Zheng

> Avoid building global dictionary from all data of fact table each time
> ----------------------------------------------------------------------
>
>                 Key: KYLIN-5019
>                 URL: https://issues.apache.org/jira/browse/KYLIN-5019
>             Project: Kylin
>          Issue Type: New Feature
>    Affects Versions: v4.0.0-beta
>            Reporter: Shengjun Zheng
>            Assignee: Shengjun Zheng
>            Priority: Major
>             Fix For: v4.0.0
>
>
> Global dictionaries will be resized to avoid single dictionary bucket exceeding its threashold
in each cubing job. To judge whether a dictionary should be resized, Kylin will read all data
from fact table and do count distinct operation. Then rebuild dictionary from all history
data.
> It’s better to use a lightweight approach, since dictionary don't need to be resized
for incremental building in most cases.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message