kylin-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ruslan Dautkhanov (JIRA)" <j...@apache.org>
Subject [jira] [Created] (KYLIN-3138) cuboids on-demand build
Date Thu, 28 Dec 2017 19:19:03 GMT
Ruslan Dautkhanov created KYLIN-3138:
----------------------------------------

             Summary: cuboids on-demand build
                 Key: KYLIN-3138
                 URL: https://issues.apache.org/jira/browse/KYLIN-3138
             Project: Kylin
          Issue Type: New Feature
          Components: General, Job Engine, Query Engine, Spark Engine
    Affects Versions: v2.2.0
            Reporter: Ruslan Dautkhanov
            Assignee: Shaofeng SHI
            Priority: Critical


We just started using Kylin and quite like it so far.

Although some of the datasets we have a quite like to even consider for OLAP cubing.
Unless those cuboids will be built on-demand.
I know some commercial non-open source products do this successfully. 

This idea is to build a cuboid only when a user actually needs it. 
So for example, our BI dashboards does a certain rollup, so then a SQL
query hits Kylin backend. Kylin realizes it hasn't built that particular cuboid just yet,
so immediately starts building it. Users has to wait a bit longer first time
it request that combination of dimensions. But all other requests or requests 
of other users will be fast from that point on.

Kylin (or any other OLAP solution) wouldn't be feasible to use on very wide datasets 
unless this on-demand functionality is implemented. For example, some datasets we have have
100-200 dimensions. And we don't know up front rollups users would want to do.

Suggesting to have a new dimension build rule "lazy / on-demand". All previous rules apply.
This new rule type would mean, a cuboid for a particular set of dimensions wouldn't be built
up-front if it's marked as "lazy / on-demand". 

Thoughts / ideas?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message