kylin-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From hosur narahari <hnr1...@gmail.com>
Subject Re: Create Views
Date Tue, 02 Oct 2018 04:48:30 GMT
Hi ShaoFeng,

It could be either like lambda architecture or just like merging cube data
with latest data for which cube has not yet been generated(in case of
periodic cube generation).

Now coming to aggregate pushdown. I'll give a very simple scenario.
Consider below query.

*Select sum(price) from kylin_sales*

when we execute it in any query engine like presto, spark, drill etc., it
doesn't execute above query but execute *Select price from kylin_sales.* After
getting all price values, it does *map-reduce* to calculate sum. It's good
for normal data sources but not for cube, since this is already
pre-calculated. Even if in future aggregate pushdown is provided, it'll be
very limited because we can't pushdown everything. For example, what do we
do when we've joins and then aggregate. And AFAIK, unlike indexes(used for
predicate pushdown), JDBC driver doesn't provide any information on pre
aggregated data.

How does views, solve this problem?

Let's say we create view  on above query like *Create view sum_view as
Select sum(price) from kylin_sales;*

In query engine we execute, *Select * from sum_view.* So kylin is queried
for sum_view, which is basically *Select sum(price) from kylin_sales.* In
this way, we can solve this problem using views.

Please let me know, if I'm not being clear.

Best Regards,
Hari

On Tue, Oct 2, 2018 at 8:52 AM ShaoFeng Shi <shaofengshi@apache.org> wrote:

> Hi Hari,
>
> Sorry for the late response. Does it like a lambda architecture? How can
> add a view to supporting aggregation pushdown? I'm not clear on this,
> please elaborate. Thank you!
>
> hosur narahari <hnr1992@gmail.com> 于2018年9月30日周日 下午1:50写道:
>
>> Hi ShaoFeng,
>>
>> Is it possible to provide that support. Because, in many cases cubes are
>> used with latest transaction data to get up to date analysis and some kind
>> of query engine will be used for merging cube with non-cube data. And most
>> of the query engines don't have aggregate pushdown, which makes it unable
>> to query from cube. If we provide view, we can solve this problem for all
>> query engines irrespective of whether they provide aggregate pushdown or
>> not, making kylin more adaptable.
>>
>> Also it's just a conceptual view, not adding any overhead.
>>
>> Best Regards,
>> Hari
>>
>> On Fri, Sep 28, 2018 at 7:40 PM ShaoFeng Shi <shaofengshi@apache.org>
>> wrote:
>>
>>> Hi Hari,
>>>
>>> Creating view on top of kylin tables is not supported in Kylin. Kylin is
>>> read-only.
>>>
>>> hosur narahari <hnr1992@gmail.com> 于2018年9月28日周五 下午1:09写道:
>>>
>>>> Hi,
>>>>
>>>> Is it possible to create views on the lines of rdbms or hive on top of
>>>> kylin tables.
>>>>
>>>> Best Regards,
>>>> Hari
>>>>
>>>
>>>
>>> --
>>> Best regards,
>>>
>>> Shaofeng Shi 史少锋
>>>
>>>
>
> --
> Best regards,
>
> Shaofeng Shi 史少锋
>
>

Mime
View raw message