kylin-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From hosur narahari <hnr1...@gmail.com>
Subject Re: Create Views
Date Sat, 06 Oct 2018 13:25:40 GMT
Hi ShaoFeng,

Exactly. It will solve the problem for all query engines whatsoever.

Best Regards,
Hari

On Sat, Oct 6, 2018 at 6:51 PM ShaoFeng Shi <shaofengshi@apache.org> wrote:

> Hi Hari,
>
> I see now; it is very similar to this integration with spark:
> https://kylin.apache.org/docs/tutorial/spark.html
>
> Spark can register a SQL as a temp table, and then use that temp table for
> subsequent analysis. But other engines may not have this function, so a
> common view in Kylin may help to fill that gap, am I correct?
>
>
>
> hosur narahari <hnr1992@gmail.com> 于2018年10月2日周二 下午12:48写道:
>
>> Hi ShaoFeng,
>>
>> It could be either like lambda architecture or just like merging cube
>> data with latest data for which cube has not yet been generated(in case of
>> periodic cube generation).
>>
>> Now coming to aggregate pushdown. I'll give a very simple scenario.
>> Consider below query.
>>
>> *Select sum(price) from kylin_sales*
>>
>> when we execute it in any query engine like presto, spark, drill etc., it
>> doesn't execute above query but execute *Select price from kylin_sales.* After
>> getting all price values, it does *map-reduce* to calculate sum. It's
>> good for normal data sources but not for cube, since this is already
>> pre-calculated. Even if in future aggregate pushdown is provided, it'll be
>> very limited because we can't pushdown everything. For example, what do we
>> do when we've joins and then aggregate. And AFAIK, unlike indexes(used for
>> predicate pushdown), JDBC driver doesn't provide any information on pre
>> aggregated data.
>>
>> How does views, solve this problem?
>>
>> Let's say we create view  on above query like *Create view sum_view as
>> Select sum(price) from kylin_sales;*
>>
>> In query engine we execute, *Select * from sum_view.* So kylin is
>> queried for sum_view, which is basically *Select sum(price) from
>> kylin_sales.* In this way, we can solve this problem using views.
>>
>> Please let me know, if I'm not being clear.
>>
>> Best Regards,
>> Hari
>>
>> On Tue, Oct 2, 2018 at 8:52 AM ShaoFeng Shi <shaofengshi@apache.org>
>> wrote:
>>
>>> Hi Hari,
>>>
>>> Sorry for the late response. Does it like a lambda architecture? How can
>>> add a view to supporting aggregation pushdown? I'm not clear on this,
>>> please elaborate. Thank you!
>>>
>>> hosur narahari <hnr1992@gmail.com> 于2018年9月30日周日 下午1:50写道:
>>>
>>>> Hi ShaoFeng,
>>>>
>>>> Is it possible to provide that support. Because, in many cases cubes
>>>> are used with latest transaction data to get up to date analysis and some
>>>> kind of query engine will be used for merging cube with non-cube data. And
>>>> most of the query engines don't have aggregate pushdown, which makes it
>>>> unable to query from cube. If we provide view, we can solve this problem
>>>> for all query engines irrespective of whether they provide aggregate
>>>> pushdown or not, making kylin more adaptable.
>>>>
>>>> Also it's just a conceptual view, not adding any overhead.
>>>>
>>>> Best Regards,
>>>> Hari
>>>>
>>>> On Fri, Sep 28, 2018 at 7:40 PM ShaoFeng Shi <shaofengshi@apache.org>
>>>> wrote:
>>>>
>>>>> Hi Hari,
>>>>>
>>>>> Creating view on top of kylin tables is not supported in Kylin. Kylin
>>>>> is read-only.
>>>>>
>>>>> hosur narahari <hnr1992@gmail.com> 于2018年9月28日周五 下午1:09写道:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Is it possible to create views on the lines of rdbms or hive on top
>>>>>> of kylin tables.
>>>>>>
>>>>>> Best Regards,
>>>>>> Hari
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Best regards,
>>>>>
>>>>> Shaofeng Shi 史少锋
>>>>>
>>>>>
>>>
>>> --
>>> Best regards,
>>>
>>> Shaofeng Shi 史少锋
>>>
>>>
>
> --
> Best regards,
>
> Shaofeng Shi 史少锋
>
>

Mime
View raw message