kylin-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From hosur narahari <hnr1...@gmail.com>
Subject Re: Create Views
Date Sat, 06 Oct 2018 14:58:11 GMT
Hi ShaoFeng,

Sure I'll do.

Best Regards,
Hari

On Sat, Oct 6, 2018 at 8:26 PM ShaoFeng Shi <shaofengshi@apache.org> wrote:

> Hi Hari,
>
> Would you like to open a JIRA to Kylin with this? Thanks!
>
> hosur narahari <hnr1992@gmail.com> 于2018年10月6日周六 下午9:25写道:
>
>> Hi ShaoFeng,
>>
>> Exactly. It will solve the problem for all query engines whatsoever.
>>
>> Best Regards,
>> Hari
>>
>> On Sat, Oct 6, 2018 at 6:51 PM ShaoFeng Shi <shaofengshi@apache.org>
>> wrote:
>>
>>> Hi Hari,
>>>
>>> I see now; it is very similar to this integration with spark:
>>> https://kylin.apache.org/docs/tutorial/spark.html
>>>
>>> Spark can register a SQL as a temp table, and then use that temp table
>>> for subsequent analysis. But other engines may not have this function, so a
>>> common view in Kylin may help to fill that gap, am I correct?
>>>
>>>
>>>
>>> hosur narahari <hnr1992@gmail.com> 于2018年10月2日周二 下午12:48写道:
>>>
>>>> Hi ShaoFeng,
>>>>
>>>> It could be either like lambda architecture or just like merging cube
>>>> data with latest data for which cube has not yet been generated(in case of
>>>> periodic cube generation).
>>>>
>>>> Now coming to aggregate pushdown. I'll give a very simple scenario.
>>>> Consider below query.
>>>>
>>>> *Select sum(price) from kylin_sales*
>>>>
>>>> when we execute it in any query engine like presto, spark, drill etc.,
>>>> it doesn't execute above query but execute *Select price from
>>>> kylin_sales.* After getting all price values, it does *map-reduce* to
>>>> calculate sum. It's good for normal data sources but not for cube, since
>>>> this is already pre-calculated. Even if in future aggregate pushdown is
>>>> provided, it'll be very limited because we can't pushdown everything. For
>>>> example, what do we do when we've joins and then aggregate. And AFAIK,
>>>> unlike indexes(used for predicate pushdown), JDBC driver doesn't provide
>>>> any information on pre aggregated data.
>>>>
>>>> How does views, solve this problem?
>>>>
>>>> Let's say we create view  on above query like *Create view sum_view as
>>>> Select sum(price) from kylin_sales;*
>>>>
>>>> In query engine we execute, *Select * from sum_view.* So kylin is
>>>> queried for sum_view, which is basically *Select sum(price) from
>>>> kylin_sales.* In this way, we can solve this problem using views.
>>>>
>>>> Please let me know, if I'm not being clear.
>>>>
>>>> Best Regards,
>>>> Hari
>>>>
>>>> On Tue, Oct 2, 2018 at 8:52 AM ShaoFeng Shi <shaofengshi@apache.org>
>>>> wrote:
>>>>
>>>>> Hi Hari,
>>>>>
>>>>> Sorry for the late response. Does it like a lambda architecture? How
>>>>> can add a view to supporting aggregation pushdown? I'm not clear on this,
>>>>> please elaborate. Thank you!
>>>>>
>>>>> hosur narahari <hnr1992@gmail.com> 于2018年9月30日周日 下午1:50写道:
>>>>>
>>>>>> Hi ShaoFeng,
>>>>>>
>>>>>> Is it possible to provide that support. Because, in many cases cubes
>>>>>> are used with latest transaction data to get up to date analysis
and some
>>>>>> kind of query engine will be used for merging cube with non-cube
data. And
>>>>>> most of the query engines don't have aggregate pushdown, which makes
it
>>>>>> unable to query from cube. If we provide view, we can solve this
problem
>>>>>> for all query engines irrespective of whether they provide aggregate
>>>>>> pushdown or not, making kylin more adaptable.
>>>>>>
>>>>>> Also it's just a conceptual view, not adding any overhead.
>>>>>>
>>>>>> Best Regards,
>>>>>> Hari
>>>>>>
>>>>>> On Fri, Sep 28, 2018 at 7:40 PM ShaoFeng Shi <shaofengshi@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Hari,
>>>>>>>
>>>>>>> Creating view on top of kylin tables is not supported in Kylin.
>>>>>>> Kylin is read-only.
>>>>>>>
>>>>>>> hosur narahari <hnr1992@gmail.com> 于2018年9月28日周五
下午1:09写道:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> Is it possible to create views on the lines of rdbms or hive
on top
>>>>>>>> of kylin tables.
>>>>>>>>
>>>>>>>> Best Regards,
>>>>>>>> Hari
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Best regards,
>>>>>>>
>>>>>>> Shaofeng Shi 史少锋
>>>>>>>
>>>>>>>
>>>>>
>>>>> --
>>>>> Best regards,
>>>>>
>>>>> Shaofeng Shi 史少锋
>>>>>
>>>>>
>>>
>>> --
>>> Best regards,
>>>
>>> Shaofeng Shi 史少锋
>>>
>>>
>
> --
> Best regards,
>
> Shaofeng Shi 史少锋
>
>

Mime
View raw message