kylin-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Billy Liu <billy...@apache.org>
Subject Re: [DISCUSS] Upgrade Kylin's dependency to Hadoop 3 / HBase 2
Date Mon, 02 Mar 2020 08:41:19 GMT
+1. Let's move to Hadoop3

With Warm regards

Billy Liu


ShaoFeng Shi <shaofengshi@apache.org> 于2020年2月27日周四 下午10:07写道:

> Hi Yang,
>
> The main difference between 2.6 and 3.0 is the new real-time OLAP feature.
> Hadoop 2 users can select either of them, depends on whether they need the
> real-time feature.
>
> After 3.0, the next major features would be the Flink cube engine (planned
> in v3.1) and the Parquet storage (early stage, maybe in v4.0).
>
> When the parquet storage is released, as the dependency on HBase can be
> dropped, then we assume the API issue will easier than today. We can
> re-evaluate the possibility to support Hadoop 2.
>
> So I think the impact on today's Hadoop 2 users is acceptable. Not mention
> that they still can manually compile that.
>
>
> Best regards,
>
> Shaofeng Shi 史少锋
> Apache Kylin PMC
> Email: shaofengshi@apache.org
>
> Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html
> Join Kylin user mail group: user-subscribe@kylin.apache.org
> Join Kylin dev mail group: dev-subscribe@kylin.apache.org
>
>
>
>
> Li Yang <liyang@apache.org> 于2020年2月27日周四 上午7:37写道:
>
>> The proposal means Kylin 3.0 will be the last major version that supports
>> Hadoop 2.
>>
>> What will be recommended version for Hadoop 2 users after this? I feel the
>> latest stable version of 2.6 is better than 3.0.
>>
>> Anyway, I'm fine with moving focus to Hadoop 3. That is the direction.
>> However we shall also think about what it means for Hadoop 2 users.
>> Questions like below shall also be answered.
>>
>> - What is the recommended version/branch for Hadoop 2? (Btw, 3.0 does not
>> sound right here.)
>> - How that version/branch will be maintained?
>>
>> +1 in general
>>
>> Regards
>> -Yang
>>
>>
>> On Wed, Feb 26, 2020 at 5:36 PM Zhou Kang <zhoukangcn@outlook.com> wrote:
>>
>> > +1
>> >
>> >
>> > > 2020年2月26日 下午3:48,ShaoFeng Shi <shaofengshi@apache.org>
写道:
>> > >
>> > > Hello, Kylin users and developers,
>> > >
>> > > As we know Hadoop 3 and HBase 2 have released for some time. Kylin
>> > starts to support Hadoop 3  since v2.5.0 in Sep 2018.  As the APIs of
>> HBase
>> > 1 and 2 are incompatible, we need to keep different branches for them.
>> And
>> > in each release, we need to build separate packages and do a round of
>> > testing for them separately. Furthermore, Cloudera's API difference with
>> > the Apache release makes the situation worse; We need to build 4 binary
>> > packages for reach release. That has spent much of our manual effort and
>> > computing resources.
>> > >
>> > > Today, Hadoop 3 + HBase 2 becomes enough mature and stable for
>> > production use; And we see more and more users are starting to use the
>> new
>> > versions. We think it is time for Kylin to totally upgrade to the new
>> > version. So that we can focus more on Kylin itself, instead of
>> environments.
>> > >
>> > >  Here is my proposal:
>> > > 1) From Kylin 3.1,  Hadoop/HBase version upgrades to 3.1/2.1 (or a
>> close
>> > version);
>> > > 2) Hadoop 2 and HBase 1 users can use Kylin 3.0 and previous releases;
>> > > 3) We will re-evaluate the need for building binary packages for
>> > Cloudera release. (we may raise another discuss)
>> > >
>> > > Please let us know your comments. And please also understand with the
>> > limited resource we couldn't support multiple Hadoop versions...
>> > >
>> > > Thanks!
>> > >
>> > > Best regards,
>> > >
>> > > Shaofeng Shi 史少锋
>> > > Apache Kylin PMC
>> > > Email: shaofengshi@apache.org
>> > >
>> > > Apache Kylin FAQ:
>> https://kylin.apache.org/docs/gettingstarted/faq.html
>> > > Join Kylin user mail group: user-subscribe@kylin.apache.org
>> > > Join Kylin dev mail group: dev-subscribe@kylin.apache.org
>> > >
>> > >
>> >
>> >
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message