kylin-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Luke Han <luke...@gmail.com>
Subject Re: [DISCUSS] about cluster management
Date Tue, 03 Nov 2015 01:06:23 GMT
Run on YARN requires to run kylin on "one" Hadoop cluster.

Based on today's practices, Kylin's services always sit beside clusters not
inside to have capability to interactive with more than 2 clusters with
just one Kylin deployment.






Best Regards!
---------------------

Luke Han

On Tue, Nov 3, 2015 at 12:12 AM, Henry Saputra <henry.saputra@gmail.com>
wrote:

> Why dont we just leverage Hadoop YARN?
>
> We could use Apache Twill [1] to wrap it as YARN application.
>
> Helix is great but one of the limitations it could not provision new
> node when not already available.
>
> [1] http://twill.incubator.apache.org/
>
> On Sun, Nov 1, 2015 at 7:01 PM, ć‘šćƒæ˜Š <qhzhou@apache.org> wrote:
> > Hi, all
> >      Kylin currently manages the cluster itself, so far it works fine
> since
> > there are only 2 roles for the cluster, job engine and query engine.
> >      In order to reduce the complexity of deployment and support the
> coming
> > streaming feature, I suggest kylin have a better cluster management.
> >      There are several candidates I think can be leveraged. Apache Helix
> > seem suitable for Kylin in my opinion.
> >      I would love to start a discussion about cluster manager of Kylin,
> how
> > we can do it the better way, and which component we should choose.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message