Mailing-List: contact dev-help@kylin.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@kylin.incubator.apache.org
MIME-Version: 1.0
In-Reply-To: 
 <CALuGr6YugQF_oWKCc63DcC0xnbY42nKsPmZOBe6csoargMansA@mail.gmail.com>
References: 
 <CAD6wy_3Z5hqZH5AeNT8fDdFbjSJHKGB1uFApYeAaFhk5NM+9vA@mail.gmail.com>
	<CALuGr6YugQF_oWKCc63DcC0xnbY42nKsPmZOBe6csoargMansA@mail.gmail.com>
Date: Tue, 3 Nov 2015 09:06:23 +0800
Message-ID: 
 <CAKmQrOaR5cSWLH7MtZ4pN7qKU0rNTHHLThimyd47bKCeG_TQcA@mail.gmail.com>
Subject: Re: [DISCUSS] about cluster management
From: Luke Han <luke.hq@gmail.com>
To: "dev@kylin.incubator.apache.org" <dev@kylin.incubator.apache.org>
Content-Type: multipart/alternative; boundary=001a113edfac9d92620523988079

--001a113edfac9d92620523988079
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Run on YARN requires to run kylin on "one" Hadoop cluster.

Based on today's practices, Kylin's services always sit beside clusters not
inside to have capability to interactive with more than 2 clusters with
just one Kylin deployment.


Best Regards!
---------------------

Luke Han

On Tue, Nov 3, 2015 at 12:12 AM, Henry Saputra <henry.saputra@gmail.com>
wrote:

> Why dont we just leverage Hadoop YARN?
>
> We could use Apache Twill [1] to wrap it as YARN application.
>
> Helix is great but one of the limitations it could not provision new
> node when not already available.
>
> [1] http://twill.incubator.apache.org/
>
> On Sun, Nov 1, 2015 at 7:01 PM, =E5=91=A8=E5=8D=83=E6=98=8A <qhzhou@apach=
e.org> wrote:
> > Hi, all
> >      Kylin currently manages the cluster itself, so far it works fine
> since
> > there are only 2 roles for the cluster, job engine and query engine.
> >      In order to reduce the complexity of deployment and support the
> coming
> > streaming feature, I suggest kylin have a better cluster management.
> >      There are several candidates I think can be leveraged. Apache Heli=
x
> > seem suitable for Kylin in my opinion.
> >      I would love to start a discussion about cluster manager of Kylin,
> how
> > we can do it the better way, and which component we should choose.
>

--001a113edfac9d92620523988079--