livy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Meisam Fathi <meisam.fa...@gmail.com>
Subject Re: Livy Installation
Date Tue, 13 Feb 2018 22:16:07 GMT
On Mon, Feb 12, 2018 at 2:01 AM Vinod Kancharana <vinod.kancharana@gmail.com>
wrote:

> Is there a recommended cluster server (edge node or master node or cluster
> node) for Livy Installation? Right now, I installed Livy on the master node
> running Yarn Resource Manager.
>
> I am not the best person to answer this question, but here is my two
cents. Livy is a lightweight process. Even with thousands of sessions
running on it, it needs less than 1GB memory, and not much CPU. The only
time Livy becomes a bottleneck is, when many sessions/jobs are submitted to
it in a short burst (because each new session/job launches a new JVM to
submit the job to YARN).


> Also, can we enable Load Balancing with two Livy instances running on the
> same cluster?
>
> No. You cannot because the two Livy instances won't coordinate. For
example, if a session is created on one Livy instance, it won't be
available on the second Livy instance.

PS: I have configured Livy to use ZooKeeper for state-store.
>
> Thanks !!
>

Mime
View raw message