zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Martin Grotzke <martin.grot...@googlemail.com>
Subject Re: How to make any service highly available using Zookeper
Date Thu, 20 Aug 2015 22:08:20 GMT
Hi Vikram,

we built s.th. for running/scheduling jobs in replicated (HA) worker nodes,
and there we're using Cassandra (lightweight transactions) for job locking:
https://github.com/Galeria-Kaufhof/ha-jobs.

I don't know the spark job scheduler, but if it's possible to embed this
and control/start/monitor it via some java/scala api, you could build a
simple wrapper app that's using ha-jobs to decide on which node the spark
job scheduler should be run.

Cheers,
Martin
Am 18.08.2015 20:54 schrieb "Vikram Kone" <vikramkone@gmail.com>:

> Hi,
> I'm a newbie to Zookeeper, so pardon any naive question I ask here.
> I have a cassandra cluster running on linux VMs and have a spark job
> scheduler service running on one of the nodes. Since cassandra has a
> peer-peer architecture there is no concept of leader.
> I want to provide high availability for this job scheduler service using
> Zookeeper. I can't make any code changes to the job scheduler service since
> it's a 3rd party app.
> I'm thinking of copying the application folder on all the servers in the
> cluster and use zookeeper to start an instance of the service on the
> leader/master node by executing /opt/job-scheduler/bin/start.sh on leader
> election.
> Is this something easy to do with zookeeper?
>
> Please point to any documentation or tutorial on how to run a bash script
> on the leader node in zoo keeper's ensemble after a node is elected as
> leader by the quorum.
>
> Thanks
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message