zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aristedes Maniatis <amania...@apache.org>
Subject Best recipe for distributed cron
Date Mon, 06 Mar 2017 12:54:35 GMT
I'm new to Zookeeper; I just landed on this project because our upgrade to Solr Cluster required
it. But I'm intrigued by what we can do outside of Solr itself.

One problem we have is distributed cron jobs. Typically jobs that might run hourly or daily,
but we want these jobs to be able to failover to another node in case of failure. Our apps
are all Java.

I've read the recipe page: https://zookeeper.apache.org/doc/trunk/recipes.html  but am still
a bit confused about the best approach.

If each node had an hourly cron script (either just a process which sleeps or a Quartz driven
thread), which then wakes up to get a lock from Zookeeper, then I'd need also to store a "next
run on" date in Zookeeper so that we don't run too often. So then I'd need a lock plus a date
property in Zookeeper, protected by that lock.

On the other hand, one node could be elected leader through zookeeper, then run jobs as needed.
All jobs would always run on one node until it lost an election.

Am I going about this the right way? Is there a library for Quartz-Zookeeper? Or some other
best practices for this type of use.


Aristedes Maniatis
GPG fingerprint CBFB 84B4 738D 4E87 5E5C  5EFA EF6A 7D2E 3E49 102A

View raw message