zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark <static.void....@gmail.com>
Subject Design question
Date Tue, 08 Nov 2011 17:18:56 GMT
I have a general design question regarding ZooKeeper.

Our use case: We currently have 3 restful recommendation servers that 
simply wrap a Mahout GenericBooleanPrefItemBasedRecommender. We started 
off using a JDBCDataModel but for performance reasons we had to switch 
to a FileDataModel so everything would be kept in memory. Although now 
that our recommendations service is blazing fast the start up/reloading 
time for each of these services are in the minutes. If we try to update 
all services at once then all recommendation requests come to a halt. As 
a result of this whenever we push a new model we have to do it in 
stages... ie disable server1, update, wait, renable, disable server2.... 
We've "automated" this using cron by simply updating one server waiting 
10 mins then updating the next and so on. We are trying to figure out if 
this coordination would be better managed via ZooKeeper.

I've read a bit into ZooKeeper and it seems like it would be easy to set 
a watch on a node to trigger when a model has changed thus triggering a 
refresh of our recommender. Where I get lost is how would I coordinate 
this so only one server at a time goes down? When it comes back up then 
the next server should be updated. Can someone please explain how this 
could be accomplished? Thanks

View raw message