zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vitalii Tymchyshyn <...@tym.im>
Subject Implementing sharding with zookeeper
Date Wed, 04 Feb 2015 20:10:49 GMT
Hello.

I have a task that looks like a fit for Zookeeper and I thought I find a
recipe in Curator, but I dont. So, It looks like I need my own recipe.

The problem is: we have a stream of tasks coming in. All tasks for the same
account must be done sequential. But for different accounts it's OK to do
in parallel, so we want workers to distribute account space between them.

The first step, of course, is to make an account hash to ensure load is
distributed as smooth as possible.

Next I could simply use a lot of locks, but this way I can't ensure
fairness.

Thats why I think I need a Leader who will distribute (or redistribute in
case of node death) the hash space between the live nodes.

What do you think? May be I am missing something and there is existing
recipe for the task?

Best regards, Vitalii Tymchyshyn

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message