zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jordan Zimmerman <jor...@jordanzimmerman.com>
Subject Re: Implementing sharding with zookeeper
Date Wed, 04 Feb 2015 21:02:50 GMT
I wrote something that might work for you. Have a look at Nirmata Workflow:  http://nirmataoss.github.io/workflow

====================
Jordan Zimmerman

> On Feb 4, 2015, at 12:10 PM, Vitalii Tymchyshyn <vit@tym.im> wrote:
> 
> Hello.
> 
> I have a task that looks like a fit for Zookeeper and I thought I find a
> recipe in Curator, but I dont. So, It looks like I need my own recipe.
> 
> The problem is: we have a stream of tasks coming in. All tasks for the same
> account must be done sequential. But for different accounts it's OK to do
> in parallel, so we want workers to distribute account space between them.
> 
> The first step, of course, is to make an account hash to ensure load is
> distributed as smooth as possible.
> 
> Next I could simply use a lot of locks, but this way I can't ensure
> fairness.
> 
> Thats why I think I need a Leader who will distribute (or redistribute in
> case of node death) the hash space between the live nodes.
> 
> What do you think? May be I am missing something and there is existing
> recipe for the task?
> 
> Best regards, Vitalii Tymchyshyn

Mime
  • Unnamed multipart/alternative (inline, 7-Bit, 0 bytes)
View raw message