hadoop-zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mahadev Konar <maha...@yahoo-inc.com>
Subject Re: Sequence Number Generation With Zookeeper
Date Fri, 06 Aug 2010 23:30:49 GMT
Hi David,
 I think it would be really useful. It would be very helpful for someone
looking for geenrating unique tokens/generations ids ( I can think of plenty
of applications for this).

Please do consider contributing it back to the community!

Thanks
mahadev


On 8/6/10 7:10 AM, "David Rosenstrauch" <darose@darose.net> wrote:

> Perhaps.  I'd have to ask my boss for permission to release the code.
> 
> Is this something that would be interesting/useful to other people?  If
> so, I can ask about it.
> 
> DR
> 
> On 08/05/2010 11:02 PM, Jonathan Holloway wrote:
>> Hi David,
>> 
>> We did discuss potentially doing this as well.  It would be nice to get some
>> recipes for Zookeeper done for this area, if people think it's useful.  Were
>> you thinking of submitting this back as a recipe, if not then I could
>> potentially work on such a recipe instead.
>> 
>> Many thanks,
>> Jon.
>> 
>> 
>>> I just ran into this exact situation, and handled it like so:
>>> 
>>> I wrote a library that uses the option (b) you described above.  Only
>>> instead of requesting a single sequence number, you request a block of them
>>> at a time from Zookeeper, and then locally use them up one by one from the
>>> block you retrieved.  Retrieving by block (e.g., by blocks of 10000 at a
>>> time) eliminates the contention issue.
>>> 
>>> Then, if you're finished assigning ID's from that block, but still have a
>>> bunch of ID's left in the block, the library has another function to "push
>>> back" the unused ID's.  They'll then get pulled again in the next block
>>> retrieval.
>>> 
>>> We don't actually have this code running in production yet, so I can't
>>> vouch for how well it works.  But the design was reviewed and given the
>>> thumbs up by the core developers on the team, and the implementation passes
>>> all my unit tests.
>>> 
>>> HTH.  Feel free to email back with specific questions if you'd like more
>>> details.
>>> 
>>> DR
>> 
> 
> 


Mime
View raw message