hadoop-zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Rosenstrauch <dar...@darose.net>
Subject Re: Sequence Number Generation With Zookeeper
Date Tue, 10 Aug 2010 16:59:37 GMT
Good news!  I got approval to release this code!  (Man, I love working 
for a startup!!!)  :-)

So anyone know:  what's the next step?  Do I need to obtain commit 
privileges?  Or do I deliver the code to someone who has commit privs 
who shepherds this for me?

Also, what (if anything) do I need to tweak in the code to make it 
release-ready.  (e.g., Change package names?  Slap an Apache license on 
it?  etc.)



On 08/06/2010 10:39 PM, David Rosenstrauch wrote:
> I'll run it by my boss next week.
> DR
> On 08/06/2010 07:30 PM, Mahadev Konar wrote:
>> Hi David,
>> I think it would be really useful. It would be very helpful for someone
>> looking for geenrating unique tokens/generations ids ( I can think of
>> plenty
>> of applications for this).
>> Please do consider contributing it back to the community!
>> Thanks
>> mahadev
>> On 8/6/10 7:10 AM, "David Rosenstrauch"<darose@darose.net> wrote:
>>> Perhaps. I'd have to ask my boss for permission to release the code.
>>> Is this something that would be interesting/useful to other people? If
>>> so, I can ask about it.
>>> DR
>>> On 08/05/2010 11:02 PM, Jonathan Holloway wrote:
>>>> Hi David,
>>>> We did discuss potentially doing this as well. It would be nice to
>>>> get some
>>>> recipes for Zookeeper done for this area, if people think it's
>>>> useful. Were
>>>> you thinking of submitting this back as a recipe, if not then I could
>>>> potentially work on such a recipe instead.
>>>> Many thanks,
>>>> Jon.
>>>>> I just ran into this exact situation, and handled it like so:
>>>>> I wrote a library that uses the option (b) you described above. Only
>>>>> instead of requesting a single sequence number, you request a block
>>>>> of them
>>>>> at a time from Zookeeper, and then locally use them up one by one
>>>>> from the
>>>>> block you retrieved. Retrieving by block (e.g., by blocks of 10000
>>>>> at a
>>>>> time) eliminates the contention issue.
>>>>> Then, if you're finished assigning ID's from that block, but still
>>>>> have a
>>>>> bunch of ID's left in the block, the library has another function
>>>>> to "push
>>>>> back" the unused ID's. They'll then get pulled again in the next block
>>>>> retrieval.
>>>>> We don't actually have this code running in production yet, so I can't
>>>>> vouch for how well it works. But the design was reviewed and given the
>>>>> thumbs up by the core developers on the team, and the
>>>>> implementation passes
>>>>> all my unit tests.
>>>>> HTH. Feel free to email back with specific questions if you'd like
>>>>> more
>>>>> details.
>>>>> DR

View raw message