zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Martin Waite <waite....@googlemail.com>
Subject Re: is there a good pattern for leases ?
Date Thu, 25 Feb 2010 10:50:57 GMT
Hi,

Another possible approach:

   1. client generates hash code to be locked.  use the first N hex digits
   (eg. first 2 digits) as a filename "FN".
   2. attempt to create (ephemeral) node "FN.lock".  loop until this
   succeeds, or abort when time budget exhausted.
   3. read node "FN" containing list of hash-codes with associated expiry
   times.  if full hash-code does not exist, or has expired, then update list
   with full hash-code and expiry time.  Otherwise, you didn't get the lock.
   4. release "FN.lock"

For 10K locked cards, this would give 256 nodes containing approx. 400
hash-codes.  Obviously this can be scaled by increasing the number of digits
in "FN".

But to do this, would I need to call sync between steps 2 and 3 to ensure
the node "FN" was up-to-date - assuming I do not know if I am connected to a
primary ZK instance ?   Would 10K sync calls within a 2 minute period be
excessive ?

regards,
Martin


On 25 February 2010 08:07, Martin Waite <waite.134@googlemail.com> wrote:

> Hi
>
> Usually, this would hold about 2k items, pushing to 10k peaks.
>
> My current understanding is that I cannot lock a node while I consider its
> contents, and so only the garbage remover would be allowed to remove locks
> (currently lock-clients can claim expired locks).
>
> The clients would simply do this:
>
>    1. try to create a node
>    2. if success, you have the lock, otherwise you don't
>
> The garbage remover would do this:
>
>    1. inspect node.
>    2. if expired, delete it.
>
> The length of time that locks are held then becomes a function of the
> length of time between garbage-removing sweeps. Perhaps that is OK.
>
> I cannot see how clients could safely inspect a node and overwrite the
> expiry time if it had expired.  That would involve multiple steps, and so
> not be atomic.
>
> regards,
> Martin
>
>
>
> On 24 February 2010 22:35, Ted Dunning <ted.dunning@gmail.com> wrote:
>
>> You can simply implement the current system if you like by keeping a file
>> per card in ZK that contains your lock expiration time.  The garbage
>> collector would work the same way.  In order to make the getchildren
>> operation in the garbage collector work well, I would recommend a
>> hierarchical naming scheme for card lock files.
>>
>> My question would be how many elements you expect to be in that card lock
>> table.  If it is less than 100K, ZK should work pretty well.
>>
>> If you need more than that, you might consider putting locks for many
>> cards
>> in a single file.
>>
>> On Wed, Feb 24, 2010 at 11:10 AM, Martin Waite <waite.134@googlemail.com
>> >wrote:
>>
>> >   2. card-locking - to reduce the risk of payments being taken twice in
>> >   quick succession from the same card, a timed lock is placed on a hash
>> of
>> > the
>> >   card number for a number of seconds (0, 30, 60, 120, as required).  No
>> > other
>> >   payment can be taken on that card while the lock is in place.
>> >
>> > Our current way of implementing (2) is to insert into a table a row
>> > containing the card-hash and the expiry time of the lock.  Another
>> process
>> > can overwrite the lock if the expiry has been exceeded.  A periodic
>> garbage
>> > remover process deletes all expired locks to keep the size of the lock
>> > table
>> > small.
>> >
>> > The trouble with managing these locks in a database is that the tables
>> are
>> > getting "hot" and becoming one of the main sources of contention.  Also,
>> > SQL
>> > is not necessarily fast for doing the required updates.
>> >
>>
>>
>>
>> --
>> Ted Dunning, CTO
>> DeepDyve
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message