hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Leach <jle...@splicemachine.com>
Subject Re: help, try to use HBase's checkAndPut() to implement distribution lock
Date Wed, 10 Aug 2016 14:58:31 GMT
Ming,

One challenge with Locking Mechanisms is that you need to account for node failure after lock
is acquired.  If a region would to be moved, split, etc, the lock needs to survive those operations.
 Most databases put the locks inline with the data they store and user their transaction resolution
mechanism to enforce or release them.

Good luck.

Regards,
John Leach
 
> On Aug 10, 2016, at 9:54 AM, Liu, Ming (Ming) <ming.liu@esgyn.cn> wrote:
> 
> Thanks Ted!
> 
> -----Original Message-----
> From: Ted Yu [mailto:yuzhihong@gmail.com] 
> Sent: Wednesday, August 10, 2016 10:17 PM
> To: user@hbase.apache.org
> Subject: Re: help, try to use HBase's checkAndPut() to implement distribution lock
> 
> Please take a look at EnableTableHandler where you can find example
> - prepare() and process().
> 
> On Tue, Aug 9, 2016 at 5:04 PM, Liu, Ming (Ming) <ming.liu@esgyn.cn> wrote:
> 
>> Thanks Ted for pointing out this. Can this TableLockManager be used 
>> from a client? I am fine to migrate if this API change for each release.
>> I am writing a client application, and need to lock a hbase table, if 
>> this can be used directly, that will be super great!
>> 
>> Thanks,
>> Ming
>> 
>> -----Original Message-----
>> From: Ted Yu [mailto:yuzhihong@gmail.com]
>> Sent: Wednesday, August 10, 2016 1:04 AM
>> To: user@hbase.apache.org
>> Subject: Re: help, try to use HBase's checkAndPut() to implement 
>> distribution lock
>> 
>> Please take a look at TableLockManager and its subclasses.
>> 
>> It is marked @InterfaceAudience.Private Meaning the API may change 
>> across releases.
>> 
>> Cheers
>> 
>> On Tue, Aug 9, 2016 at 9:58 AM, Liu, Ming (Ming) <ming.liu@esgyn.cn>
>> wrote:
>> 
>>> Thanks Ted for the questions.
>>> 
>>> 1. what if the process of owner of the lock dies?
>>> I didn't think of this... This is really an issue. I don't have a 
>>> good answer. One possibility is to have a lease of each lock, owner 
>>> must renew it periodically until release it. The getLock can check 
>>> the last timestamp to see if it is expired. But I have no idea how 
>>> the lock owner can periodic renew the lock lease, spawn a new thread 
>>> to do that? So, this makes me trying to abandon this idea.
>>> 
>>> 2. How can other processes obtain the lock?
>>> The getLock() has input param of table, so any process can give the 
>>> tablename and invoke getLock(), it check the row 0 value in an 
>>> atomic check and put operation. So if the 'table lock' is free, 
>>> anyone should be able to get it I think.
>>> 
>>> Maybe I have to study the Zookeeper's distributed lock recipes?
>>> 
>>> Thanks,
>>> Ming
>>> 
>>> -----Original Message-----
>>> From: Ted Yu [mailto:yuzhihong@gmail.com]
>>> Sent: Wednesday, August 10, 2016 12:26 AM
>>> To: user@hbase.apache.org
>>> Subject: Re: help, try to use HBase's checkAndPut() to implement 
>>> distribution lock
>>> 
>>> What if the process of owner of the lock dies ?
>>> How can other processes obtain the lock ?
>>> 
>>> Cheers
>>> 
>>> On Tue, Aug 9, 2016 at 8:19 AM, Liu, Ming (Ming) <ming.liu@esgyn.cn>
>>> wrote:
>>> 
>>>> Hi, all,
>>>> 
>>>> I want to implement a simple 'table lock' in HBase. My current 
>>>> idea is for each table, I choose a special rowkey which NEVER be 
>>>> used in real data, and then use this row as a 'table level lock'.
>>>> 
>>>> The getLock() will be:
>>>> 
>>>>  getLock(table, cf, cl)
>>>>  {
>>>>     rowkey = 0 ; //which never used in real data
>>>> 
>>>>    //generate a UUID, thread ID + an atomic increamental sequence 
>>>> number for example. Call it myid
>>>>     genMyid(myid);
>>>> 
>>>>     while(TRUE)
>>>>     {
>>>>         ret = table.checkAndPut ( rowkey, cf , cl , '0' , myid);
>>>>         if (ret == true ) get the lock
>>>>             return myid;
>>>>         else
>>>>             sleep and continue; //retry, maybe can retry for 
>>>> timeout here, or wait forever here
>>>>      }
>>>>   }
>>>> 
>>>> The releaseLock() may be:
>>>> 
>>>>   releaseLock(table, cf, cl, myid)
>>>>   {
>>>>       Rowkey = 0;
>>>>        ret = checkAndPut ( rowkey , cf , cl , myid , '0' );  //If 
>>>> I am holding the lock
>>>>        if (ret == TRUE) return TRUE;
>>>>        else  return FALSE;
>>>> }
>>>> 
>>>> So one caller get lock, and others must wait until the caller 
>>>> release it, and only the lock holder can release the lock. So if 
>>>> one getLock(), it can then modify the table, others during this 
>>>> period must
>>> wait.
>>>> 
>>>> I am very new in lock implementation, so I fear there are basic 
>>>> problems in this 'design'.
>>>> So please help me to review if there are any big issues about this
>> idea?
>>>> Any help will be very appreciated.
>>>> 
>>>> Thanks a lot,
>>>> Ming
>>>> 
>>> 
>> 


Mime
View raw message