Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
Received-SPF: pass (athena.apache.org: domain of svarma.ng@gmail.com
 designates 209.85.214.41 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAB5sDNLRbJ_-RLBwuV4_QpOA-+u76F6T=PWn8R+wnzh_7No=QA@mail.gmail.com>
References: 
 <CAN89x=x_wD9nB0bXy+VmpF-sPv4V7YcswcvpVz0fjHHQG7OUxA@mail.gmail.com>
	<CAB5sDNLRbJ_-RLBwuV4_QpOA-+u76F6T=PWn8R+wnzh_7No=QA@mail.gmail.com>
Date: Sat, 3 Dec 2011 16:54:17 -0800
Message-ID: 
 <CAN89x=wQrDYksViHJxCwuQhPYpFxT8HNzVZDTHHvtC2qNDmX7Q@mail.gmail.com>
Subject: Re: Question on Coprocessors and Atomicity
From: Suraj Varma <svarma.ng@gmail.com>
To: dev@hbase.apache.org
Content-Type: text/plain; charset=ISO-8859-1

Just so my question is clear ... everything I'm suggesting is in the
context of a single row (not cross row / table). - so, yes, I'm
guessing obtaining a RowLock on the region side during preCheckAndPut
/ postCheckAndPut would certainly work. Which was why I was asking
whether the pre/postCheckAndPut obtains the row lock or whether the
row lock is only obtained within checkAndPut.

Let's say the coprocessor takes a rowlock in preCheckAndPut ... will
that even work? i.e. can the same rowlock be inherited by the
checkAndPut api within that thread's context? Or will preCheckAndPut
have to release the lock so that checkAndPut can take it (which won't
work for my case, as it has to be atomic between the preCheck and
Put.)

Thanks for pointing me to the Constraints functionality - I'll take a
look at whether it could potentially work.
--Suraj

On Sat, Dec 3, 2011 at 10:25 AM, Jesse Yates <jesse.k.yates@gmail.com> wrote:
> I think the feature you are looking for is a Constraint. Currently they are
> being added to 0.94 in
> HBASE-4605<https://issues.apache.org/jira/browse/HBASE-4605>;
> they are almost ready to be rolled in, and backporting to 0.92 is
> definitely doable.
>
> However, Constraints aren't going to be quite flexible enough to
> efficiently support what you are describing. For instance, with a
> constraint, you are ideally just checking the put value against some simple
> constraint (never over 10 or always an integer), but looking at the current
> state of the table before allowing the put would currently require creating
> a full blown connection to the local table through another HTable.
>
> In the short term, you could write a simple coprocessor to do this checking
> and then move over to constraints (which are a simpler, more flexible, way
> of doing this) when the necessary features have been added.
>
> It is worth discussing if it makes sense to have access to the local region
> through a constraint, though that breaks the idea a little bit, it would
> certainly be useful and not overly wasteful in terms of runtime.
>
> Supposing the feature would be added to talk to the local table, and since
> the puts are going to be serialized on the regionserver (at least to that
> single row you are trying to update), you will never get a situation where
> the value added is over the threshold. If you were really worried about the
> atomicity of the operation, then when doing the put, first get the RowLock,
> then do the put and release the RowLock. However, that latter method is
> going to be really slow, so should only be used as a stop gap if the
> constraint doesn't work as expected, until a patch is made for constraints.
>
> Feel free to open up a ticket and link it to 4605 for adding the local
> table access functionality, and we can discuss the de/merits of adding the
> access.
>
> -Jesse
>
> On Sat, Dec 3, 2011 at 6:24 AM, Suraj Varma <svarma.ng@gmail.com> wrote:
>
>> I'm looking at the preCheckAndPut / postCheckAndPut api with
>> coprocessors and I'm wondering ... are these pre/post checks done
>> _after_ taking the row lock or is the row lock only done within the
>> checkAndPut api.
>>
>> I'm interested in seeing if we can implement something like:
>> (in pseudo sql)
>> update table-name
>> set column-name = new-value
>> where (column-value - new-value) > threshold-value
>>
>> Basically ... I want to enhance the checkAndPut to not just compare
>> "values" ... but apply an arbitrary function on the value _atomically_
>> in the Put call. Multiple threads would be firing these mutations and
>> I'd like the threshold-value above to never be breached under any
>> circumstance.
>>
>> Is there a solution that can be implemented either via checkAndPut or
>> using coprocessors preCheckAndPut? If not, would this be a useful
>> feature to build in HBase?
>>
>> Thanks,
>> --Suraj
>>
>
>
>
> --
> -------------------
> Jesse Yates
> 240-888-2200
> @jesse_yates