hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Umesh Agashe (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HBASE-18703) Inconsistent behavior for preBatchMutate in doMiniBatchMutate and processRowsWithLocks
Date Thu, 05 Oct 2017 18:27:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-18703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16193379#comment-16193379
] 

Umesh Agashe edited comment on HBASE-18703 at 10/5/17 6:26 PM:
---------------------------------------------------------------

[~anoop.hbase], thanks for your comments and review. I liked your suggestion to split changes
into multiple JIRAs.

Regarding giving user an option to specify read or write lock, depending on use case this
can be implemented in future. Currently I am trying retain the functionality with single code
path. Most use cases need exclusive/ write locks on multiple rows. Do you have any use case
for which shared lock on multiple rows is enough?

bq. Yes. For the custom compare logic, prePut() CP can be used and based on the result, we
can allow/bypass the put op.

There are multiple ways user can achieve, reading and modifying multiple rows under exclusive
lock/s. User can specify custom rows to lock by calling one of the Region APIs that take rowsToLock
as an input. When prePut() hook of CPs is called its seeded with WALEdit but this hook is
called prior to locking rows. Then rows are locked and preBatchMutate() CP hook is called
with mutations as an input. Based on this list user CP can update CP mutations or WALEdit
(seeded with prePut() previously) or both.

I think you are right, we need to simplify this code and split changes into multiple JIRAs.
Let's not commit this patch and discuss things over it. Once we like the direction of the
changes, I can work on splitting the changes into multiple JIRAs. Thanks!


was (Author: uagashe):
[~anoop.hbase], thanks for your comments and review. I liked your suggestion to split changes
into multiple JIRAs.

Regarding giving user an option to specify read or write lock, depending on use case this
can be implemented in future. Currently I am trying retain functionality with single code
path. Most use case need exclusive/ write locks on multiple rows. Do you have any use case
for which shared lock on multiple rows is enough?

bq. Yes. For the custom compare logic, prePut() CP can be used and based on the result, we
can allow/bypass the put op.

There are multiple ways user can achieve, reading and modifying multiple rows under exclusive
lock/s. User can specify custom rows to lock by calling one of the Region APIs that take rowsToLock
as an input. When prePut() hook of CPs is called its seeded with WALEdit but this hook is
called prior to locking rows. The rows are locked and preBatchMutate() CP hook is called with
mutations as an input. Based on this list user CP can update CP mutations or WALEdit (seeded
with prePut() previously) or both.

I think you are right, we need to simplify this code and split changes into multiple JIRAs.
Let's not commit this patch and discuss things over it. Once we like the direction of the
changes, I can work on splitting the changes into multiple JIRAs. Thanks!

> Inconsistent behavior for preBatchMutate in doMiniBatchMutate and processRowsWithLocks
> --------------------------------------------------------------------------------------
>
>                 Key: HBASE-18703
>                 URL: https://issues.apache.org/jira/browse/HBASE-18703
>             Project: HBase
>          Issue Type: Sub-task
>          Components: Coprocessors
>            Reporter: Duo Zhang
>            Assignee: Umesh Agashe
>            Priority: Critical
>             Fix For: 2.0.0-alpha-4
>
>         Attachments: hbase-18703.master.001.patch, hbase-18703.master.002.patch, hbase-18703.master.003.patch,
hbase-18703.master.004.patch, hbase-18703.master.005.patch, hbase-18703.master.005.patch,
hbase-18703.master.005.patch, hbase-18703.master.006.patch, hbase-18703.master.007.patch
>
>
> In doMiniBatchMutate, the preBatchMutate is called before building WAL, but in processRowsWithLocks,
we suggest the RowProcessor implementation to build WAL in process  method, which is ahead
of preBatchMutate.
> If a CP modifies the mutations, especially if it removes some cells from the mutations,
then the behavior of processRowsWithLocks is broken. The changes applied to memstore and WAL
will be different. And there is no way to remove entries from a WALEdit through CP. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message