hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HBASE-3787) Increment is non-idempotent but client retries RPC
Date Thu, 31 Jan 2013 17:53:13 GMT

    [ https://issues.apache.org/jira/browse/HBASE-3787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13567864#comment-13567864
] 

Andrew Purtell edited comment on HBASE-3787 at 1/31/13 5:52 PM:
----------------------------------------------------------------

I think the above comments all taken together are a reasonable thing to try:

- Introduce a nonce (generated internally by the client) on non-idempotent operations to convert
them into idempotent ones.

- nonce = hash(client address, table, row, timestamp)

- HBaseClient should generate a new nonce whenever a new op is sent to new server. Reuse the
nonce for any retry.

- Server tracks nonces by (client address, nonce, timestamp). Expire entries after some grace
period. Restart the expiration timer whenever the nonce is checked as part of op processing.
Lazily clean up expired entries either as part of add/remove or via a chore.

- Add the entry when op processing starts, remove it when finished or failed, refuse to process
an op twice by sending back a DoNotRetryException. Perhaps we introduce a new exception type
like OperationInProgressException which inherits from DoNotRetryException so the client understands
the retry operation was failed because the previous attempt is still pending server side.

- We should append the nonce to the WALEdit, and recover them along with the entry data. 

                
      was (Author: apurtell):
    I think the above comments all taken together are a reasonable thing to try:

- Introduce a nonce (generated internally by the client) on non-idempotent operations to convert
them into idempotent ones.

- nonce = hash(client address, table, row, timestamp)

- HBaseClient should generate a new nonce whenever a new op is sent to new server. Reuse the
nonce for any retry.

- Server tracks nonces by (client address, nonce, timestamp)

- Add the entry when op processing starts, remove it when finished or failed, refuse to process
an op twice by sending back a DoNotRetryException. Perhaps we introduce a new exception type
like OperationInProgressException which inherits from DoNotRetryException so the client understands
the retry operation was failed because the previous attempt is still pending server side.

- We should append the nonce to the WALEdit, and recover them along with the entry data. 


                  
> Increment is non-idempotent but client retries RPC
> --------------------------------------------------
>
>                 Key: HBASE-3787
>                 URL: https://issues.apache.org/jira/browse/HBASE-3787
>             Project: HBase
>          Issue Type: Bug
>          Components: Client
>    Affects Versions: 0.96.0, 0.94.4
>            Reporter: dhruba borthakur
>            Priority: Critical
>             Fix For: 0.96.0
>
>
> The HTable.increment() operation is non-idempotent. The client retries the increment
RPC a few times (as specified by configuration) before throwing an error to the application.
This makes it possible that the same increment call be applied twice at the server.
> For increment operations, is it better to use HConnectionManager.getRegionServerWithoutRetries()?
Another  option would be to enhance the IPC module to make the RPC server correctly identify
if the RPC is a retry attempt and handle accordingly.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message