hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-3787) Increment is non-idempotent but client retries RPC
Date Wed, 08 May 2013 19:21:16 GMT

    [ https://issues.apache.org/jira/browse/HBASE-3787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13652248#comment-13652248

stack commented on HBASE-3787:

bq. There are some design questions on r. Perhaps we should flesh out the design before I
make any major changes.

Sounds good.  Suggest doing it up in a one page doc rather than inline in issue in a comment
field because will be hard to fine otherwise.

bq. ...I have added that for sequential nonces but I wonder if it's worth it for simple nonces.

What is the 'that' in above ('tricks or epic locks'?)

bq. Client ID, for now, will be produced from IP, process id, and thread id. It will be hashed
to 8 bytes and written into nonceGroup.

If opaque, just uuid it?  You'd still have to make a long of it.

Aside: Reading, tripped over this comment on securerandom "....initialize SecureRandom (this
may be a lengthy operation)" -- the lengthy operation part.

There is also http://docs.oracle.com/javase/6/docs/api/java/rmi/server/UID.html which would
be more lightweight than uuid.  Still > 64bits though.

bq. With 5 minutes we'd have something like ~400Mb of RAM for hash table, which is not totally
horrible (especially for 10k QPS ).


bq. However that relies on synchronized clocks....

Yeah.  How bad is that though?  Currently if a regionserver clock is out of sync w/ master
we'll reject it (IIRC).  We'd do something similar for a client that is trying to do a nonce-operation?
 If its mutation is outside of our nonce-keeping window (because we dropped our server-side
record, we can't be sure it not a retry coming in outside of our bounds).  Would have to >
long GC though (smile).

Is the attempt at sequential ids attached here?

Good stuff Sergey.

> Increment is non-idempotent but client retries RPC
> --------------------------------------------------
>                 Key: HBASE-3787
>                 URL: https://issues.apache.org/jira/browse/HBASE-3787
>             Project: HBase
>          Issue Type: Bug
>          Components: Client
>    Affects Versions: 0.94.4, 0.95.2
>            Reporter: dhruba borthakur
>            Assignee: Sergey Shelukhin
>            Priority: Critical
>             Fix For: 0.95.1
>         Attachments: HBASE-3787-partial.patch, HBASE-3787-v0.patch, HBASE-3787-v1.patch,
HBASE-3787-v2.patch, HBASE-3787-v3.patch
> The HTable.increment() operation is non-idempotent. The client retries the increment
RPC a few times (as specified by configuration) before throwing an error to the application.
This makes it possible that the same increment call be applied twice at the server.
> For increment operations, is it better to use HConnectionManager.getRegionServerWithoutRetries()?
Another  option would be to enhance the IPC module to make the RPC server correctly identify
if the RPC is a retry attempt and handle accordingly.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message