hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-3787) Increment is non-idempotent but client retries RPC
Date Tue, 21 May 2013 20:43:20 GMT

    [ https://issues.apache.org/jira/browse/HBASE-3787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13663383#comment-13663383

Sergey Shelukhin commented on HBASE-3787:

Sorry, got distracted and didn't post continuation comment.
Another concern is that if you receive smth like a network timeout you don't know whether
it's safe to retry even to the same server, the server could have processed the request after
all. With memory-only nonces I guess we could retry to the same server and significantly reduce
the potential of duplicates. Otherwise on some errors we cannot retry at all.
But it will be much simpler, no WAL involved. For proper implementation it seems we'd need
separate WAL-like thingie for nonces and all this stuff.
As I said it's hard for me to judge because I don't know how important or prevalent the problem
is. It doesn't seem like it would be very frequent. Maybe then we could indeed just say "(almost)
no retries for increment/append"

By the way, I just had the random idea that we could use instead. Please shoot it down or
expand it :)
Keep current increment as is. Add idempotent(atomic? checked?)Increment for those who want
First, send a request to the server that creates a nonce on server-side, stores it (in-memory
only) and returns it.
Then, send increment with the nonce; server-side, that removes the nonce and does the increment.
If increment arrives and nonce is not there the server responds that the operation is done
(or expired).
If getting nonce fails non-trivially, it can be retried with no harm, old nonce will be expired
by the server (see below).
If increment fails with trivial error (region moved/not there etc.), nonce is removed from
server and retries are unrestricted; client has to go back to requesting nonce.
If increment fails non-trivially, we cannot go to a different server unless we get some definite
error from current one later, after which retries are unrestricted again; or unless we preserve
the nonces in-flight during recovery (which is less data than nonces of successful operations
and can be added later). However we can still retry to the same server safely.
Requested nonces are expired by time.
Because server does nonce issuing sequential nonces per region the old ones can be stored
for longer, they are easy to roll.
> Increment is non-idempotent but client retries RPC
> --------------------------------------------------
>                 Key: HBASE-3787
>                 URL: https://issues.apache.org/jira/browse/HBASE-3787
>             Project: HBase
>          Issue Type: Bug
>          Components: Client
>    Affects Versions: 0.94.4, 0.95.2
>            Reporter: dhruba borthakur
>            Assignee: Sergey Shelukhin
>            Priority: Critical
>             Fix For: 0.95.1
>         Attachments: HBASE-3787-partial.patch, HBASE-3787-v0.patch, HBASE-3787-v1.patch,
HBASE-3787-v2.patch, HBASE-3787-v3.patch, HBASE-3787-v4.patch, HBASE-3787-v5.patch, HBASE-3787-v5.patch
> The HTable.increment() operation is non-idempotent. The client retries the increment
RPC a few times (as specified by configuration) before throwing an error to the application.
This makes it possible that the same increment call be applied twice at the server.
> For increment operations, is it better to use HConnectionManager.getRegionServerWithoutRetries()?
Another  option would be to enhance the IPC module to make the RPC server correctly identify
if the RPC is a retry attempt and handle accordingly.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message