hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-10277) refactor AsyncProcess
Date Wed, 22 Jan 2014 00:43:21 GMT

    [ https://issues.apache.org/jira/browse/HBASE-10277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13878076#comment-13878076

Sergey Shelukhin commented on HBASE-10277:

bq. This seems great. Would it be possible then to have a single AsyncProcess per HConnection,
shared between the different htables objects? This would make
Not for "legacy" mode, because then the cross-put behavior will also be cross-HTable.
For individual requests, yeah, that can be done.
Also the sentence appears to be unfinished.

bq. Side question: would it make sense to use the multiget path for a single get, instead
of having two different paths?
Yeah, that is possible, but it is in scope of different JIRA.

bq. The scenario is already there: it's how to manage the errors with the write buffer. I
didn't want to make the interface public (as once it's public you should not change it), but
at the end of the day, the callback is the most obvious solution to the problem. Having it
here sets a base for the discussion. If your patch allows to have a common resource management
per HTable, I'm happy to lose the callbacks as a side effect of the patch, but having both
would be better imho.
Can you elaborate on the error management? Right now the patch preserves the cross-put-errors
mode for HTable, without the callback.

bq. What's deprecated is mainly that the batch interfaces were in HConnection instead of HTable.
The Object[] is ugly, but is still the 'recommended' way
Yeah, for these paths and without the custom pool is where we reuse the same AsyncProcess.

> refactor AsyncProcess
> ---------------------
>                 Key: HBASE-10277
>                 URL: https://issues.apache.org/jira/browse/HBASE-10277
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Sergey Shelukhin
>            Assignee: Sergey Shelukhin
>         Attachments: HBASE-10277.patch
> AsyncProcess currently has two patterns of usage, one from HTable flush w/o callback
and with reuse, and one from HCM/HTable batch call, with callback and w/o reuse. In the former
case (but not the latter), it also does some throttling of actions on initial submit call,
limiting the number of outstanding actions per server.
> The latter case is relatively straightforward. The former appears to be error prone due
to reuse - if, as javadoc claims should be safe, multiple submit calls are performed without
waiting for the async part of the previous call to finish, fields like hasError become ambiguous
and can be used for the wrong call; callback for success/failure is called based on "original
index" of an action in submitted list, but with only one callback supplied to AP in ctor it's
not clear to which submit call the index belongs, if several are outstanding.
> I was going to add support for HBASE-10070 to AP, and found that it might be difficult
to do cleanly.
> It would be nice to normalize AP usage patterns; in particular, separate the "global"
part (load tracking) from per-submit-call part.
> Per-submit part can more conveniently track stuff like initialActions, mapping of indexes
and retry information, that is currently passed around the method calls.
> -I am not sure yet, but maybe sending of the original index to server in "ClientProtos.MultiAction"
can also be avoided.- Cannot be avoided because the API to server doesn't have one-to-one
correspondence between requests and responses in an individual call to multi (retries/rearrangement
have nothing to do with it)

This message was sent by Atlassian JIRA

View raw message