hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anoop Sam John (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-18500) Performance issue: Don't use BufferedMutator for HTable's put method
Date Thu, 10 Aug 2017 13:54:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-18500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16121630#comment-16121630
] 

Anoop Sam John commented on HBASE-18500:
----------------------------------------

Thanks for the nice debug.  Things are clear now.
This Table.put() happens within the RS. We are already in AccessController.  Means already
there was an RPC happened to server (Actual grant call from client side). At RpcServer we
have a ThreadLocal where we keep the CallContext and inside that the current RPC called user
info is set. The table on which put was called is created via the RegionCP env and that uses
a special Connection. The normal RPC channel will be by passed here means there would have
no further contact on to the RpcServer. So the ThreadLocal is never getting reset.  We ran
the new put as a super user (User.runAsLoginUser where the login user is the user who started
RS process) but still as per the RPC context it is the old user. When this patch changed the
Table.put(Put) flow so that there is no more AsyncProcess involvement, the same thread executing
the put. So the Rpc context ThreadLocal is still accessible and we see the user as old Rpc
user.  When AsyncProcess was used, the execute happen via another thread from pool and so
old ThreadLocal variable is not accessible and so it looks as if no Rpc context and we were
relying on the super user who starts the RS process. 
We can solve this particular issue in diff ways. But this investigation raises few Qs.
1. We provide CPs a way to make Table ops. The Table can be obtained from the env and may
be the flow will get RPC bypassed if the calling region on same RS. In that case if the Table.put(Put)
is called we will see the old RPC user in the second region call path also. But if it was
AsyncProcess involved flow, we wont see.  How/whether we should make possible to get the Rpc
context in the second region flow also? To know the user. And we have to solve this incosistency
based on API usage any way.
2. When doing a runAsUser, how/whether we should reset the Rpc context to have the new user.
We should address it via some other jira.  As of here, just do as what u did in patch with
a TODO to fix the issue.

> Performance issue: Don't use BufferedMutator for HTable's put method
> --------------------------------------------------------------------
>
>                 Key: HBASE-18500
>                 URL: https://issues.apache.org/jira/browse/HBASE-18500
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Guanghao Zhang
>            Assignee: Guanghao Zhang
>         Attachments: HBASE-18500-v1.patch, HBASE-18500-v2.patch, HBASE-18500-v3.patch,
HBASE-18500-v4.patch, HBASE-18500-v5.patch, HBASE-18500-v5.patch, HBASE-18500-v5.patch
>
>
> Copied the test result from HBASE-17994.
> Run start-hbase.sh in my local computer and use the default config to test with PE tool.
> {code}
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=100000 --nomapred --autoFlush=True
randomWrite 1
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=100000 --nomapred --autoFlush=True
asyncRandomWrite 1
> {code}
> Mean latency test result.
> || || Test1 || Test2 || Test3 || Test4 || Test5 ||
> | randomWrite | 164.39 | 161.22 | 164.78 | 140.61 | 151.69 |
> | asyncRandomWrite | 122.29 | 125.58 | 122.23 | 113.18 | 123.02 |
> 50th latency test result.
> || || Test1 || Test2 || Test3 || Test4 || Test5 ||
> | randomWrite | 130.00 | 125.00 | 123.00 | 112.00 | 121.00 |
> | asyncRandomWrite | 95.00 | 97.00 | 95.00 | 88.00 | 95.00 |
> 99th latency test result.
> || || Test1 || Test2 || Test3 || Test4 || Test5 ||
> | randomWrite | 600.00 | 600.00 | 650.00 | 404.00 | 425.00 |
> | asyncRandomWrite | 339.00 | 327.00 | 297.00 | 311.00 | 318.00 |
> In our internal 0.98 branch, the PE test result shows the async write has the almost
same latency with the blocking write. But for master branch, the result shows the async write
has better latency than the blocking client.  Take a look about the code, I thought the difference
is the BufferedMutator. For master branch, HTable don't have a write buffer and all write
request will be flushed directly. And user can use BufferedMutator when user want to perform
client-side buffering of writes. For the performance issue (autoFlush=True), I thought we
can use rpc caller directly in HTable's put method. Thanks.
> Review: https://reviews.apache.org/r/61454/



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message