hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-11355) a couple of callQueue related improvements
Date Tue, 01 Jul 2014 11:20:24 GMT

    [ https://issues.apache.org/jira/browse/HBASE-11355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14048771#comment-14048771
] 

Hudson commented on HBASE-11355:
--------------------------------

SUCCESS: Integrated in HBase-TRUNK #5254 (See [https://builds.apache.org/job/HBase-TRUNK/5254/])
HBASE-11355 a couple of callQueue related improvements (matteo.bertozzi: rev 0e8e41d0ef5985aa3d424621eb8fcb1ca68838ab)
* hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcExecutor.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/MultipleQueueRpcExecutor.java
* hbase-common/src/main/java/org/apache/hadoop/hbase/util/ReflectionUtils.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SingleQueueRpcExecutor.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RWQueueRpcExecutor.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/SimpleRpcScheduler.java
* hbase-common/src/main/resources/hbase-default.xml


> a couple of callQueue related improvements
> ------------------------------------------
>
>                 Key: HBASE-11355
>                 URL: https://issues.apache.org/jira/browse/HBASE-11355
>             Project: HBase
>          Issue Type: Improvement
>          Components: IPC/RPC, Performance
>    Affects Versions: 0.99.0, 0.94.20
>            Reporter: Liang Xie
>            Assignee: Matteo Bertozzi
>             Fix For: 0.99.0, 0.98.4
>
>         Attachments: HBASE-11355-v0.patch, HBASE-11355-v1.patch, HBASE-11355-v2.patch,
gets.png
>
>
> In one of my in-memory read only testing(100% get requests), one of the top scalibility
bottleneck came from the single callQueue. A tentative sharing this callQueue according to
the rpc handler number showed a big throughput improvement(the original get() qps is around
60k, after this one and other hotspot tunning, i got 220k get() qps in the same single region
server) in a YCSB read only scenario.
> Another stuff we can do is seperating the queue into read call queue and write call queue,
we had done it in our internal branch, it would helpful in some outages, to avoid all read
or all write requests ran out of all handler threads.
> One more stuff is changing the current blocking behevior once the callQueue is full,
considering the full callQueue almost means the backend processing is slow somehow, so a fail-fast
here should be more reasonable if we using HBase as a low latency processing system. see "callQueue.put(call)"



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message