hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ChiaPing Tsai (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-16224) Reduce the number of RPCs for the large PUTs
Date Sat, 27 Aug 2016 20:30:20 GMT

    [ https://issues.apache.org/jira/browse/HBASE-16224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15442184#comment-15442184

ChiaPing Tsai commented on HBASE-16224:

hi [~chenheng]

The test is as follows.

||region count, put count, kv count||elapsed time for new count checker||elapsed time for
previous count checker||
|6, 2000, 2000|6.8s|27.1s|
|6, 2000, 1000|3.6s|9.9s|
|6, 1000, 2000|3.6s|9.8s|
|6, 1000, 1000|2.2s|3.3s|


> Reduce the number of RPCs for the large PUTs
> --------------------------------------------
>                 Key: HBASE-16224
>                 URL: https://issues.apache.org/jira/browse/HBASE-16224
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: ChiaPing Tsai
>            Assignee: ChiaPing Tsai
>            Priority: Minor
>         Attachments: HBASE-16224-v1.patch, HBASE-16224-v2.patch, HBASE-16224-v3.patch,
HBASE-16224-v4.patch, HBASE-16224-v5.patch, HBASE-16224-v6.patch, HBASE-16224-v7.patch, HBASE-16224-v8.patch,
HBASE-16224-v9.patch, HBASE-16224.branch-1.v9.patch, IntegrationTestBigLinkedList with chaos
monkey (serverKilling) (with backup master).txt, IntegrationTestBigLinkedList with chaos monkey
(serverKilling).txt, IntegrationTestBigLinkedList with chaos monkey (slowDeterministic).txt,
IntegrationTestBigLinkedList with chaos monkey (unbalance).txt, experiment-v9.patch.xlsx,
> This patch is proposed to reduce the number of RPC for the large PUTs 
> The number and data size of write thread(SingleServerRequestRunnable) is a result of
three main factors:
> 1) The flush size taken by BufferedMutatorImpl#backgroundFlushCommits
> 2) The limit of task number
> 3) ClientBackoffPolicy
> A lot of requests created with less MUTATIONs is a result of two reason: 
> 1) many regions of target table are in different server.
> 2) flush size in step one is summed by “all” server rather than “individual”
> This patch removes the limit of flush size in step one and add maximum size to submit
for each server in the AsyncProcess

This message was sent by Atlassian JIRA

View raw message