hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-16224) Reduce the number of RPCs for the large PUTs
Date Tue, 30 Aug 2016 12:07:20 GMT

    [ https://issues.apache.org/jira/browse/HBASE-16224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15448852#comment-15448852
] 

Hudson commented on HBASE-16224:
--------------------------------

FAILURE: Integrated in Jenkins build HBase-1.4 #380 (See [https://builds.apache.org/job/HBase-1.4/380/])
HBASE-16224 Reduce the number of RPCs for the large PUTs (ChiaPing Tsai) (chenheng: rev ce9b6c0abfc2d8bb2a9d22a4ed4c5508d0567987)
* (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/MultiRowMutationEndpoint.java
* (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/BufferedMutatorImpl.java
* (add) hbase-client/src/main/java/org/apache/hadoop/hbase/client/RowAccess.java
* (edit) hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestAsyncProcess.java
* (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncProcess.java


> Reduce the number of RPCs for the large PUTs
> --------------------------------------------
>
>                 Key: HBASE-16224
>                 URL: https://issues.apache.org/jira/browse/HBASE-16224
>             Project: HBase
>          Issue Type: Improvement
>          Components: Client
>    Affects Versions: 2.0.0
>            Reporter: ChiaPing Tsai
>            Assignee: ChiaPing Tsai
>            Priority: Minor
>             Fix For: 2.0.0, 1.4.0
>
>         Attachments: HBASE-16224-v1.patch, HBASE-16224-v2.patch, HBASE-16224-v3.patch,
HBASE-16224-v4.patch, HBASE-16224-v5.patch, HBASE-16224-v6.patch, HBASE-16224-v7.patch, HBASE-16224-v8.patch,
HBASE-16224-v9.patch, HBASE-16224.branch-1.v10.patch, HBASE-16224.branch-1.v9.patch, HBASE-16224.v10.patch,
IntegrationTestBigLinkedList with chaos monkey (serverKilling) (with backup master).txt, IntegrationTestBigLinkedList
with chaos monkey (serverKilling).txt, IntegrationTestBigLinkedList with chaos monkey (slowDeterministic).txt,
IntegrationTestBigLinkedList with chaos monkey (unbalance).txt, experiment-v9.patch.xlsx,
experiment.xlsx
>
>
> This patch is proposed to reduce the number of RPC for the large PUTs 
> The number and data size of write thread(SingleServerRequestRunnable) is a result of
three main factors:
> 1) The flush size taken by BufferedMutatorImpl#backgroundFlushCommits
> 2) The limit of task number
> 3) ClientBackoffPolicy
> A lot of requests created with less MUTATIONs is a result of two reason: 
> 1) many regions of target table are in different server.
> 2) flush size in step one is summed by “all” server rather than “individual”
server
> This patch removes the limit of flush size in step one and add maximum size to submit
for each server in the AsyncProcess



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message