hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-18500) Performance issue: Don't use BufferedMutator for HTable's put method
Date Fri, 11 Aug 2017 13:02:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-18500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16123305#comment-16123305
] 

Hudson commented on HBASE-18500:
--------------------------------

FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #3512 (See [https://builds.apache.org/job/HBase-Trunk_matrix/3512/])
HBASE-18500 Performance issue: Don't use BufferedMutator for HTable's (zghao: rev cabdbf181a0b44bd1f9c32aa67b8a2ee3b863758)
* (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/Table.java
* (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java
* (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/BufferedMutatorImpl.java
* (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java
* (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationSmallTests.java
* (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMultiParallel.java
* (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/client/HTableWrapper.java
* (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationBase.java
* (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/RegionAsTable.java
* (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestMultiSlaveReplication.java
* (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/TableBuilderBase.java
* (edit) hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestAsyncProcess.java
* (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java
* (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestClientPushback.java
* (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestNamespaceCommands.java
* (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/TableBuilder.java
* (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java
* (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java
* (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestMasterReplication.java
* (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationSyncUpTool.java
* (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestHTableWrapper.java
* (edit) hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/client/RemoteHTable.java
* (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java
* (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessControlLists.java
* (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestServerBusyException.java


> Performance issue: Don't use BufferedMutator for HTable's put method
> --------------------------------------------------------------------
>
>                 Key: HBASE-18500
>                 URL: https://issues.apache.org/jira/browse/HBASE-18500
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Guanghao Zhang
>            Assignee: Guanghao Zhang
>             Fix For: 2.0.0, 3.0.0
>
>         Attachments: HBASE-18500-v1.patch, HBASE-18500-v2.patch, HBASE-18500-v3.patch,
HBASE-18500-v4.patch, HBASE-18500-v5.patch, HBASE-18500-v5.patch, HBASE-18500-v5.patch, HBASE-18500-v6.patch
>
>
> Copied the test result from HBASE-17994.
> Run start-hbase.sh in my local computer and use the default config to test with PE tool.
> {code}
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=100000 --nomapred --autoFlush=True
randomWrite 1
> ./bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=100000 --nomapred --autoFlush=True
asyncRandomWrite 1
> {code}
> Mean latency test result.
> || || Test1 || Test2 || Test3 || Test4 || Test5 ||
> | randomWrite | 164.39 | 161.22 | 164.78 | 140.61 | 151.69 |
> | asyncRandomWrite | 122.29 | 125.58 | 122.23 | 113.18 | 123.02 |
> 50th latency test result.
> || || Test1 || Test2 || Test3 || Test4 || Test5 ||
> | randomWrite | 130.00 | 125.00 | 123.00 | 112.00 | 121.00 |
> | asyncRandomWrite | 95.00 | 97.00 | 95.00 | 88.00 | 95.00 |
> 99th latency test result.
> || || Test1 || Test2 || Test3 || Test4 || Test5 ||
> | randomWrite | 600.00 | 600.00 | 650.00 | 404.00 | 425.00 |
> | asyncRandomWrite | 339.00 | 327.00 | 297.00 | 311.00 | 318.00 |
> In our internal 0.98 branch, the PE test result shows the async write has the almost
same latency with the blocking write. But for master branch, the result shows the async write
has better latency than the blocking client.  Take a look about the code, I thought the difference
is the BufferedMutator. For master branch, HTable don't have a write buffer and all write
request will be flushed directly. And user can use BufferedMutator when user want to perform
client-side buffering of writes. For the performance issue (autoFlush=True), I thought we
can use rpc caller directly in HTable's put method. Thanks.
> Review: https://reviews.apache.org/r/61454/
> Copy the comments from [~chia7712]. Remove the BufferdMutator brings four benefits.
> 1. correct the metrics (see HBASE-18476)
> 2. make HTable thread-safe (see HBASE-17368)
> 3. reduce the latency
> 4. get rid of some deprecated methods in Table



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message