hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ted Yu (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-20601) Add multiPut support and other miscellaneous to PE
Date Sat, 19 May 2018 13:23:00 GMT

     [ https://issues.apache.org/jira/browse/HBASE-20601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ted Yu updated HBASE-20601:
---------------------------
    Release Note: 
1. Add multiPut support
Set --multiPut=number to enable batchput(meanwhile, --autoflush need be set to false)

2. Add Connection Count support
Added a new parameter connCount to PE. set --connCount=2 means all threads will share 2 connections.
oneCon option takes precedence over connCount

3. Add avg RT and avg TPS/QPS statstic for all threads

4. Delete some redundant code
Now RandomWriteTest is inherited from SequentialWrite.

  was:
1. Add multiPut support
Set --multiPut=number to enable batchput(meanwhile, --autoflush need be set to false)

2. Add Connection Count support
Added a new parameter connCount to PE. set --connCount=2 means all threads will share 2 connections.

3. Add avg RT and avg TPS/QPS statstic for all threads

4. Delete some redundant code
Now RandomWriteTest is inherited from SequentialWrite.


> Add multiPut support and other miscellaneous to PE
> --------------------------------------------------
>
>                 Key: HBASE-20601
>                 URL: https://issues.apache.org/jira/browse/HBASE-20601
>             Project: HBase
>          Issue Type: Bug
>          Components: tooling
>    Affects Versions: 2.0.0
>            Reporter: Allan Yang
>            Assignee: Allan Yang
>            Priority: Minor
>             Fix For: 2.1.0
>
>         Attachments: HBASE-20601.002.patch, HBASE-20601.branch-2.002.patch, HBASE-20601.branch-2.003.patch,
HBASE-20601.branch-2.004.patch, HBASE-20601.branch-2.005.patch, HBASE-20601.branch-2.patch,
HBASE-20601.patch
>
>
> Add some useful stuff and some refinement to PE tool
> 1. Add multiPut support
> Though we have BufferedMutator, sometimes we need to benchmark batch put in a certain
number.
> Set --multiPut=number to enable batchput(meanwhile, --autoflush need be set to false)
> 2. Add Connection Number support
> Before, there is only on parameter to control the connection used by threads. oneCon=true
means all threads use one connection, false means each thread has it own connection.
> When thread number is high and oneCon=false, we noticed high context switch frequency
in the machine which PE run on, disturbing the benchmark results(each connection has its own
netty worker threads, 2*CPU IIRC).  
> So, added a new parameter connCount to PE. set --connCount=2 means all threads will share
2 connections.
> 3. Add avg RT and avg TPS/QPS statstic for all threads
> Useful when we want to meansure the total throughtput of the cluster
> 4. Delete some redundant code
> Now RandomWriteTest is inherited from SequentialWrite.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message