hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Liang Xie (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-7495) parallel seek in StoreScanner
Date Fri, 01 Feb 2013 10:05:13 GMT

    [ https://issues.apache.org/jira/browse/HBASE-7495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13568620#comment-13568620
] 

Liang Xie commented on HBASE-7495:
----------------------------------

The failed case seems not related with this patch,  and i ran it on my box and passed.
About testing in real cluster, i did it today after applied v8 patch into 0.94.3:

$./hdfs dfs -du -s -h hdfs://lgxl-xieliang/
619.0g  hdfs://lgxl-xieliang/

one billion rows, 10 RegionServer&Datanode, each with 2T*12 SATA disk. Disabled block
cache in RS, 10 regions in each RS, most of numberOfStorefiles in each region are around 40,
and most of single storefileSizes are around 160M+.  I ran with 1 thread in YCSB:

With original 0.94.3:

2013-02-01 14:33:39:563 300 sec: 570 operations; 2.3 current ops/sec; [READ AverageLatency(us)=419965.74]
2013-02-01 14:33:39:599 300 sec: 571 operations; 27.78 current ops/sec; [READ AverageLatency(us)=616807]
[OVERALL], RunTime(ms), 300051.0
[OVERALL], Throughput(ops/sec), 1.9030098216636504
[READ], Operations, 571
[READ], AverageLatency(us), 525441.6672504378
[READ], MinLatency(us), 78542
[READ], MaxLatency(us), 1481186
[READ], 50thPercentileLatency(ms), 448
[READ], 95thPercentileLatency(ms), 854
[READ], Return=0, 571

Applied v8 patch, hbase.storescanner.parallel.seek.enable = false :

2013-02-01 16:49:27:624 300 sec: 643 operations; 2.5 current ops/sec; [READ AverageLatency(us)=401727.8]
2013-02-01 16:49:28:032 300 sec: 644 operations; 2.45 current ops/sec; [READ AverageLatency(us)=474225]
[OVERALL], RunTime(ms), 300424.0
[OVERALL], Throughput(ops/sec), 2.143636993049823
[READ], Operations, 644
[READ], AverageLatency(us), 466452.77639751555
[READ], MinLatency(us), 156654
[READ], MaxLatency(us), 1279945
[READ], 50thPercentileLatency(ms), 439
[READ], 95thPercentileLatency(ms), 724
[READ], Return=0, 644

Applied v8 patch, hbase.storescanner.parallel.seek.enable = true :

2013-02-01 16:59:34:689 300 sec: 4594 operations; 16.01 current ops/sec; [READ AverageLatency(us)=63779.27]
[OVERALL], RunTime(ms), 300008.0
[OVERALL], Throughput(ops/sec), 15.31292498866697
[READ], Operations, 4594
[READ], AverageLatency(us), 65277.21745755333
[READ], MinLatency(us), 7661
[READ], MaxLatency(us), 1026107
[READ], 50thPercentileLatency(ms), 52
[READ], 95thPercentileLatency(ms), 129
[READ], 99thPercentileLatency(ms), 272
[READ], Return=0, 4594
                
> parallel seek in StoreScanner
> -----------------------------
>
>                 Key: HBASE-7495
>                 URL: https://issues.apache.org/jira/browse/HBASE-7495
>             Project: HBase
>          Issue Type: Bug
>          Components: Scanners
>    Affects Versions: 0.94.3, 0.96.0
>            Reporter: Liang Xie
>            Assignee: Liang Xie
>         Attachments: HBASE-7495.txt, HBASE-7495.txt, HBASE-7495.txt, HBASE-7495-v2.txt,
HBASE-7495-v3.txt, HBASE-7495-v4.txt, HBASE-7495-v4.txt, HBASE-7495-v5.txt, HBASE-7495-v6.txt,
HBASE-7495-v7.txt, HBASE-7495-v8.txt
>
>
> seems there's a potential improvable space before doing scanner.next:
> {code:title=StoreScanner.java|borderStyle=solid}
>     if (explicitColumnQuery && lazySeekEnabledGlobally) {
>       for (KeyValueScanner scanner : scanners) {
>         scanner.requestSeek(matcher.getStartKey(), false, true);
>       }
>     } else {
>       for (KeyValueScanner scanner : scanners) {
>         scanner.seek(matcher.getStartKey());
>       }
>     }
> {code} 
> we can do scanner.requestSeek or scanner.seek in parallel, instead of current serialization,
to reduce latency for special case.
> Any ideas on it ?  I'll have a try if the comments/suggestions are positive:)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message