hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-9488) Improve performance for small scan
Date Wed, 11 Sep 2013 05:22:53 GMT

    [ https://issues.apache.org/jira/browse/HBASE-9488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13764000#comment-13764000
] 

stack commented on HBASE-9488:
------------------------------



+1 on commit for trunk and 0.96.

I tried it.  Here is scan of two rows without setting 'short' scan (Below is rpcserver debug
on regionserver):

{code}
2013-09-10 22:16:57,525 DEBUG [RpcServer.handler=19,port=60020] ipc.RpcServer: callId: 4 service:
ClientService methodName: Scan size: 143.0 connection: 192.168.1.129:53587 executing as stack
2013-09-10 22:16:57,526 DEBUG [RpcServer.handler=19,port=60020] ipc.RpcServer: RpcServer.responder:
callId: 4 wrote 25 bytes.
2013-09-10 22:16:57,526 DEBUG [RpcServer.handler=24,port=60020] ipc.RpcServer: callId: 5 service:
ClientService methodName: Scan size: 29.0 connection: 192.168.1.129:53587 executing as stack
2013-09-10 22:16:57,526 DEBUG [RpcServer.handler=24,port=60020] ipc.RpcServer: RpcServer.responder:
callId: 5 wrote 2150 bytes.
2013-09-10 22:16:57,527 DEBUG [RpcServer.handler=26,port=60020] ipc.RpcServer: callId: 6 service:
ClientService methodName: Scan size: 27.0 connection: 192.168.1.129:53587 executing as stack
2013-09-10 22:16:57,527 DEBUG [RpcServer.handler=26,port=60020] ipc.RpcServer: RpcServer.responder:
callId: 6 wrote 21 bytes.
{code}

Thats open, next, and close (I presume); all call the 'scan' method.

Here is what I see when I set short scan to true:

{code}
2013-09-10 22:20:54,164 DEBUG [RpcServer.handler=11,port=60020] ipc.RpcServer: callId: 4 service:
ClientService methodName: Scan size: 145.0 connection: 192.168.1.129:53605 executing as stack
2013-09-10 22:20:54,164 DEBUG [RpcServer.handler=11,port=60020] ipc.RpcServer: RpcServer.responder:
callId: 4 wrote 2146 bytes.
{code}

Just one call in which open, next, and close are all done.

Please change this log to trace level on commit because it will get annoying pretty quick
given DEBUG is our default log level:

+    if (LOG.isDebugEnabled()) {
+      LOG.debug("Advancing internal small scanner to startKey at '"
+          + Bytes.toStringBinary(localStartKey) + "'");
+    }


Nice one [~zjushch]

[~davelatham] You want this don't you?
                
> Improve performance for small scan
> ----------------------------------
>
>                 Key: HBASE-9488
>                 URL: https://issues.apache.org/jira/browse/HBASE-9488
>             Project: HBase
>          Issue Type: Improvement
>          Components: Client, Performance, Scanners
>            Reporter: chunhui shen
>            Assignee: chunhui shen
>             Fix For: 0.98.0, 0.94.13
>
>         Attachments: hbase-9488-94-v3.patch, HBASE-9488-trunk.patch, HBASE-9488-trunkV2.patch,
HBASE-9488-trunkV3.patch, test results.jpg
>
>
> review board:
> https://reviews.apache.org/r/14059/
> *Performance Improvement*
> Test shows about 1.5~3X improvement for small scan where limit<=50 under cache hit
ratio=100%.
> See more performance test result from the picture attachment
> *Usage:*
> Scan scan = new Scan(startRow,stopRow);
> scan.setSmall(true);
> ResultScanner scanner = table.getScanner(scan);
> Set the new 'small' attribute as true for scan object, others are the same
> Now, one scan operation would call 3 RPC at least:
> openScanner();
> next();
> closeScanner();
> I think we could reduce the RPC call to one for small scan to get better performance
> Also using pread is better than seek+read for small scan (For this point, see more on
HBASE-7266)
> Implements such a small scan as the patch, and take the performance test as following:
> a.Environment´╝Ü
> patched on 0.94 version
> one regionserver; 
> one client with 50 concurrent threads;
> KV size:50/100;
> 100% LRU cache hit ratio;
> Random start row of scan
> b.Results:
> See the picture attachment
>  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message