hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Weiping Qu ...@informatik.uni-kl.de>
Subject Re: 答复: setMaxResultSize method in Scan
Date Mon, 17 Mar 2014 19:19:07 GMT
I am doing a mult-thread(100) scan test over hbase.
If one request with given key-range matches a large number of
correspoding rows in hbase, my request is waiting for this scan to complete.
The throughput is really slow.
For test purpose, I'd like to use LIMIT to reduce the time on scanning
and transferring results back from hbase to increase the throughput.
Do you think the "hbase.client.scan.max.result.size" or
setMaxResultSize(in bytes) could help HBase to stop scan at the LIMIT
before scanning complete corresponding rows?

As you mentioned that there is no query optimizer in HBase, I assume
that region servers will not stop scanning the rows in this key-range in
this case until it gets all the results and limit the results to max
size which is sent to the client.
If so, there is not much I can do to compare the throughput with that in
relational databases like MySQL.

Thanks,
Cheers.
> Limit clause in SQL Select statement makes sense because it allows query optimizer to
plan accordingly.
> It does not make sense in HBase as since there is no query planner and or optimization
involved during
> scanning HBase table. You can easily mimic this functionality on a client side (I mean
- limit).
>
> Best regards,
> Vladimir Rodionov
> Principal Platform Engineer
> Carrier IQ, www.carrieriq.com
> e-mail: vrodionov@carrieriq.com
>
> ________________________________________
> From: Weiping Qu [qu@informatik.uni-kl.de]
> Sent: Monday, March 17, 2014 4:48 AM
> To: dev@hbase.apache.org
> Subject: Re: 答复: setMaxResultSize method in Scan
>
> Thanks.
>
> I'd like to assume that setMaxResultSize is equivalent to the SQL Limit
> clause as expected which is specified each time a SQL statement is
> executed .
> Now through "hbase.client.scanner.max.result.size" can the limitation of
> number of row returned only apply to all the scanner instances.
> I am wondering why the setMaxResultSize is removed now.
>
>> No such method for Scan in 0.94.x.
>>
>> If you want to set the max result size for a scan, you can achieve this by setting
the "hbase.client.scanner.max.result.size" configuration, the default for which is Long.MAX_VALUE
(no limited)
>> ________________________________________
>> 发件人: Weiping Qu [qu@informatik.uni-kl.de]
>> 发送时间: 2014年3月17日 18:50
>> 收件人: dev@hbase.apache.org
>> 主题: setMaxResultSize method in Scan
>>
>> Hello,
>>
>> I could not find the method setMaxResultSize(long m)
>> (http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/Scan.html)
>> in my Scanclass (0.94.13 version).
>> Can anyone help me? Thanks
>>
>> Weiping
>
> --
> Mit freundlichen Grü?en / Kind Regards
>
> *Weiping Qu*
>
> University of Kaiserslautern
> Department of Computer Science
> Heterogeneous Information Systems Group
> P.O. Box 3049
> 67653 Kaiserslautern, Germany
>
> Email: qu (at) informatik.uni-kl.de
> Phone: +49 631 205 3264
> Fax: +49 631 205 3299
> Room: 36/331
>
> Confidentiality Notice:  The information contained in this message, including any attachments
hereto, may be confidential and is intended to be read only by the individual or entity to
whom this message is addressed. If the reader of this message is not the intended recipient
or an agent or designee of the intended recipient, please note that any review, use, disclosure
or distribution of this message or its attachments, in any form, is strictly prohibited. 
If you have received this message in error, please immediately notify the sender and/or Notifications@carrieriq.com
and delete or destroy any copy of this message and its attachments.


-- 
Mit freundlichen Grü?en / Kind Regards

*Weiping Qu*

University of Kaiserslautern
Department of Computer Science
Heterogeneous Information Systems Group
P.O. Box 3049
67653 Kaiserslautern, Germany

Email: qu (at) informatik.uni-kl.de
Phone: +49 631 205 3264
Fax: +49 631 205 3299
Room: 36/331

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message