hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vladimir Rodionov <vrodio...@carrieriq.com>
Subject RE: 答复: setMaxResultSize method in Scan
Date Mon, 17 Mar 2014 21:36:43 GMT
Sure.

Best regards,
Vladimir Rodionov
Principal Platform Engineer
Carrier IQ, www.carrieriq.com
e-mail: vrodionov@carrieriq.com

________________________________________
From: Ted Yu [yuzhihong@gmail.com]
Sent: Monday, March 17, 2014 1:34 PM
To: dev@hbase.apache.org
Subject: Re: 答复: setMaxResultSize method in Scan

bq. Scan.setRowCaching()

I think you meant Scan.setCaching()


On Mon, Mar 17, 2014 at 1:28 PM, Vladimir Rodionov
<vrodionov@carrieriq.com>wrote:

>
> HBase RegionServer does scanning in batches, client requests next batch
> from server
> and server reads and merge the data from cache/disk. You can control batch
> data size by setting both:
>
> Scan.setRowCaching(number of rows to send in one RPC request)
>
> Technically speaking, this allows you to control LIMIT from the client
> side. Your overhead will never be larger than the limit set by
> setRowCaching.
>
> Best regards,
> Vladimir Rodionov
> Principal Platform Engineer
> Carrier IQ, www.carrieriq.com
> e-mail: vrodionov@carrieriq.com
>
> ________________________________________
> From: Weiping Qu [qu@informatik.uni-kl.de]
> Sent: Monday, March 17, 2014 12:19 PM
> To: dev@hbase.apache.org
> Subject: Re: 答复: setMaxResultSize method in Scan
>
> I am doing a mult-thread(100) scan test over hbase.
> If one request with given key-range matches a large number of
> correspoding rows in hbase, my request is waiting for this scan to
> complete.
> The throughput is really slow.
> For test purpose, I'd like to use LIMIT to reduce the time on scanning
> and transferring results back from hbase to increase the throughput.
> Do you think the "hbase.client.scan.max.result.size" or
> setMaxResultSize(in bytes) could help HBase to stop scan at the LIMIT
> before scanning complete corresponding rows?
>
> As you mentioned that there is no query optimizer in HBase, I assume
> that region servers will not stop scanning the rows in this key-range in
> this case until it gets all the results and limit the results to max
> size which is sent to the client.
> If so, there is not much I can do to compare the throughput with that in
> relational databases like MySQL.
>
> Thanks,
> Cheers.
> > Limit clause in SQL Select statement makes sense because it allows query
> optimizer to plan accordingly.
> > It does not make sense in HBase as since there is no query planner and
> or optimization involved during
> > scanning HBase table. You can easily mimic this functionality on a
> client side (I mean - limit).
> >
> > Best regards,
> > Vladimir Rodionov
> > Principal Platform Engineer
> > Carrier IQ, www.carrieriq.com
> > e-mail: vrodionov@carrieriq.com
> >
> > ________________________________________
> > From: Weiping Qu [qu@informatik.uni-kl.de]
> > Sent: Monday, March 17, 2014 4:48 AM
> > To: dev@hbase.apache.org
> > Subject: Re: 答复: setMaxResultSize method in Scan
> >
> > Thanks.
> >
> > I'd like to assume that setMaxResultSize is equivalent to the SQL Limit
> > clause as expected which is specified each time a SQL statement is
> > executed .
> > Now through "hbase.client.scanner.max.result.size" can the limitation of
> > number of row returned only apply to all the scanner instances.
> > I am wondering why the setMaxResultSize is removed now.
> >
> >> No such method for Scan in 0.94.x.
> >>
> >> If you want to set the max result size for a scan, you can achieve this
> by setting the "hbase.client.scanner.max.result.size" configuration, the
> default for which is Long.MAX_VALUE (no limited)
> >> ________________________________________
> >> 发件人: Weiping Qu [qu@informatik.uni-kl.de]
> >> 发送时间: 2014年3月17日 18:50
> >> 收件人: dev@hbase.apache.org
> >> 主题: setMaxResultSize method in Scan
> >>
> >> Hello,
> >>
> >> I could not find the method setMaxResultSize(long m)
> >> (
> http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/Scan.html)
> >> in my Scanclass (0.94.13 version).
> >> Can anyone help me? Thanks
> >>
> >> Weiping
> >
> > --
> > Mit freundlichen Grü?en / Kind Regards
> >
> > *Weiping Qu*
> >
> > University of Kaiserslautern
> > Department of Computer Science
> > Heterogeneous Information Systems Group
> > P.O. Box 3049
> > 67653 Kaiserslautern, Germany
> >
> > Email: qu (at) informatik.uni-kl.de
> > Phone: +49 631 205 3264
> > Fax: +49 631 205 3299
> > Room: 36/331
> >
> > Confidentiality Notice:  The information contained in this message,
> including any attachments hereto, may be confidential and is intended to be
> read only by the individual or entity to whom this message is addressed. If
> the reader of this message is not the intended recipient or an agent or
> designee of the intended recipient, please note that any review, use,
> disclosure or distribution of this message or its attachments, in any form,
> is strictly prohibited.  If you have received this message in error, please
> immediately notify the sender and/or Notifications@carrieriq.com and
> delete or destroy any copy of this message and its attachments.
>
>
> --
> Mit freundlichen Grü?en / Kind Regards
>
> *Weiping Qu*
>
> University of Kaiserslautern
> Department of Computer Science
> Heterogeneous Information Systems Group
> P.O. Box 3049
> 67653 Kaiserslautern, Germany
>
> Email: qu (at) informatik.uni-kl.de
> Phone: +49 631 205 3264
> Fax: +49 631 205 3299
> Room: 36/331
>

Confidentiality Notice:  The information contained in this message, including any attachments
hereto, may be confidential and is intended to be read only by the individual or entity to
whom this message is addressed. If the reader of this message is not the intended recipient
or an agent or designee of the intended recipient, please note that any review, use, disclosure
or distribution of this message or its attachments, in any form, is strictly prohibited. 
If you have received this message in error, please immediately notify the sender and/or Notifications@carrieriq.com
and delete or destroy any copy of this message and its attachments.
Mime
View raw message