hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From James Taylor <jtay...@salesforce.com>
Subject Re: 答复: setMaxResultSize method in Scan
Date Mon, 17 Mar 2014 19:56:37 GMT
Hi Weiping,
Take a look at Apache Phoenix (http://phoenix.incubator.apache.org/). It's
a SQL layer on top of HBase and has support for LIMIT and a query planner
and optimizer.
Thanks,
James


On Mon, Mar 17, 2014 at 12:19 PM, Weiping Qu <qu@informatik.uni-kl.de>wrote:

> I am doing a mult-thread(100) scan test over hbase.
> If one request with given key-range matches a large number of
> correspoding rows in hbase, my request is waiting for this scan to
> complete.
> The throughput is really slow.
> For test purpose, I'd like to use LIMIT to reduce the time on scanning
> and transferring results back from hbase to increase the throughput.
> Do you think the "hbase.client.scan.max.result.size" or
> setMaxResultSize(in bytes) could help HBase to stop scan at the LIMIT
> before scanning complete corresponding rows?
>
> As you mentioned that there is no query optimizer in HBase, I assume
> that region servers will not stop scanning the rows in this key-range in
> this case until it gets all the results and limit the results to max
> size which is sent to the client.
> If so, there is not much I can do to compare the throughput with that in
> relational databases like MySQL.
>
> Thanks,
> Cheers.
> > Limit clause in SQL Select statement makes sense because it allows query
> optimizer to plan accordingly.
> > It does not make sense in HBase as since there is no query planner and
> or optimization involved during
> > scanning HBase table. You can easily mimic this functionality on a
> client side (I mean - limit).
> >
> > Best regards,
> > Vladimir Rodionov
> > Principal Platform Engineer
> > Carrier IQ, www.carrieriq.com
> > e-mail: vrodionov@carrieriq.com
> >
> > ________________________________________
> > From: Weiping Qu [qu@informatik.uni-kl.de]
> > Sent: Monday, March 17, 2014 4:48 AM
> > To: dev@hbase.apache.org
> > Subject: Re: 答复: setMaxResultSize method in Scan
> >
> > Thanks.
> >
> > I'd like to assume that setMaxResultSize is equivalent to the SQL Limit
> > clause as expected which is specified each time a SQL statement is
> > executed .
> > Now through "hbase.client.scanner.max.result.size" can the limitation of
> > number of row returned only apply to all the scanner instances.
> > I am wondering why the setMaxResultSize is removed now.
> >
> >> No such method for Scan in 0.94.x.
> >>
> >> If you want to set the max result size for a scan, you can achieve this
> by setting the "hbase.client.scanner.max.result.size" configuration, the
> default for which is Long.MAX_VALUE (no limited)
> >> ________________________________________
> >> 发件人: Weiping Qu [qu@informatik.uni-kl.de]
> >> 发送时间: 2014年3月17日 18:50
> >> 收件人: dev@hbase.apache.org
> >> 主题: setMaxResultSize method in Scan
> >>
> >> Hello,
> >>
> >> I could not find the method setMaxResultSize(long m)
> >> (
> http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/Scan.html)
> >> in my Scanclass (0.94.13 version).
> >> Can anyone help me? Thanks
> >>
> >> Weiping
> >
> > --
> > Mit freundlichen Grü?en / Kind Regards
> >
> > *Weiping Qu*
> >
> > University of Kaiserslautern
> > Department of Computer Science
> > Heterogeneous Information Systems Group
> > P.O. Box 3049
> > 67653 Kaiserslautern, Germany
> >
> > Email: qu (at) informatik.uni-kl.de
> > Phone: +49 631 205 3264
> > Fax: +49 631 205 3299
> > Room: 36/331
> >
> > Confidentiality Notice:  The information contained in this message,
> including any attachments hereto, may be confidential and is intended to be
> read only by the individual or entity to whom this message is addressed. If
> the reader of this message is not the intended recipient or an agent or
> designee of the intended recipient, please note that any review, use,
> disclosure or distribution of this message or its attachments, in any form,
> is strictly prohibited.  If you have received this message in error, please
> immediately notify the sender and/or Notifications@carrieriq.com and
> delete or destroy any copy of this message and its attachments.
>
>
> --
> Mit freundlichen Grü?en / Kind Regards
>
> *Weiping Qu*
>
> University of Kaiserslautern
> Department of Computer Science
> Heterogeneous Information Systems Group
> P.O. Box 3049
> 67653 Kaiserslautern, Germany
>
> Email: qu (at) informatik.uni-kl.de
> Phone: +49 631 205 3264
> Fax: +49 631 205 3299
> Room: 36/331
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message