hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gary Helmling <ghelml...@gmail.com>
Subject Re: Record limit in scan api?
Date Fri, 20 Nov 2009 20:35:05 GMT
To set this per scan you should be able to do:

Scan s = new Scan()
s.setCaching(...)

(I think this works anyway)


The other thing that I've found useful is using a PageFilter on scans:
http://hadoop.apache.org/hbase/docs/r0.20.2/api/org/apache/hadoop/hbase/filter/PageFilter.html

I believe this is applied independently on each region server (?) so you
still need to do your own counting in iterating the results, but it can be
used to early out on the server side separately from the scanner caching
value.

--gh

On Fri, Nov 20, 2009 at 3:04 PM, stack <stack@duboce.net> wrote:

> There is this in the configuration:
>
>  <property>
>    <name>hbase.client.scanner.caching</name>
>    <value>1</value>
>    <description>Number of rows that will be fetched when calling next
>    on a scanner if it is not served from memory. Higher caching values
>    will enable faster scanners but will eat up more memory and some
>    calls of next may take longer and longer times when the cache is empty.
>    </description>
>  </property>
>
>
> Being able to do it per Scan sounds like something we should add.
>
> St.Ack
>
>
> On Fri, Nov 20, 2009 at 11:43 AM, Adam Silberstein
> <silberst@yahoo-inc.com>wrote:
>
> >   Hi,
> > Is there a way to specify a limit on number of returned records for scan?
> >  I
> > don¹t see any way to do this when building the scan.  If there is, that
> > would be great.  If not, what about when iterating over the result?  If I
> > exit the loop when I reach my limit, will that approximate this clause?
> I
> > guess my real question is about how scan is implemented in the client.
> >  I.e.
> > How many records are returned from Hbase at a time as I iterate through
> the
> > scan result?  If I want 1,000 records and 100 get returned at a time,
> then
> > I¹m in good shape.  On the other hand, if I want 10 records and get 100
> at
> > a
> > time, it¹s a bit wasteful, though the waste is bounded.
> >
> > Thanks,
> > Adam
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message