hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ryan Rawson <ryano...@gmail.com>
Subject Re: htable.getScanner() slow?
Date Fri, 29 May 2009 01:56:40 GMT
Hi,

You should consider setting scanner caching to reduce the number of
server-round trips.

But slow scanners is a known problem with 0.19.  HBase 0.20 aims to fix this
substantially.  Shocking speed gains are hopefully going to be par for the
course.

-ryan

On Thu, May 28, 2009 at 6:47 PM, Xinan Wu <wuxinan@gmail.com> wrote:

> Hi,
>
> I've been experimenting row scanning in hbase recently, following
> advice from
> http://devblog.streamy.com/2009/04/23/hbase-row-key-design-for-paging-limit-offset-queries/
> ?.
>
> One thing I notice is htable.getScanner() function call is very slow...
>
> My table schema is very simple. Integer (as binary 4 bytes) as rowKey,
> and single column family..
>
> If I store 100 records in the same row with different columns, I can
> get all the them with a single API call, at about 350 requests per
> second (but paging would not be very scalable if records# gets
> larger).
>
> If I store 100 records in 100 different rows (with sort-key appended
> to rowKey), then I can use scanner to get them (and paging would be
> more scalable). However, getScanner() call takes about 60 ms to
> return, and subsequent scanner.next() calls are very fast. Overall,
> this gives me only 15 requests per second.
>
> My dev box is ubuntu 8.04 2.4GHz Quad, 4GB mem, pretty typical one.
>
> Anyone has experience with slow scanner creation? Any suggestions?
>
> Xinan
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message