hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Xinan Wu <wuxi...@gmail.com>
Subject Re: htable.getScanner() slow?
Date Fri, 29 May 2009 02:06:46 GMT
Ryan,

Thanks for the reply. I tried tweaking scanner caching but did not
change the speed much. The test I ended up doing was just getScanner()
and then immedietely scanner.close() without issuing scanner.next()...

Anyway, it's good to know HBase 0.20 may improve the speed. Is slow
scanner a known issue with hbase < 0.19 too? (I am using 0.19.2/3, but
am just curious...)

Xinan

On Thu, May 28, 2009 at 6:56 PM, Ryan Rawson <ryanobjc@gmail.com> wrote:
> Hi,
>
> You should consider setting scanner caching to reduce the number of
> server-round trips.
>
> But slow scanners is a known problem with 0.19.  HBase 0.20 aims to fix this
> substantially.  Shocking speed gains are hopefully going to be par for the
> course.
>
> -ryan
>
> On Thu, May 28, 2009 at 6:47 PM, Xinan Wu <wuxinan@gmail.com> wrote:
>
>> Hi,
>>
>> I've been experimenting row scanning in hbase recently, following
>> advice from
>> http://devblog.streamy.com/2009/04/23/hbase-row-key-design-for-paging-limit-offset-queries/
>> ?.
>>
>> One thing I notice is htable.getScanner() function call is very slow...
>>
>> My table schema is very simple. Integer (as binary 4 bytes) as rowKey,
>> and single column family..
>>
>> If I store 100 records in the same row with different columns, I can
>> get all the them with a single API call, at about 350 requests per
>> second (but paging would not be very scalable if records# gets
>> larger).
>>
>> If I store 100 records in 100 different rows (with sort-key appended
>> to rowKey), then I can use scanner to get them (and paging would be
>> more scalable). However, getScanner() call takes about 60 ms to
>> return, and subsequent scanner.next() calls are very fast. Overall,
>> this gives me only 15 requests per second.
>>
>> My dev box is ubuntu 8.04 2.4GHz Quad, 4GB mem, pretty typical one.
>>
>> Anyone has experience with slow scanner creation? Any suggestions?
>>
>> Xinan
>>
>

Mime
View raw message