hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Gray <jl...@streamy.com>
Subject Re: HBase-0.20.0 randomRead
Date Tue, 18 Aug 2009 18:56:55 GMT
With all that memory, you're likely seeing such good performance because 
of filesystem caching.  As you say, 2ms is extraordinarily fast for a 
disk read, but since your rows are relatively small, you are loading up 
all that data into memory (not only the fs cache, but also hbase's block 
cache which makes it even faster).

JG

Jean-Daniel Cryans wrote:
> Well it seems there's something wrong with the way you modified PE. It
> is not really testing your table unless the row keys are built the
> same way as TestTable is, to me it seems that you are testing on only
> 20000 rows so caching is easy. A better test would just be to use PE
> the way it currently is but with ROW_LENGTH = 4k.
> 
> WRT Jetty, make sure you optimized it with
> http://jetty.mortbay.org/jetty5/doc/optimization.html
> 
> J-D
> 
> On Tue, Aug 18, 2009 at 12:08 PM, Murali Krishna.
> P<muralikpbhat@yahoo.com> wrote:
>> Ahh, mistake, I just took it as seconds.
>>
>> Now I wonder whether it can really do that fast ?? wont it take atleast 2ms for disk
read? ( I have given 8G heapspace for RegionServer, is it caching so much?). Has anyone seen
these kind of numbers ?
>>
>>
>> Actually, my initial problem was that I have a jetty infront of this hbase to serve
this 4k value and when bench marked, it took 200+milliseconds for each record with 100 clients.
That is why decided to benchmark without jetty first.
>>
>> Thanks,
>> Murali Krishna
>>
>>
>>
>>
>> ________________________________
>> From: Jean-Daniel Cryans <jdcryans@apache.org>
>> To: hbase-user@hadoop.apache.org
>> Sent: Tuesday, 18 August, 2009 9:13:40 PM
>> Subject: Re: HBase-0.20.0 randomRead
>>
>> Murali,
>>
>> I'm not reading the same thing as you.
>>
>> client-0 Finished randomRead in 2867ms at offset 0 for 10000 rows
>>
>> That means 2867 / 10000 = 0.2867ms per row. It's kinda fast.
>>
>> J-D
>>
>> On Tue, Aug 18, 2009 at 11:35 AM, Murali Krishna.
>> P<muralikpbhat@yahoo.com> wrote:
>>> Hi all,
>>>  (Saw a related thread on performance, but starting a different one because my
setup is slightly different).
>>>
>>> I have an one node setup with hbase-0..20(alpha). It has around 11million rows
with ~250 regions. Each row with ~20 bytes sized key and ~4k sized value.
>>> Since my primary concern is randomRead, modified the performanceEvaluation code
to read from this particular table. The randomRead test gave following result.
>>>
>>> 09/08/18 08:20:41 INFO hbase.PerformanceEvaluation: client-1 Finished randomRead
in 2813ms at offset 10000 for 10000 rows
>>> 09/08/18 08:20:41 INFO hbase.PerformanceEvaluation: Finished 1 in 2813ms writing
10000 rows
>>> 09/08/18 08:20:41 INFO hbase.PerformanceEvaluation: client-0 Finished randomRead
in 2867ms at offset 0 for 10000 rows
>>> 09/08/18 08:20:41 INFO hbase.PerformanceEvaluation: Finished 0 in 2867ms writing
10000 rows
>>>
>>>
>>> So, looks like it is taking around 280ms per record. Looking at the latest hbase
performance claims, I was expecting it below 10ms. Am  I doing something basically wrong,
since such a hiuge difference :( ? Please help me fix the latency.
>>>
>>> The machine config is:
>>> Processors:    2 x Xeon L5420 2.50GHz (8 cores)
>>> Memory:        13.7GB
>>> 12 Disks of 1TB each.
>>>
>>> Let me know if you need anymore details
>>>
>>> Thanks,
>>> Murali Krishna
> 

Mime
View raw message