hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ryan Rawson <ryano...@gmail.com>
Subject Re: Hypertable claiming upto >900% random-read throughput vs HBase
Date Wed, 15 Dec 2010 20:34:38 GMT
I've looked in to this a lot, and the summary is 'easier said than
done'.  If you look at Terracotta they are using serialization of data
structures to off-heap ram, so it really is kind of like those EMM
systems from ye olde dose days.

Having done some prototypes of this, the most likely use case is to
store the heap blocks in off-heap ram, but still iterate over them in
java space.  To make life possible/easy we'd have to copy KeyValues
out of the block buffer in to per-scanner chunks, and this overhead
might negate other benefits.  Without building the whole stack out its
hard to construct a microbenchmark that isnt hopelessly naive.

In the mean time, the state of the GC space is getting more attention.
  G1 seems like it should be soon a viable alternative to those
systems which aren't sensitive to 250ms minor collection pauses.

-ryan

On Wed, Dec 15, 2010 at 12:30 PM, Todd Lipcon <todd@cloudera.com> wrote:
> On Wed, Dec 15, 2010 at 12:27 PM, Vladimir Rodionov
> <vrodionov@carrieriq.com> wrote:
>> Why do not you use off heap memory for this purpose? If its block cache (all blocks
are of equal sizes)
>> alloc/free algorithm is pretty much simple - you do not have to re-implement malloc
in Java.
>
> The block cache unfortunately isn't all equal size - if you have a
> single cell larger than the hfile block size, the block expands to fit
> it.
>
> That said we could use a fairly simple slab allocator.
>
> The bigger difficulty is in reference counting/tracking - the hfile
> blocks are zero-copied out all the way to the RPC implementation so
> tracking references is not straightforward.
>
> -Todd
>
>>
>> I think something like open source version of Terracotta BigMemory is a good candidate
for
>> Apache project. I see at least  several large Hadoops : HBase, HDFS DataNodes, TaskTrackers
and NameNode who suffer a lot from GC timeouts.
>>
>>
>> Best regards,
>> Vladimir Rodionov
>> Principal Platform Engineer
>> Carrier IQ, www.carrieriq.com
>> e-mail: vrodionov@carrieriq.com
>>
>> ________________________________________
>> From: Ryan Rawson [ryanobjc@gmail.com]
>> Sent: Wednesday, December 15, 2010 11:52 AM
>> To: dev@hbase.apache.org
>> Subject: Re: Hypertable claiming upto >900% random-read throughput vs HBase
>>
>> The malloc thing was pointing out that we have to contend with Xmx and
>> GC.  So it makes it harder for us to maximally use all the available
>> ram for block cache in the regionserver.  Which you may or may not
>> want to do for alternative reasons.  At least with Xmx you can plan
>> and control your deployments, and you wont suffer from heap growth due
>> to heap fragmentation.
>>
>> -ryan
>>
>> On Wed, Dec 15, 2010 at 11:49 AM, Todd Lipcon <todd@cloudera.com> wrote:
>>> On Wed, Dec 15, 2010 at 11:44 AM, Gaurav Sharma
>>> <gaurav.gs.sharma@gmail.com> wrote:
>>>> Thanks Ryan and Ted. I also think if they were using tcmalloc, it would have
>>>> given them a further advantage but as you said, not much is known about the
>>>> test source code.
>>>
>>> I think Hypertable does use tcmalloc or jemalloc (forget which)
>>>
>>> You may be interested in this thread from back in August:
>>> http://search-hadoop.com/m/pG6SM1xSP7r/hypertable&subj=Re+Finding+on+HBase+Hypertable+comparison
>>>
>>> -Todd
>>>
>>>>
>>>> On Wed, Dec 15, 2010 at 2:22 PM, Ryan Rawson <ryanobjc@gmail.com> wrote:
>>>>
>>>>> So if that is the case, I'm not sure how that is a fair test.  One
>>>>> system reads from RAM, the other from disk.  The results as expected.
>>>>>
>>>>> Why not test one system with SSDs and the other without?
>>>>>
>>>>> It's really hard to get apples/oranges comparison. Even if you are
>>>>> doing the same workloads on 2 diverse systems, you are not testing the
>>>>> code quality, you are testing overall systems and other issues.
>>>>>
>>>>> As G1 GC improves, I expect our ability to use larger and larger heaps
>>>>> would blunt the advantage of a C++ program using malloc.
>>>>>
>>>>> -ryan
>>>>>
>>>>> On Wed, Dec 15, 2010 at 11:15 AM, Ted Dunning <tdunning@maprtech.com>
>>>>> wrote:
>>>>> > From the small comments I have heard, the RAM versus disk difference
is
>>>>> > mostly what I have heard they were testing.
>>>>> >
>>>>> > On Wed, Dec 15, 2010 at 11:11 AM, Ryan Rawson <ryanobjc@gmail.com>
>>>>> wrote:
>>>>> >
>>>>> >> We dont have the test source code, so it isnt very objective.
 However
>>>>> >> I believe there are 2 things which help them:
>>>>> >> - They are able to harness larger amounts of RAM, so they are
really
>>>>> >> just testing that vs HBase
>>>>> >>
>>>>> >
>>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Todd Lipcon
>>> Software Engineer, Cloudera
>>>
>>
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>

Mime
View raw message