hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lars George <lars.geo...@gmail.com>
Subject Re: Multiple HBase instances per machine
Date Sat, 16 Jun 2012 13:51:07 GMT
Hi,

I have done this at a customer site to overcome the 0.90.x slow WAL performance. With one
RS per DN we bottlenecked, with 5-7 RS per DN we were able to hit the target rate. 

Please note that we did this in lieu of the proper built-in options like WAL compression,
multiple WAL, or n-way writes for WALs. 

Lars 

On Jun 16, 2012, at 10:53, Em <mailformailinglists@yahoo.de> wrote:

> Stack,
> 
> I have no issues with HBase, the question is purely theoretical.
> 
>> So, you intend doubling the datanode instances per machine too?
> Everything else would not make sense to me, or what do you think?
> 
> Thanks for your feedback!
> 
> Regards,
> Em
> 
> Am 16.06.2012 07:12, schrieb Stack:
>> On Fri, Jun 15, 2012 at 10:50 AM, Em <mailformailinglists@yahoo.de> wrote:
>>> what are your experience with having more than one HBase-instance per
>>> machine?
>>> 
>> 
>> My experience is ittle.
>> 
>> Its been done though by others on this list.
>> 
>>> One reason to do this could be that you got i.e. 32 GB RAM and want to
>>> give 12-14 GB of RAM to each HBase instance so that you got little
>>> issues with Garbage Collection while using the available RAM-capacity
>>> for caching.
>>> 
>> 
>> You should leave some RAM for the file system cache.
>> 
>> Are you unable to tame your GCs?  Is that why the two servers per node?
>> 
>>> One thing I worry about is replication of data. Say two of three
>>> replicas reside at the same box in two distinct HBase instances.
>> 
>> So, you intend doubling the datanode instances per machine too?
>> 
>> St.Ack
>> 

Mime
View raw message