hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Segel <mse...@segel.com>
Subject Re: Multiple region servers per physical node
Date Wed, 31 Jul 2013 18:51:48 GMT
Its possible to walk to the middle of the Golden Gate Bridge, evade security and take a swan
dive in to the Bay. 
If you're in NY, substitute Golden Gate with Brooklyn .... 

The point is that while something is possible, doesn't mean that its a good idea to do it.


Elliot makes this point...
>  It's a head-ache to run and administer, but if you have
> machines with lots of spindles it's an option.  Though it's not the
> recommended way to run HBase so there may be scary edges.

The key phrase ... "Its a head-ache to run and administer" 
In the real world... there are things like BOFHs and its best not to get on their bad side.
;-) 

But there are other options... 
If you do have too man spindles, too much memory and too many cores.... you have another option...

Virtualize your nodes. You will take a performance hit on disk and network I/O but it gets
balanced out. 
VMWare (err Pivotal) has made claims about improving their virtualization software so that
there is less overhead. Note that YMMV. 

This way you can run more RS on the existing hardware. Note that you don't run HBase alone
and  with the loss of memory and cores, you would end up reducing the number of slots per
server. 

But its a viable solution and its recommended over trying to run multiple RS on a node. 

HTH

-Mike


On Jul 29, 2013, at 5:14 PM, Elliott Clark <eclark@apache.org> wrote:

> Currently it's very possible to run multiple region servers per
> machine.  People who have benchmarked it have even found that it's
> faster (there are more wal's being used in addition to the extra
> heap).  It's a head-ache to run and administer, but if you have
> machines with lots of spindles it's an option.  Though it's not the
> recommended way to run HBase so there may be scary edges.
> 
> Yes the G1 looks promising if you have a very read heavy workload
> (We've been running it for integration tests for about a month or so).
> But until multiple wal's are in HBase (it's going to be a while) some
> users might want try other solutions.
> 
> 
> On Mon, Jul 29, 2013 at 2:22 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>> Have you seen this thread ?
>> 
>> http://search-hadoop.com/m/PhUxhdw5Mz1/otis+g1gc&subj=Re+G1+before+after+GC+time+graph
>> 
>> On Mon, Jul 29, 2013 at 2:19 PM, Sudarshan Kadambi (BLOOMBERG/ 731 LEXIN) <
>> skadambi@bloomberg.net> wrote:
>> 
>>> If someone was contemplating running multiple HBase region servers per
>>> node, what would you tell him? I've lots of RAM on my machines and I want
>>> HBase to be able to make the most of it. Heaps larger than 12-16G have
>>> significant gc penalties. I could try to have a larger cache off-heap, but
>>> that feature is considered to be not fully baked in yet (according to
>>> people in the know). The option left over is to run multiple region servers
>>> per node. What are the downsides to that?
>>> 
>>> Regards,
>>> -sudarshan
> 


Mime
View raw message