hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lars hofhansl <la...@apache.org>
Subject Re: introducing nodes w/ more storage
Date Mon, 23 Mar 2015 03:56:35 GMT
Seems that it should not be too hard to add that to the stochastic load balancer.
We could add a spaceCost or something.



----- Original Message -----
From: Jean-Marc Spaggiari <jean-marc@spaggiari.org>
To: user <user@hbase.apache.org>
Cc: Development <Development@mentacapital.com>
Sent: Thursday, March 19, 2015 12:55 PM
Subject: Re: introducing nodes w/ more storage

You can extend the default balancer and assign the regions based on
that.But at the end, the replicated blocks might still go all over the
cluster and your "small" nodes are going to be full and will not be able to
get anymore writes even for the regions they are supposed to get.

I'm not sure there is a good solution for what you are looking for :(

I build my own balancer but because of differences in the CPUs, not because
of differences of the storage space...


2015-03-19 15:50 GMT-04:00 Nick Dimiduk <ndimiduk@gmail.com>:

> Seems more fantasy than fact, I'm afraid. The default load balancer [0]
> takes store file size into account, but has no concept of capacity. It
> doesn't know that nodes in a heterogenous environment have different
> capacity.
>
> This would be a good feature to add though.
>
> [0]:
>
> https://github.com/apache/hbase/blob/branch-1.0/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java
>
> On Tue, Mar 17, 2015 at 7:26 AM, Ted Tuttle <ted@mentacapital.com> wrote:
>
> > Hello-
> >
> > Sometime back I asked a question about introducing new nodes w/ more
> > storage that existing nodes.  I was told at the time that HBase will not
> be
> > able to utilize the additional storage; I assumed at the time that
> regions
> > are allocated to nodes in something like a round-robin fashion and the
> node
> > with the least storage sets the limit for how much each node can utilize.
> >
> > My question this time around has to do with nodes w/ unequal numbers of
> > volumes: Does HBase allocate regions based on nodes or volumes on the
> > nodes?  I am hoping I can add a node with 8 volumes totaling 8X TB and
> all
> > the volumes will be filled.  This even though legacy nodes have 5 volumes
> > and total storage of 5X TB.
> >
> > Fact or fantasy?
> >
> > Thanks,
> > Ted
> >
> >
>

Mime
View raw message