hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From divye sheth <divs.sh...@gmail.com>
Subject Re: DFS Balancer with Hbase
Date Tue, 04 Mar 2014 13:24:41 GMT
Thanks Jean, but why does only a couple of RS get loaded with data? We are
seeing out of 5 only 2 datanodes have around 90% of disk usage. Where as
the rest are at around 40%.

We have run the hbase balancer, and on an average we have around 500
regions per regionserver and a total of 5 RS's. We have even disabled
number of tables which are not required and currently the count of
regions/RS is around 120.

Another question that comes to my mind is. Somewhere down the line the
Hadoop cluster tends to be imbalanced and lead to 100% disk utilization and
the balancer activity has to be triggered, how do you guys handle such
problem in your hbase cluster?

Just a thought, could we execute the DFS balancer and after the balancing
activity trigger major compaction for each table?

Divye Sheth

On Tue, Mar 4, 2014 at 6:45 PM, Jean-Marc Spaggiari <jean-marc@spaggiari.org
> wrote:

> Hi Divye,
> the DFS balancer is that last thing you want to run in your HBase
> cluster.That will break all the data locallity for the compacted regions.
> On compaction, a region write the files on the local server first, then the
> 2 other replicates are going on different datanodes. so on read, HBase can
> garantee that data is read from local datanode dans not from another
> datanode over the network.
> Have you run the HBase balancer? How many regions do you have per region
> server?
> JM

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message