hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lars hofhansl <la...@apache.org>
Subject Re: Hbase region count and RS count for 2TB+
Date Fri, 15 Aug 2014 18:12:51 GMT
HBase initially tries to spread the load out to more region servers by splitting regions early
when there aren't many (per table).
As more regions are seen split size approaches the configured max file size.
This early splitting turned out to be a bit too aggressive (see HBASE-10501, which modifies
it a bit)... Even then 800 regions is a lot.

As others have asked, what's 

hbase.hregion.max.filesize set to? Make sure it's at least 10gb.


-- Lars



________________________________
 From: Juraj jiv <fatcap.jiv@gmail.com>
To: user@hbase.apache.org 
Sent: Friday, August 15, 2014 2:08 AM
Subject: Re: Hbase region count and RS count for 2TB+
 

Hi,
size 2.2TB i got from cmd "hdfs dfs -du -h /hbase/data"

So i think yes, 650MB per RS. If its small then why hbase created so many
regions?

Thanks
JV



On Fri, Aug 15, 2014 at 9:51 AM, Mikhail Antonov <olorinbant@gmail.com>
wrote:

> So you have 3200 regions in total, and that means each region is about
> 650mb in size only (if you have 2.2 Tb in total), unless I'm missing
> something? If that's the case your regions are probably way too small.
>
> -Mikhail
>
>
> 2014-08-15 0:44 GMT-07:00 Juraj jiv <fatcap.jiv@gmail.com>:
>
> > Hi all,
> > i read in documentation that max region count per region server should be
> > around 200. I unfortunately have currently 800 regions for RS.
> > Those tables are not presplitted (all handled by hbase), does manual
> > spliting helps in this case? hdfs fodler /hbase/data currently hold
> around
> > 2.2TB of data and i have 4 region servers, is it enough or i need more
> RS?
> >
> > Also will Hbase compact these tables automatically to have less regions?
> >
> > Hbase version 0.96.1.1-cdh5.0.1
> > Thanks for any guidance.
> >
> > JV
> >
>
>
>
> --
> Thanks,
> Michael Antonov
>
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message