hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Hbase region count and RS count for 2TB+
Date Fri, 15 Aug 2014 19:15:54 GMT
In that case, HBASE-7403 would come to the rescue.

Cheers


On Fri, Aug 15, 2014 at 12:09 PM, Andrey Stepachev <octo47@gmail.com> wrote:

> Here can be another problem - timestamp based keys or too many deletion.
> Compaction removes most and regions become nearly empty.
>
>   lars hofhansl <larsh@apache.org>
>  15 Aug 2014 22:12
> HBase initially tries to spread the load out to more region servers by
> splitting regions early when there aren't many (per table).
> As more regions are seen split size approaches the configured max file
> size.
> This early splitting turned out to be a bit too aggressive (see
> HBASE-10501, which modifies it a bit)... Even then 800 regions is a lot.
>
> As others have asked, what's
>
> hbase.hregion.max.filesize set to? Make sure it's at least 10gb.
>
>
> -- Lars
>
>
>
> ________________________________
> From: Juraj jiv <fatcap.jiv@gmail.com> <fatcap.jiv@gmail.com>
> To: user@hbase.apache.org
> Sent: Friday, August 15, 2014 2:08 AM
> Subject: Re: Hbase region count and RS count for 2TB+
>
>
> Hi,
> size 2.2TB i got from cmd "hdfs dfs -du -h /hbase/data"
>
> So i think yes, 650MB per RS. If its small then why hbase created so many
> regions?
>
> Thanks
> JV
>
>
>
> On Fri, Aug 15, 2014 at 9:51 AM, Mikhail Antonov <olorinbant@gmail.com>
> <olorinbant@gmail.com>
>   Juraj jiv <fatcap.jiv@gmail.com>
>  15 Aug 2014 13:08
> Hi,
> size 2.2TB i got from cmd "hdfs dfs -du -h /hbase/data"
>
> So i think yes, 650MB per RS. If its small then why hbase created so many
> regions?
>
> Thanks
> JV
>
>
> On Fri, Aug 15, 2014 at 9:51 AM, Mikhail Antonov <olorinbant@gmail.com>
> <olorinbant@gmail.com>
>
>   Mikhail Antonov <olorinbant@gmail.com>
>  15 Aug 2014 11:51
> So you have 3200 regions in total, and that means each region is about
> 650mb in size only (if you have 2.2 Tb in total), unless I'm missing
> something? If that's the case your regions are probably way too small.
>
> -Mikhail
>
>
>
>
>
>   Juraj jiv <fatcap.jiv@gmail.com>
>  15 Aug 2014 11:44
> Hi all,
> i read in documentation that max region count per region server should be
> around 200. I unfortunately have currently 800 regions for RS.
> Those tables are not presplitted (all handled by hbase), does manual
> spliting helps in this case? hdfs fodler /hbase/data currently hold around
> 2.2TB of data and i have 4 region servers, is it enough or i need more RS?
>
> Also will Hbase compact these tables automatically to have less regions?
>
> Hbase version 0.96.1.1-cdh5.0.1
> Thanks for any guidance.
>
> JV
>
>

Mime
  • Unnamed multipart/related (inline, None, 0 bytes)
View raw message