hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Hbase Unusable after auto split to 1024 regions
Date Thu, 06 Nov 2014 06:26:42 GMT
You can use ConstantSizeRegionSplitPolicy.
Split policy can be specified per table. See the following example
in create.rb :

  hbase> create 't1', 'f1', {NUMREGIONS => 15, SPLITALGO =>
'HexStringSplit'}

In 0.94.18, there isn't online merge. So you have to use other method to
merge the small regions.

Cheers

On Wed, Nov 5, 2014 at 10:14 PM, Pere Kyle <pere@whisper.sh> wrote:

> Ted,
>
> Thanks so much for that information. I now see why this split too often,
> but what I am not sure of is how to fix this without blowing away the
> cluster. Add more heap?
>
> Another symptom I have noticed is that load on the Master instance hbase
> daemon has been pretty high (load average 4.0, whereas it used to be 1.0)
>
> Thanks,
> Pere
>
> On Nov 5, 2014, at 9:56 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>
> > IncreasingToUpperBoundRegionSplitPolicy is the default split policy.
> >
> > You can read the javadoc of this class to see how it works.
> >
> > Cheers
> >
> > On Wed, Nov 5, 2014 at 9:39 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> >
> >> Can you provide a bit more information (such as HBase release) ?
> >>
> >> If you pastebin one of the region servers' log, that would help us
> >> determine the cause.
> >>
> >> Cheers
> >>
> >>
> >> On Wed, Nov 5, 2014 at 9:29 PM, Pere Kyle <pere@whisper.sh> wrote:
> >>
> >>> Hello,
> >>>
> >>> Recently our cluster which has been running fine for 2 weeks split to
> >>> 1024 regions at 1GB per region, after this split the cluster is
> unusable.
> >>> Using the performance benchmark I was getting a little better than 100
> w/s,
> >>> whereas before it was 5000 w/s. There are 15 nodes of m2.2xlarge with
> 8GB
> >>> heap reserved for Hbase
> >>>
> >>> Any Ideas? I am stumped:
> >>>
> >>> Thanks,
> >>> Pere
> >>>
> >>> Here is the current
> >>> hbase-site.xml
> >>> <?xml version="1.0"?>
> >>> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
> >>> <configuration>
> >>> <property>
> >>>    <name>hbase.snapshot.enabled</name>
> >>>    <value>true</value>
> >>>  </property>
> >>>
> >>>
> <property><name>fs.hdfs.impl</name><value>emr.hbase.fs.BlockableFileSystem</value></property>
> >>>
> >>>
> <property><name>hbase.regionserver.handler.count</name><value>50</value></property>
> >>>
> >>>
> <property><name>hbase.cluster.distributed</name><value>true</value></property>
> >>>
> >>>
> <property><name>hbase.tmp.dir</name><value>/mnt/var/lib/hbase/tmp-data</value></property>
> >>>
> >>>
> <property><name>hbase.master.wait.for.log.splitting</name><value>true</value></property>
> >>>
> >>>
> <property><name>hbase.hregion.memstore.flush.size</name><value>134217728</value></property>
> >>>  <property><name>hbase.hregion.max.filesize</name><value>5073741824
> >>> </value></property>
> >>>
> >>>
> <property><name>zookeeper.session.timeout</name><value>60000</value></property>
> >>>
> >>>
> <property><name>hbase.thrift.maxQueuedRequests</name><value>0</value></property>
> >>>
> >>>
> <property><name>hbase.client.scanner.caching</name><value>1000</value></property>
> >>>
> >>>
> <property><name>hbase.hregion.memstore.block.multiplier</name><value>4</value></property>
> >>> </configuration>
> >>>
> >>> hbase-env.sh
> >>> # The maximum amount of heap to use, in MB. Default is 1000.
> >>> export HBASE_HEAPSIZE=8000
> >>>
> >>> # Extra Java runtime options.
> >>> # Below are what we set by default.  May only work with SUN JVM.
> >>> # For more on why as well as other possible settings,
> >>> # see http://wiki.apache.org/hadoop/PerformanceTuning
> >>> export HBASE_OPTS="-XX:+UseConcMarkSweepGC”
> >>>
> >>> hbase-env.sh
> >>
> >>
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message