hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pere Kyle <p...@whisper.sh>
Subject Re: Hbase Unusable after auto split to 1024 regions
Date Thu, 06 Nov 2014 06:03:57 GMT
Here is a paste from one of the region servers
http://pastebin.com/H3BaHdPq

I am running Hbase on EMR

HBase Version	0.94.18, re17f91a1f107923d2defc7f18dbca59983f0a69f

Thanks,
Pere

On Nov 5, 2014, at 9:39 PM, Ted Yu <yuzhihong@gmail.com> wrote:

> Can you provide a bit more information (such as HBase release) ?
> 
> If you pastebin one of the region servers' log, that would help us
> determine the cause.
> 
> Cheers
> 
> 
> On Wed, Nov 5, 2014 at 9:29 PM, Pere Kyle <pere@whisper.sh> wrote:
> 
>> Hello,
>> 
>> Recently our cluster which has been running fine for 2 weeks split to 1024
>> regions at 1GB per region, after this split the cluster is unusable. Using
>> the performance benchmark I was getting a little better than 100 w/s,
>> whereas before it was 5000 w/s. There are 15 nodes of m2.2xlarge with 8GB
>> heap reserved for Hbase
>> 
>> Any Ideas? I am stumped:
>> 
>> Thanks,
>> Pere
>> 
>> Here is the current
>> hbase-site.xml
>> <?xml version="1.0"?>
>> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
>> <configuration>
>> <property>
>>    <name>hbase.snapshot.enabled</name>
>>    <value>true</value>
>>  </property>
>> 
>> <property><name>fs.hdfs.impl</name><value>emr.hbase.fs.BlockableFileSystem</value></property>
>> 
>> <property><name>hbase.regionserver.handler.count</name><value>50</value></property>
>> 
>> <property><name>hbase.cluster.distributed</name><value>true</value></property>
>> 
>> <property><name>hbase.tmp.dir</name><value>/mnt/var/lib/hbase/tmp-data</value></property>
>> 
>> <property><name>hbase.master.wait.for.log.splitting</name><value>true</value></property>
>> 
>> <property><name>hbase.hregion.memstore.flush.size</name><value>134217728</value></property>
>> 
>> <property><name>hbase.hregion.max.filesize</name><value>5073741824</value></property>
>> 
>> <property><name>zookeeper.session.timeout</name><value>60000</value></property>
>> 
>> <property><name>hbase.thrift.maxQueuedRequests</name><value>0</value></property>
>> 
>> <property><name>hbase.client.scanner.caching</name><value>1000</value></property>
>> 
>> <property><name>hbase.hregion.memstore.block.multiplier</name><value>4</value></property>
>> </configuration>
>> 
>> hbase-env.sh
>> # The maximum amount of heap to use, in MB. Default is 1000.
>> export HBASE_HEAPSIZE=8000
>> 
>> # Extra Java runtime options.
>> # Below are what we set by default.  May only work with SUN JVM.
>> # For more on why as well as other possible settings,
>> # see http://wiki.apache.org/hadoop/PerformanceTuning
>> export HBASE_OPTS="-XX:+UseConcMarkSweepGC”
>> 
>> hbase-env.sh


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message