hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Daniel Cryans <jdcry...@apache.org>
Subject Re: too many regions cause OME ?
Date Tue, 12 Apr 2011 00:55:16 GMT
Ok that looks "fine", did the region server die under heavy load by
any chance? Or was it big scans? Or just normal load?

J-D

On Mon, Apr 11, 2011 at 5:50 PM, 陈加俊 <cjjvictory@gmail.com> wrote:
> my configuration is follows:
>  <property>
>                 <name>hbase.client.write.buffer</name>
>                 <value>2097152</value>
>                 <description>
>                         1024*1024*2=2097152
>                 </description>
>         </property>
>         <property>
>                 <name>hbase.hstore.blockingStoreFiles</name>
>                 <value>14</value>
>         </property>
>         <property>
>                 <name>hbase.hregion.memstore.block.multiplier</name>
>                 <value>2</value>
>         </property>
>         <property>
>                 <name>hbase.regionserver.handler.count</name>
>                 <value>30</value>
>         </property>
>  <property>
>                 <name>zookeeper.session.timeout</name>
>                 <value>90000</value>
>         </property>
>         <property>
>                  <name>hbase.zookeeper.property.tickTime</name>
>                  <value>9000</value>
>         </property>
>         <property>
>                  <name>hbase.hregion.max.filesize</name>
>                  <value>1073741824</value>
>                 <description>
>                 Maximum HStoreFile size. If any one of a column families'
> HStoreFiles has grown to exceed this value,
>                 the hosting HRegion is split in two. Default: 256M.
>                 1024*1024*1024=1073741824 (1G)
>                 </description>
>         </property>
>
> On Tue, Apr 12, 2011 at 8:43 AM, Jean-Daniel Cryans <jdcryans@apache.org>
> wrote:
>>
>> And where will they go? The issue isn't the number of regions per say,
>> it's the amount of data being served by that region server. Also I
>> still don't know if that's really your issue or it's a configuration
>> issue (which I have yet to see).
>>
>> J-D
>>
>> On Mon, Apr 11, 2011 at 5:41 PM, 陈加俊 <cjjvictory@gmail.com> wrote:
>> > Can I limit the numbers of regions  on one RegionServer ?
>> >
>> > On Tue, Apr 12, 2011 at 8:37 AM, Jean-Daniel Cryans
>> > <jdcryans@apache.org>wrote:
>> >
>> >> It's really a lot yes, but it could also be weird configurations or
>> >> too big values.
>> >>
>> >> J-D
>> >>
>> >> On Mon, Apr 11, 2011 at 5:35 PM, 陈加俊 <cjjvictory@gmail.com>
wrote:
>> >> > Is it too many regions ? Is the memory enough ?
>> >> > HBase-0.20.6
>> >> >
>> >> > 2011-04-12 00:16:31,844 FATAL
>> >> > org.apache.hadoop.hbase.regionserver.HRegionServer: OutOfMemoryError,
>> >> > aborting.
>> >> > java.lang.OutOfMemoryError: Java heap space
>> >> >        at
>> >> java.io.BufferedInputStream.<init>(BufferedInputStream.java:178)
>> >> >        at
>> >> >
>> >>
>> >> org.apache.hadoop.hdfs.DFSClient$BlockReader.newBlockReader(DFSClient.java:1380)
>> >> >        at
>> >> >
>> >>
>> >> org.apache.hadoop.hdfs.DFSClient$DFSInputStream.blockSeekTo(DFSClient.java:1648)
>> >> >        at
>> >> >
>> >> > org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:1767)
>> >> >        at java.io.DataInputStream.readFully(DataInputStream.java:178)
>> >> >        at java.io.DataInputStream.readFully(DataInputStream.java:152)
>> >> >        at
>> >> >
>> >>
>> >> org.apache.hadoop.hbase.io.hfile.HFile$FixedFileTrailer.deserialize(HFile.java:1368)
>> >> >        at
>> >> >
>> >> > org.apache.hadoop.hbase.io.hfile.HFile$Reader.readTrailer(HFile.java:848)
>> >> >        at
>> >> >
>> >>
>> >> org.apache.hadoop.hbase.io.hfile.HFile$Reader.loadFileInfo(HFile.java:793)
>> >> >        at
>> >> >
>> >> > org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:273)
>> >> >        at
>> >> >
>> >> > org.apache.hadoop.hbase.regionserver.StoreFile.<init>(StoreFile.java:129)
>> >> >        at
>> >> >
>> >> > org.apache.hadoop.hbase.regionserver.Store.loadStoreFiles(Store.java:410)
>> >> >        at
>> >> org.apache.hadoop.hbase.regionserver.Store.<init>(Store.java:221)
>> >> >        at
>> >> >
>> >>
>> >> org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:1636)
>> >> >        at
>> >> >
>> >> > org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:321)
>> >> >        at
>> >> >
>> >>
>> >> org.apache.hadoop.hbase.regionserver.HRegionServer.instantiateRegion(HRegionServer.java:1566)
>> >> >        at
>> >> >
>> >>
>> >> org.apache.hadoop.hbase.regionserver.HRegionServer.openRegion(HRegionServer.java:1533)
>> >> >        at
>> >> >
>> >>
>> >> org.apache.hadoop.hbase.regionserver.HRegionServer$Worker.run(HRegionServer.java:1453)
>> >> >        at java.lang.Thread.run(Thread.java:619)
>> >> > 2011-04-12 00:16:31,847 INFO
>> >> > org.apache.hadoop.hbase.regionserver.HRegionServer: Dump of metrics:
>> >> > request=0.0, regions=1624, stores=3075, storefiles=2526,
>> >> > storefileIndexSize=374, memstoreSize=339, compactionQueueSize=0,
>> >> > usedHeap=3990, maxHeap=3991, blockCacheSize=706107016,
>> >> > blockCacheFree=131010552, blockCacheCount=10631,
>> >> > blockCacheHitRatio=58,
>> >> > fsReadLatency=0, fsWriteLatency=0, fsSyncLatency=0
>> >> > 2011-04-12 00:16:31,848 INFO
>> >> > org.apache.hadoop.hbase.regionserver.HRegionServer: worker thread
>> >> > exiting
>> >> >
>> >> >
>> >> > --
>> >> > Thanks & Best regards
>> >> > jiajun
>> >> >
>> >>
>> >
>> >
>> >
>> > --
>> > Thanks & Best regards
>> > jiajun
>> >
>
>
>
> --
> Thanks & Best regards
> jiajun
>
>

Mime
View raw message