hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Daniel Cryans <jdcry...@apache.org>
Subject Re: OOME during frequent updates...
Date Wed, 09 Jun 2010 18:38:04 GMT
Yes, we are on 64-bits, it's 2010 right? :P

J-D

On Wed, Jun 9, 2010 at 11:34 AM, Vidhyashankar Venkataraman
<vidhyash@yahoo-inc.com> wrote:
>>> You wrote you have 24GB available, why only give 3 to HBase? You don't like him?
> Hehe..
> So, are you running 64-bit Java then? I thought java-32 doesn't allow more than around
3-4 gigs of RAM..
>
> On 6/9/10 11:26 AM, "Jean-Daniel Cryans" <jdcryans@apache.org> wrote:
>
> On Wed, Jun 9, 2010 at 11:16 AM, Vidhyashankar Venkataraman
> <vidhyash@yahoo-inc.com> wrote:
>> What do you mean by pastebinning it? I will try hosting it on a webserver..
>
> pastebin.com
>
>>
>> I know that OOME is Java running out of heap space: Can you let me know what are
the usual causes for OOME happening in Hbase? Was I pounding the servers a bit too hard with
updates?
>
> Well you know why it happens then ;)
>
> So, since you sent me the log in private until you get it somewhere
> public, I see this very important line:
>
> 2010-06-09 02:31:23,874 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Dump of metrics:
> request=0.0, regions=467, stores=933, storefiles=3272,
> storefileIndexSize=0, memstoreSize=1181, compactionQueueSize=59,
> usedHeap=2796, maxHeap=2999, blockCacheSize=681651688,
> blockCacheFree=261889816, blockCacheCount=7, blockCacheHitRatio=61,
> fsReadLatency=0, fsWriteLatency=0, fsSyncLatency=0
>
> This is outputted by the region servers when they shut down. As you
> can see, the used heap is 2796 over 2999, and then the OOME was
> triggered during:
>
> 2010-06-09 02:31:23,874 FATAL
> org.apache.hadoop.hbase.regionserver.HRegionServer: Set stop flag in
> regionserver/74.6.71.59:60020.compactor
> java.lang.OutOfMemoryError: Java heap space
>        at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:39)
>        at java.nio.ByteBuffer.allocate(ByteBuffer.java:312)
>        at org.apache.hadoop.hbase.io.hfile.HFile$Reader.decompress(HFile.java:1019)
>        at org.apache.hadoop.hbase.io.hfile.HFile$Reader.readBlock(HFile.java:971)
>        at org.apache.hadoop.hbase.io.hfile.HFile$Reader$Scanner.next(HFile.java:1163)
>        at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:58)
>        at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:79)
>        at org.apache.hadoop.hbase.regionserver.MinorCompactingStoreScanner.next(MinorCompactingStoreScanner.java:123)
>
> Since your max file size is 2GB, which btw is fairly big compared to
> your actual available heap, I'd say your settings are set way too high
> for what you gave to HBase to play with. You wrote you have 24GB
> available, why only give 3 to HBase? You don't like him?
>
> Jokes apart, HBase is a database, database needs memory, etc. We have
> the exact same HW here, and we give 8GB to HBase. Try also setting
> compressed object pointers on the JVM, passing it through hbase-env.sh
>
> J-D
>
>

Mime
View raw message